diff --git a/dev/404.html b/dev/404.html index 94f50d497c..6af885a56f 100644 --- a/dev/404.html +++ b/dev/404.html @@ -5,10 +5,10 @@ 404 | Lux.jl Docs - - + + - + @@ -25,7 +25,7 @@
- + \ No newline at end of file diff --git a/dev/api/Accelerator_Support/MLDataDevices.html b/dev/api/Accelerator_Support/MLDataDevices.html index e4495a55bb..5637cb517b 100644 --- a/dev/api/Accelerator_Support/MLDataDevices.html +++ b/dev/api/Accelerator_Support/MLDataDevices.html @@ -5,12 +5,12 @@ MLDataDevices | Lux.jl Docs - - + + - + - + @@ -52,7 +52,7 @@ (i, summary(x)) = (1, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}") (i, summary(x)) = (2, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}") (i, summary(x)) = (3, "3×7 CuArray{Float32, 2, CUDA.DeviceMemory}")

source

- + \ No newline at end of file diff --git a/dev/api/Building_Blocks/LuxCore.html b/dev/api/Building_Blocks/LuxCore.html index 976877960c..bac2728ac3 100644 --- a/dev/api/Building_Blocks/LuxCore.html +++ b/dev/api/Building_Blocks/LuxCore.html @@ -5,12 +5,12 @@ LuxCore | Lux.jl Docs - - + + - + - + @@ -28,7 +28,7 @@
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

LuxCore ​

LuxCore.jl defines the abstract layers for Lux. Allows users to be compatible with the entirely of Lux.jl without having such a heavy dependency. If you are depending on Lux.jl directly, you do not need to depend on LuxCore.jl (all the functionality is exported via Lux.jl).

Abstract Types ​

LuxCore.AbstractLuxLayer Type
julia
abstract type AbstractLuxLayer

Abstract Type for all Lux Layers

Users implementing their custom layer, must implement

  • initialparameters(rng::AbstractRNG, layer::CustomAbstractLuxLayer) – This returns a NamedTuple containing the trainable parameters for the layer.

  • initialstates(rng::AbstractRNG, layer::CustomAbstractLuxLayer) – This returns a NamedTuple containing the current state for the layer. For most layers this is typically empty. Layers that would potentially contain this include BatchNorm, LSTM, GRU, etc.

Optionally:

  • parameterlength(layer::CustomAbstractLuxLayer) – These can be automatically calculated, but it is recommended that the user defines these.

  • statelength(layer::CustomAbstractLuxLayer) – These can be automatically calculated, but it is recommended that the user defines these.

See also AbstractLuxContainerLayer

source

LuxCore.AbstractLuxWrapperLayer Type
julia
abstract type AbstractLuxWrapperLayer{layer} <: AbstractLuxLayer

See AbstractLuxContainerLayer for detailed documentation. This abstract type is very similar to AbstractLuxContainerLayer except that it allows for a single layer to be wrapped in a container.

Additionally, on calling initialparameters and initialstates, the parameters and states are not wrapped in a NamedTuple with the same name as the field.

As a convenience, we define the fallback call (::AbstractLuxWrapperLayer)(x, ps, st), which calls getfield(x, layer)(x, ps, st).

source

LuxCore.AbstractLuxContainerLayer Type
julia
abstract type AbstractLuxContainerLayer{layers} <: AbstractLuxLayer

Abstract Container Type for certain Lux Layers. layers is a tuple containing fieldnames for the layer, and constructs the parameters and states using those.

Users implementing their custom layer can extend the same functions as in AbstractLuxLayer.

Advanced Structure Manipulation

Advanced structure manipulation of these layers post construction is possible via Functors.fmap. For a more flexible interface, we recommend using Lux.Experimental.@layer_map.

fmap Support

fmap support needs to be explicitly enabled by loading Functors.jl and Setfield.jl.

Changes from Pre-1.0 Behavior

Previously if layers was a singleton tuple, initialparameters and initialstates would return the parameters and states for the single field layers. From v1.0.0 onwards, even for singleton tuples, the parameters/states are wrapped in a NamedTuple with the same name as the field. See AbstractLuxWrapperLayer to replicate the previous behavior of singleton tuples.

source

General ​

LuxCore.apply Function
julia
apply(model, x, ps, st)

In most cases this function simply calls model(x, ps, st). However, it is still recommended to call apply instead of model(x, ps, st) directly. Some of the reasons for this include:

  1. For certain types of inputs x, we might want to perform preprocessing before calling model. For eg, if x is an Array of ReverseDiff.TrackedReals this can cause significant regressions in model(x, ps, st) (since it won't hit any of the BLAS dispatches). In those cases, we would automatically convert x to a ReverseDiff.TrackedArray.

  2. Certain user defined inputs need to be applied to specific layers but we want the datatype of propagate through all the layers (even unsupported ones). In these cases, we can unpack the input in apply and pass it to the appropriate layer and then repack it before returning. See the Lux manual on Custom Input Types for a motivating example.

Tip

apply is integrated with DispatchDoctor.jl that allows automatic verification of type stability. By default this is "disable"d. For more information, see the documentation.

source

LuxCore.stateless_apply Function
julia
stateless_apply(model, x, ps)

Calls apply and only returns the first argument. This function requires that model has an empty state of NamedTuple(). Behavior of other kinds of models are undefined and it is the responsibility of the user to ensure that the model has an empty state.

source

LuxCore.check_fmap_condition Function
julia
check_fmap_condition(cond, tmatch::Union{Type, Nothing}, x) -> Bool

fmaps into the structure x and see if cond is satisfied for any of the leaf elements.

Arguments

  • cond - A function that takes a single argument and returns a Bool.

  • tmatch - A shortcut to check if x is of type tmatch. Can be disabled by passing nothing.

  • x - The structure to check.

Returns

A Boolean Value

source

LuxCore.contains_lux_layer Function
julia
contains_lux_layer(l) -> Bool

Check if the structure l is a Lux AbstractLuxLayer or a container of such a layer.

source

LuxCore.display_name Function
julia
display_name(layer::AbstractLuxLayer)

Printed Name of the layer. If the layer has a field name that is used, else the type name is used.

source

LuxCore.replicate Function
julia
replicate(rng::AbstractRNG)

Creates a copy of the rng state depending on its type.

source

LuxCore.setup Function
julia
setup(rng::AbstractRNG, layer)

Shorthand for getting the parameters and states of the layer l. Is equivalent to (initialparameters(rng, l), initialstates(rng, l)).

Warning

This function is not pure, it mutates rng.

source

Parameters ​

LuxCore.initialparameters Function
julia
initialparameters(rng::AbstractRNG, layer)

Generate the initial parameters of the layer l.

source

LuxCore.parameterlength Function
julia
parameterlength(layer)

Return the total number of parameters of the layer l.

source

States ​

LuxCore.initialstates Function
julia
initialstates(rng::AbstractRNG, layer)

Generate the initial states of the layer l.

source

LuxCore.statelength Function
julia
statelength(layer)

Return the total number of states of the layer l.

source

LuxCore.testmode Function
julia
testmode(st::NamedTuple)

Make all occurrences of training in state st – Val(false).

source

LuxCore.trainmode Function
julia
trainmode(st::NamedTuple)

Make all occurrences of training in state st – Val(true).

source

LuxCore.update_state Function
julia
update_state(st::NamedTuple, key::Symbol, value; exclude=Internal.isleaf)

Recursively update all occurrences of the key in the state st with the value. exclude is a function that is passed to Functors.fmap_with_path's exclude keyword.

Needs Functors.jl

This function requires Functors.jl to be loaded.

source

Layer size ​

LuxCore.outputsize Function
julia
outputsize(layer, x, rng)

Return the output size of the layer.

The fallback implementation of this function assumes the inputs were batched, i.e., if any of the outputs are Arrays, with ndims(A) > 1, it will return size(A)[1:(end - 1)]. If this behavior is undesirable, provide a custom outputsize(layer, x, rng) implementation).

Fallback Implementation

The fallback implementation of this function is defined once Lux.jl is loaded.

Changes from Pre-1.0 Behavior

Previously it was possible to override this function by defining outputsize(layer). However, this can potentially introduce a bug that is hard to bypass. See this PR for more information.

source

- + \ No newline at end of file diff --git a/dev/api/Building_Blocks/LuxLib.html b/dev/api/Building_Blocks/LuxLib.html index 3152c412cc..c6569998af 100644 --- a/dev/api/Building_Blocks/LuxLib.html +++ b/dev/api/Building_Blocks/LuxLib.html @@ -5,12 +5,12 @@ LuxLib | Lux.jl Docs - - + + - + - + @@ -40,7 +40,7 @@ epsilon = eps(eltype(x)) ^ (5 // 7))

Instance Normalization. For details see [1].

Instance Normalization computes the mean and variance for each D1×...×DN−2×1×1 input slice and normalises the input accordingly.

Arguments

Returns

Normalized Array of same size as x. And a Named Tuple containing the updated running mean and variance.

References

[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

source

LuxLib.API.layernorm Function
julia
layernorm(x::AbstractArray{xT, N}, scale, bias, σ = identity, dims=1:(N - 1),
     epsilon = eps(eltype(x)) ^ (5 / 7)) where {xT, N}

Layer Normalization. For details see [1].

Given an input array x, this layer computes

y=x−E[x]Var[x]+ϵ∗γ+β

and applies the activation function σ elementwise to y.

Arguments

Returns

Normalized Array of same size as x.

References

[1] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).

source

Helper Functions ​

LuxLib.internal_operation_mode Function
julia
internal_operation_mode(xs::Tuple)
 internal_operation_mode(x::AbstractArray)

Returns the internal operation mode for the given array(s). This is useful to define custom implementations using different backends like simple Julia broadcasting, Kernel Abstractions, Loop Vectorization, etc.

Currently supported modes are:

source

- + \ No newline at end of file diff --git a/dev/api/Building_Blocks/WeightInitializers.html b/dev/api/Building_Blocks/WeightInitializers.html index 5e6c06e83a..75bd747026 100644 --- a/dev/api/Building_Blocks/WeightInitializers.html +++ b/dev/api/Building_Blocks/WeightInitializers.html @@ -5,12 +5,12 @@ WeightInitializers | Lux.jl Docs - - + + - + - + @@ -79,7 +79,7 @@ kwargs...) -> AbstractArray{ComplexF64, length(size)}

Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of ones.

source

WeightInitializers.randC64 Function
julia
randC64([::AbstractRNG=Utils.default_rng()], size...;
     kwargs...) -> AbstractArray{ComplexF64, length(size)}

Return an AbstractArray{ComplexF64} of the given size containing random numbers from a uniform distribution.

source

WeightInitializers.randnC64 Function
julia
randnC64([::AbstractRNG=Utils.default_rng()], size...;
     kwargs...) -> AbstractArray{ComplexF64, length(size)}

Return an AbstractArray{ComplexF64} of the given size containing random numbers from a standard normal distribution.

source

- + \ No newline at end of file diff --git a/dev/api/Lux/autodiff.html b/dev/api/Lux/autodiff.html index 921d2c2ecd..a4fe73b33f 100644 --- a/dev/api/Lux/autodiff.html +++ b/dev/api/Lux/autodiff.html @@ -5,14 +5,14 @@ Automatic Differentiation Helpers | Lux.jl Docs - - + + - + - + - + @@ -27,8 +27,8 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Automatic Differentiation Helpers ​

JVP & VJP Wrappers ​

Lux.jacobian_vector_product Function
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)

Compute the Jacobian-Vector Product (∂f∂x)u. This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the JVP.

  • x: The input to the function.

  • u: An object of the same structure as x.

Returns

  • v: The Jacobian Vector Product.

source

Lux.vector_jacobian_product Function
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)

Compute the Vector-Jacobian Product (∂f∂x)Tu. This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the VJP.

  • x: The input to the function.

  • u: An object of the same structure as f(x).

Returns

  • v: The Vector Jacobian Product.

source

Batched AD ​

Lux.batched_jacobian Function
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the jacobian.

  • x: The input to the function. Must have ndims(x) ≥ 2.

Returns

  • J: The Jacobian of f with respect to x. This will be a 3D Array. If the dimensions of x are (N₁, N₂, ..., Nₙ, B) and of y are (M₁, M₂, ..., Mₘ, B), then J will be a ((M₁ × M₂ × ... × Mₘ), (N₁ × N₂ × ... × Nₙ), B) Array.

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

Nested 2nd Order AD ​

Consult the manual page on Nested AD for information on nested automatic differentiation.

- +
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Automatic Differentiation Helpers ​

JVP & VJP Wrappers ​

Lux.jacobian_vector_product Function
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)

Compute the Jacobian-Vector Product (∂f∂x)u. This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the JVP.

  • x: The input to the function.

  • u: An object of the same structure as x.

Returns

  • v: The Jacobian Vector Product.

source

Lux.vector_jacobian_product Function
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)

Compute the Vector-Jacobian Product (∂f∂x)Tu. This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the VJP.

  • x: The input to the function.

  • u: An object of the same structure as f(x).

Returns

  • v: The Vector Jacobian Product.

source

Batched AD ​

Lux.batched_jacobian Function
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the jacobian.

  • x: The input to the function. Must have ndims(x) ≥ 2.

Returns

  • J: The Jacobian of f with respect to x. This will be a 3D Array. If the dimensions of x are (N₁, N₂, ..., Nₙ, B) and of y are (M₁, M₂, ..., Mₘ, B), then J will be a ((M₁ × M₂ × ... × Mₘ), (N₁ × N₂ × ... × Nₙ), B) Array.

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

Nested 2nd Order AD ​

Consult the manual page on Nested AD for information on nested automatic differentiation.

+ \ No newline at end of file diff --git a/dev/api/Lux/contrib.html b/dev/api/Lux/contrib.html index be4c49eb56..2a46c38413 100644 --- a/dev/api/Lux/contrib.html +++ b/dev/api/Lux/contrib.html @@ -5,14 +5,14 @@ Experimental Features | Lux.jl Docs - - + + - + - + - + @@ -28,8 +28,8 @@
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Experimental Features ​

All features listed on this page are experimental which means:

  1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.

  2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.

  3. None of the features are exported.

Parameter Freezing ​

Lux.Experimental.FrozenLayer Type
julia
FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})

Freeze the parameters with name which_params of the layer l.

Use Lux.Experimental.freeze instead

It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.

No checks for which_params

There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.

Arguments

  • l: Lux AbstractLuxLayer.

  • which_params: Parameter Names to be Frozen. Can be set to nothing, in which case all parameters are frozen.

Extended Help

Parameters

  • Parameters of the layer l excluding which_params.

States

  • frozen_params: Parameters that are frozen, i.e., which_params.

  • states: The state of the inner layer l.

Note on Internal Layer Implementation

The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.

Example

julia
julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
-FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

source

Lux.Experimental.freeze Function
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

source

julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
-    which_params::Optional{Tuple} = nothing)

Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

source

Lux.Experimental.unfreeze Function
julia
unfreeze(l::FrozenLayer)

Unfreezes the layer l.

source

julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)

Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

source

For detailed usage example look at the manual page.

Map over Layer ​

Lux.Experimental.layer_map Function
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

KeyPath provided to the function

The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

Call Signature for f

  • Must take 4 inputs – AbstractLuxLayer, Corresponding Parameters, Corresponding States, and the Functors.KeyPath to the layer.

  • Must return a tuple of 3 elements – AbstractLuxLayer, new parameters and the new states.

Extended Help

Example

julia
julia> using Lux, Random
+FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

source

Lux.Experimental.freeze Function
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

source

julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
+    which_params::Optional{Tuple} = nothing)

Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

source

Lux.Experimental.unfreeze Function
julia
unfreeze(l::FrozenLayer)

Unfreezes the layer l.

source

julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)

Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

source

For detailed usage example look at the manual page.

Map over Layer ​

Lux.Experimental.layer_map Function
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

KeyPath provided to the function

The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

Call Signature for f

  • Must take 4 inputs – AbstractLuxLayer, Corresponding Parameters, Corresponding States, and the Functors.KeyPath to the layer.

  • Must return a tuple of 3 elements – AbstractLuxLayer, new parameters and the new states.

Extended Help

Example

julia
julia> using Lux, Random
 
 julia> c = Parallel(
            +; chain=Chain(; dense_1=Dense(2 => 3), bn=BatchNorm(3), dense_2=Dense(3 => 5)),
@@ -56,10 +56,10 @@
 julia> all(iszero, (ps_new.chain.dense_1.weight, ps_new.chain.dense_1.bias,
                     ps_new.chain.dense_2.weight, ps_new.chain.dense_2.bias,
                     ps_new.dense_3.weight, ps_new.dense_3.bias))
-true

source

Debugging Functionality ​

Model not working properly! Here are some functionalities to help you debug you Lux model.

Lux.Experimental.@debug_mode Macro
julia
@debug_mode layer kwargs...

Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

source

Lux.Experimental.DebugLayer Type
julia
DebugLayer(layer::AbstractLuxLayer;
+true

source

Debugging Functionality ​

Model not working properly! Here are some functionalities to help you debug you Lux model.

Lux.Experimental.@debug_mode Macro
julia
@debug_mode layer kwargs...

Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

source

Lux.Experimental.DebugLayer Type
julia
DebugLayer(layer::AbstractLuxLayer;
     nan_check::Union{Symbol, StaticSymbol, Val}=static(:both),
     error_check::Union{StaticBool, Bool, Val{true}, Val{false}}=True(),
-    location::KeyPath=KeyPath())

A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

Arguments

  • layer: The layer to be wrapped.

Extended Help

Keyword Arguments

  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.

  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.

  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.

Input / Output

Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

ChainRules Compatible Reverse Mode AD Tools

nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

Disable After Debugging

This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

See Lux.Experimental.@debug_mode to construct this layer.

source

Tied Parameters ​

Lux.Experimental.share_parameters Function
julia
share_parameters(ps, sharing)
+    location::KeyPath=KeyPath())

A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

Arguments

  • layer: The layer to be wrapped.

Extended Help

Keyword Arguments

  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.

  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.

  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.

Input / Output

Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

ChainRules Compatible Reverse Mode AD Tools

nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

Disable After Debugging

This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

See Lux.Experimental.@debug_mode to construct this layer.

source

Tied Parameters ​

Lux.Experimental.share_parameters Function
julia
share_parameters(ps, sharing)
 share_parameters(ps, sharing, new_parameters)

Updates the parameters in ps with a common set of parameters new_parameters that are shared between each list in the nested list sharing. (That was kind of a mouthful, the example should make it clear).

Arguments

  • ps: Original parameters.

  • sharing: A nested list of lists of accessors of ps which need to shate the parameters (See the example for details). (Each list in the list must be disjoint)

  • new_parameters: If passed the length of new_parameters must be equal to the length of sharing. For each vector in sharing the corresponding parameter in new_parameters will be used. (If not passed, the parameters corresponding to the first element of each vector in sharing will be used).

Returns

Updated Parameters having the same structure as ps.

Example

julia
julia> model = Chain(; d1=Dense(2 => 4, tanh),
            d3=Chain(; l1=Dense(4 => 2), l2=Dense(2 => 4)), d2=Dense(4 => 2))
 Chain(
@@ -81,8 +81,8 @@
            ps.d3.l2.bias === ps.d1.bias &&
            ps.d2.weight === ps.d3.l1.weight &&
            ps.d2.bias === ps.d3.l1.bias
-true

ComponentArrays

ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

source

- +true

ComponentArrays

ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

source

+ \ No newline at end of file diff --git a/dev/api/Lux/distributed_utils.html b/dev/api/Lux/distributed_utils.html index fe2ffdffe2..deef76cfd4 100644 --- a/dev/api/Lux/distributed_utils.html +++ b/dev/api/Lux/distributed_utils.html @@ -5,14 +5,14 @@ Distributed Utils | Lux.jl Docs - - + + - + - + - + @@ -27,11 +27,11 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

Lux.MPIBackend Type
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

Lux.NCCLBackend Type
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

Initialization ​

Lux.DistributedUtils.initialize Function
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

source

Lux.DistributedUtils.initialized Function
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

Lux.DistributedUtils.get_distributed_backend Function
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

Helper Functions ​

Lux.DistributedUtils.local_rank Function
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

Lux.DistributedUtils.total_workers Function
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

Communication Primitives ​

Lux.DistributedUtils.allreduce! Function
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
-allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.bcast! Function
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
-bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

Lux.DistributedUtils.reduce! Function
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
-reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.synchronize!! Function
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

Optimizers.jl Integration ​

Lux.DistributedUtils.DistributedOptimizer Type
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

  • optimizer: An Optimizer compatible with the Optimisers.jl package

source

MLUtils.jl Integration ​

Lux.DistributedUtils.DistributedDataContainer Type
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

- +
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

Lux.MPIBackend Type
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

Lux.NCCLBackend Type
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

Initialization ​

Lux.DistributedUtils.initialize Function
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

source

Lux.DistributedUtils.initialized Function
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

Lux.DistributedUtils.get_distributed_backend Function
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

Helper Functions ​

Lux.DistributedUtils.local_rank Function
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

Lux.DistributedUtils.total_workers Function
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

Communication Primitives ​

Lux.DistributedUtils.allreduce! Function
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
+allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.bcast! Function
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
+bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

Lux.DistributedUtils.reduce! Function
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
+reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.synchronize!! Function
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

Optimizers.jl Integration ​

Lux.DistributedUtils.DistributedOptimizer Type
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

  • optimizer: An Optimizer compatible with the Optimisers.jl package

source

MLUtils.jl Integration ​

Lux.DistributedUtils.DistributedDataContainer Type
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

+ \ No newline at end of file diff --git a/dev/api/Lux/interop.html b/dev/api/Lux/interop.html index d7887a472d..b08679ff3e 100644 --- a/dev/api/Lux/interop.html +++ b/dev/api/Lux/interop.html @@ -5,14 +5,14 @@ Interoperability between Lux and other packages | Lux.jl Docs - - + + - + - + - + @@ -27,7 +27,7 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Interoperability between Lux and other packages ​

Switching from older frameworks ​

Flux Models to Lux Models ​

Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.

Tip

Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.

Adapt.adapt Method
julia
Adapt.adapt(from::FromFluxAdaptor, L)

Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.

source

Lux.FromFluxAdaptor Type
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)

Convert a Flux Model to Lux Model.

active field

This always ignores the active field of some of the Flux layers. This is almost never going to be supported.

Keyword Arguments

  • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.

  • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.

Example

julia
julia> import Flux
+    
Lux.jl Docs
Appearance

 Trusted by 
Scientific Computing
SciML.ai
Machine Learning
Interoperability between Lux and other packages ​
Switching from older frameworks ​
Flux Models to Lux Models ​
Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.
Tip
Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.
Adapt.adapt Method
julia
Adapt.adapt(from::FromFluxAdaptor, L)
Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.
source
Lux.FromFluxAdaptor Type
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)
Convert a Flux Model to Lux Model.
active field
This always ignores the active field of some of the Flux layers. This is almost never going to be supported.
Keyword Arguments
  • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.
  • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.
Example
julia
julia> import Flux
 
 julia> using Adapt, Lux, Random
 
@@ -40,7 +40,7 @@
 julia> ps, st = Lux.setup(Random.default_rng(), m2);
 
 julia> size(first(m2(x, ps, st)))
-(2, 32)
source
Lux.FluxLayer Type
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
Adapt.adapt Method
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
Lux.ToSimpleChainsAdaptor Type
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
+(2, 32)
source
Lux.FluxLayer Type
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
Adapt.adapt Method
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
Lux.ToSimpleChainsAdaptor Type
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
 
 julia> using Adapt, Lux, Random
 
@@ -57,9 +57,9 @@
 julia> x = randn(Float32, 28, 28, 1, 1);
 
 julia> size(first(simple_chains_model(x, ps, st)))
-(10, 1)
source
Lux.SimpleChainsLayer Type
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
-SimpleChainsLayer(layer, lux_layer, to_array)
Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.
ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.
Arguments
  • layer: SimpleChains layer
  • lux_layer: Potentially equivalent Lux layer that is used for printing
source

-    
+(10, 1)

source

Lux.SimpleChainsLayer Type
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
+SimpleChainsLayer(layer, lux_layer, to_array)

Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.

ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.

Arguments

  • layer: SimpleChains layer

  • lux_layer: Potentially equivalent Lux layer that is used for printing

source

+ \ No newline at end of file diff --git a/dev/api/Lux/layers.html b/dev/api/Lux/layers.html index 3b7e8f4b19..697afdebbc 100644 --- a/dev/api/Lux/layers.html +++ b/dev/api/Lux/layers.html @@ -5,14 +5,14 @@ Built-In Layers | Lux.jl Docs - - + + - + - + - + @@ -34,7 +34,7 @@ layer_2 = NoOpLayer(), layer_3 = NoOpLayer(), ) # Total: 0 parameters, - # plus 0 states.

source

Lux.Chain Type
julia
Chain(layers...; name=nothing)
+          #        plus 0 states.

source

Lux.Chain Type
julia
Chain(layers...; name=nothing)
 Chain(; layers..., name=nothing)

Collects multiple layers / functions to be called in sequence on a given input.

Arguments

Extended Help

Inputs

Input x is passed sequentially to each layer, and must conform to the input requirements of the internal layers.

Returns

Parameters

States

Miscellaneous Properties

Example

julia
julia> Chain(Dense(2, 3, relu), BatchNorm(3), Dense(3, 2))
 Chain(
     layer_1 = Dense(2 => 3, relu),      # 9 parameters
@@ -49,7 +49,7 @@
     layer_2 = BatchNorm(3, affine=true, track_stats=true),  # 6 parameters, plus 7
     layer_3 = Dense(3 => 2),            # 8 parameters
 )         # Total: 23 parameters,
-          #        plus 7 states.

source

Lux.PairwiseFusion Type
julia
PairwiseFusion(connection, layers...; name=nothing)
+          #        plus 7 states.

source

Lux.PairwiseFusion Type
julia
PairwiseFusion(connection, layers...; name=nothing)
 PairwiseFusion(connection; name=nothing, layers...)
 PairwiseFusion(; connection, layers..., name=nothing)
x1 → layer1 → y1 ↘
                   connection → layer2 → y2 ↘
@@ -60,7 +60,7 @@
 end
  1. Any other kind of input
julia
y = x
 for i in 1:N
     y = connection(x, layers[i](y))
-end

Returns

Parameters

States

source

Lux.Parallel Type
julia
Parallel(connection, layers...; name=nothing)
+end

Returns

Parameters

States

source

Lux.Parallel Type
julia
Parallel(connection, layers...; name=nothing)
 Parallel(connection; name=nothing, layers...)
 Parallel(; connection, layers..., name=nothing)

Create a layer which passes an input to each path in layers, before reducing the output with connection.

Arguments

Extended Help

Inputs

Returns

Parameters

States

See also SkipConnection which is Parallel with one identity.

Example

julia
julia> model = Parallel(nothing, Dense(2, 1), Dense(2, 1))
 Parallel(
@@ -76,37 +76,37 @@
        x2 = randn(rng, Float32, 2);
 
 julia> size.(first(model((x1, x2), ps, st)))
-((1,), (1,))

source

Lux.SkipConnection Type
julia
SkipConnection(layers, connection; name=nothing)
-SkipConnection(; layers, connection, name=nothing)

Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

The simplest "ResNet"-type connection is just SkipConnection(layer, +).

Arguments

Extended Help

Inputs

Returns

Parameters

States

See Parallel for a more general implementation.

source

Lux.RepeatedLayer Type
julia
RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

julia
res = x
+((1,), (1,))

source

Lux.SkipConnection Type
julia
SkipConnection(layers, connection; name=nothing)
+SkipConnection(; layers, connection, name=nothing)

Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

The simplest "ResNet"-type connection is just SkipConnection(layer, +).

Arguments

Extended Help

Inputs

Returns

Parameters

States

See Parallel for a more general implementation.

source

Lux.RepeatedLayer Type
julia
RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

julia
res = x
 for i in 1:repeats
     res, st = model(res, ps, st)
 end
julia
res = x
 for i in 1:repeats
     res, st = model((res, x), ps, st)
-end

It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Convolutional Layers ​

Lux.Conv Type
julia
Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+end

It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Convolutional Layers ​

Lux.Conv Type
julia
Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
      activation=identity; init_weight=nothing, init_bias=nothing, stride=1,
-     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

Standard convolutional layer.

Conv 2D

Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

Warning

Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

Arguments

Extended Help

Keyword Arguments

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

Parameters

source

Lux.ConvTranspose Type
julia
ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

Standard convolutional layer.

Conv 2D

Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

Warning

Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

Arguments

Extended Help

Keyword Arguments

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

Parameters

source

Lux.ConvTranspose Type
julia
ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
               activation=identity; init_weight=glorot_uniform, init_bias=zeros32,
               stride=1, pad=0, outpad=0, dilation=1, groups=1, use_bias=True(),
-              cross_correlation=False())

Standard convolutional transpose layer.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Dropout Layers ​

Lux.AlphaDropout Type
julia
AlphaDropout(p::Real)

AlphaDropout layer.

Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also Dropout, VariationalHiddenDropout

source

Lux.Dropout Type
julia
Dropout(p; dims=:)

Dropout layer.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, VariationalHiddenDropout

source

Lux.VariationalHiddenDropout Type
julia
VariationalHiddenDropout(p; dims=:)

VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, Dropout

source

Pooling Layers ​

Lux.AdaptiveLPPool Type
julia
AdaptiveLPPool(output_size; p=2)

Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.AdaptiveMaxPool Type
julia
AdaptiveMaxPool(output_size)

Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.AdaptiveMeanPool Type
julia
AdaptiveMeanPool(output_size)

Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.GlobalLPPool Type
julia
GlobalLPPool(; p=2)

Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.GlobalMaxPool Type
julia
GlobalMaxPool()

Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.GlobalMeanPool Type
julia
GlobalMeanPool()

Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.LPPool Type
julia
LPPool(window; stride=window, pad=0, dilation=1, p=2)

LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

Arguments

Keyword Arguments

GPU Support

This layer is currently only supported on CPU.

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MaxPool Type
julia
MaxPool(window; stride=window, pad=0, dilation=1)

Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MeanPool Type
julia
MeanPool(window; stride=window, pad=0, dilation=1)

Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Recurrent Layers ​

Lux.GRUCell Type
julia
GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
-        init_weight=nothing, init_bias=nothing, init_state=zeros32)

Gated Recurrent Unit (GRU) Cell

r=σ(Wir×x+bir+Whr×hprev+bhr)z=σ(Wiz×x+biz+Whz×hprev+bhz)n=tanh⁡(Win×x+bin+r⋅(Whn×hprev+bhn))hnew=(1−z)⋅n+z⋅hprev

Arguments

Inputs

Returns

Parameters

States

source

Lux.LSTMCell Type
julia
LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
+              cross_correlation=False())

Standard convolutional transpose layer.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Dropout Layers ​

Lux.AlphaDropout Type
julia
AlphaDropout(p::Real)

AlphaDropout layer.

Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also Dropout, VariationalHiddenDropout

source

Lux.Dropout Type
julia
Dropout(p; dims=:)

Dropout layer.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, VariationalHiddenDropout

source

Lux.VariationalHiddenDropout Type
julia
VariationalHiddenDropout(p; dims=:)

VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, Dropout

source

Pooling Layers ​

Lux.AdaptiveLPPool Type
julia
AdaptiveLPPool(output_size; p=2)

Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.AdaptiveMaxPool Type
julia
AdaptiveMaxPool(output_size)

Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.AdaptiveMeanPool Type
julia
AdaptiveMeanPool(output_size)

Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.GlobalLPPool Type
julia
GlobalLPPool(; p=2)

Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.GlobalMaxPool Type
julia
GlobalMaxPool()

Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.GlobalMeanPool Type
julia
GlobalMeanPool()

Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.LPPool Type
julia
LPPool(window; stride=window, pad=0, dilation=1, p=2)

LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

Arguments

Keyword Arguments

GPU Support

This layer is currently only supported on CPU.

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MaxPool Type
julia
MaxPool(window; stride=window, pad=0, dilation=1)

Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MeanPool Type
julia
MeanPool(window; stride=window, pad=0, dilation=1)

Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Recurrent Layers ​

Lux.GRUCell Type
julia
GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
+        init_weight=nothing, init_bias=nothing, init_state=zeros32)

Gated Recurrent Unit (GRU) Cell

r=σ(Wir×x+bir+Whr×hprev+bhr)z=σ(Wiz×x+biz+Whz×hprev+bhz)n=tanh⁡(Win×x+bin+r⋅(Whn×hprev+bhn))hnew=(1−z)⋅n+z⋅hprev

Arguments

Inputs

Returns

Parameters

States

source

Lux.LSTMCell Type
julia
LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
          train_memory::Bool=false, init_weight=nothing, init_bias=nothing,
-         init_state=zeros32, init_memory=zeros32)

Long Short-Term (LSTM) Cell

i=σ(Wii×x+Whi×hprev+bi)f=σ(Wif×x+Whf×hprev+bf)g=tanh(Wig×x+Whg×hprev+bg)o=σ(Wio×x+Who×hprev+bo)cnew=f⋅cprev+i⋅ghnew=o⋅tanh(cnew)

Arguments

Inputs

Returns

Parameters

States

source

Lux.RNNCell Type
julia
RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
-    init_bias=nothing, init_weight=nothing, init_state=zeros32)

An Elman RNNCell cell with activation (typically set to tanh or relu).

hnew=activation(weightih×x+biasih+weighthh×hprev+biashh)

Arguments

Inputs

Returns

Parameters

States

source

Lux.Recurrence Type
julia
Recurrence(cell;
+         init_state=zeros32, init_memory=zeros32)

Long Short-Term (LSTM) Cell

i=σ(Wii×x+Whi×hprev+bi)f=σ(Wif×x+Whf×hprev+bf)g=tanh(Wig×x+Whg×hprev+bg)o=σ(Wio×x+Who×hprev+bo)cnew=f⋅cprev+i⋅ghnew=o⋅tanh(cnew)

Arguments

Inputs

Returns

Parameters

States

source

Lux.RNNCell Type
julia
RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
+    init_bias=nothing, init_weight=nothing, init_state=zeros32)

An Elman RNNCell cell with activation (typically set to tanh or relu).

hnew=activation(weightih×x+biasih+weighthh×hprev+biashh)

Arguments

Inputs

Returns

Parameters

States

source

Lux.Recurrence Type
julia
Recurrence(cell;
     ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex(),
-    return_sequence::Bool=false)

Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.

Relation to Flux.Recur

This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Tip

Frameworks like Tensorflow have special implementation of MultiRNNCell to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.

Chain(
+    return_sequence::Bool=false)

Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.

Relation to Flux.Recur

This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Tip

Frameworks like Tensorflow have special implementation of StackedRNNCells to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.

Chain(
     Recurrence(RNNCell(inputsize => latentsize); return_sequence=true),
     Recurrence(RNNCell(latentsize => latentsize); return_sequence=true),
     :
     x -> stack(x; dims=2)
-)

For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

source

Lux.StatefulRecurrentCell Type
julia
StatefulRecurrentCell(cell)

Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

Arguments

Inputs

Returns

States

source

Lux.BidirectionalRNN Type
julia
BidirectionalRNN(cell::AbstractRecurrentCell,
+)

For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

source

Lux.StatefulRecurrentCell Type
julia
StatefulRecurrentCell(cell)

Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

Arguments

Inputs

Returns

States

source

Lux.BidirectionalRNN Type
julia
BidirectionalRNN(cell::AbstractRecurrentCell,
     backward_cell::Union{AbstractRecurrentCell, Nothing}=nothing;
     merge_mode::Union{Function, Nothing}=vcat,
-    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

Bidirectional RNN wrapper.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Linear Layers ​

Lux.Bilinear Type
julia
Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
+    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

Bidirectional RNN wrapper.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Linear Layers ​

Lux.Bilinear Type
julia
Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
          init_bias=nothing, use_bias=True())
 Bilinear(in12_dims => out, activation=identity; init_weight=nothing,
-         init_bias=nothing, use_bias=True())

Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

z[i] = activation(x' * W[i, :, :] * y + bias[i])

If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Dense Type
julia
Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
-      init_bias=nothing, use_bias=True())

Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Embedding Type
julia
Embedding(in_dims => out_dims; init_weight=rand32)

A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

This layer is often used to store word embeddings and retrieve them using indices.

Arguments

Keyword Arguments

Input

Returns

source

Lux.Scale Type
julia
Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Misc. Helper Layers ​

Lux.FlattenLayer Type
julia
FlattenLayer(; N = nothing)

Flattens the passed array into a matrix.

Keyword Arguments

Inputs

Returns

Example

julia
julia> model = FlattenLayer()
+         init_bias=nothing, use_bias=True())

Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

z[i] = activation(x' * W[i, :, :] * y + bias[i])

If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Dense Type
julia
Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
+      init_bias=nothing, use_bias=True())

Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Embedding Type
julia
Embedding(in_dims => out_dims; init_weight=rand32)

A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

This layer is often used to store word embeddings and retrieve them using indices.

Arguments

Keyword Arguments

Input

Returns

source

Lux.Scale Type
julia
Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Misc. Helper Layers ​

Lux.FlattenLayer Type
julia
FlattenLayer(; N = nothing)

Flattens the passed array into a matrix.

Keyword Arguments

Inputs

Returns

Example

julia
julia> model = FlattenLayer()
 FlattenLayer{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -116,9 +116,9 @@
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(8, 2)

source

Lux.Maxout Type
julia
Maxout(layers...)
+(8, 2)

source

Lux.Maxout Type
julia
Maxout(layers...)
 Maxout(; layers...)
-Maxout(f::Function, n_alts::Int)

This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

See also Parallel to reduce with other operators.

Arguments

Extended Help

Inputs

Returns

Parameters

States

References

[1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

source

Lux.NoOpLayer Type
julia
NoOpLayer()

As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

Example

julia
julia> model = NoOpLayer()
+Maxout(f::Function, n_alts::Int)

This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

See also Parallel to reduce with other operators.

Arguments

Extended Help

Inputs

Returns

Parameters

States

References

[1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

source

Lux.NoOpLayer Type
julia
NoOpLayer()

As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

Example

julia
julia> model = NoOpLayer()
 NoOpLayer()
 
 julia> rng = Random.default_rng();
@@ -128,7 +128,7 @@
 1
 
 julia> y, st_new = model(x, ps, st)
-(1, NamedTuple())

source

Lux.ReshapeLayer Type
julia
ReshapeLayer(dims)

Reshapes the passed array to have a size of (dims..., :)

Arguments

Inputs

Returns

Example

julia
julia> model = ReshapeLayer((2, 2))
+(1, NamedTuple())

source

Lux.ReshapeLayer Type
julia
ReshapeLayer(dims)

Reshapes the passed array to have a size of (dims..., :)

Arguments

Inputs

Returns

Example

julia
julia> model = ReshapeLayer((2, 2))
 ReshapeLayer(output_dims = (2, 2, :))
 
 julia> rng = Random.default_rng();
@@ -138,7 +138,7 @@
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(2, 2, 3)

source

Lux.SelectDim Type
julia
SelectDim(dim, i)

Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

Arguments

Inputs

Returns

source

Lux.WrappedFunction Type
julia
WrappedFunction(f)

Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

Arguments

Inputs

Returns

source

Lux.ReverseSequence Type
julia
ReverseSequence(dim = nothing)

Reverse the specified dimension dims of the passed array

Arguments

Inputs

Returns

Example

julia
julia> model = ReverseSequence()
+(2, 2, 3)

source

Lux.SelectDim Type
julia
SelectDim(dim, i)

Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

Arguments

Inputs

Returns

source

Lux.WrappedFunction Type
julia
WrappedFunction(f)

Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

Arguments

Inputs

Returns

source

Lux.ReverseSequence Type
julia
ReverseSequence(dim = nothing)

Reverse the specified dimension dims of the passed array

Arguments

Inputs

Returns

Example

julia
julia> model = ReverseSequence()
 ReverseSequence{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -147,7 +147,7 @@
        x = [1.0, 2.0, 3.0];
 
 julia> y, st_new = model(x, ps, st)
-([3.0, 2.0, 1.0], NamedTuple())

source

Normalization Layers ​

Lux.BatchNorm Type
julia
BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+([3.0, 2.0, 1.0], NamedTuple())

source

Normalization Layers ​

Lux.BatchNorm Type
julia
BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
           affine=True(), track_stats=True(), epsilon=1f-5, momentum=0.1f0)

Batch Normalization layer.

BatchNorm computes the mean and variance for each D1×...×DN−2×1×DN input slice and normalises the input accordingly.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

Use Lux.testmode during inference.

Example

julia
julia> Chain(Dense(784 => 64), BatchNorm(64, relu), Dense(64 => 10), BatchNorm(10))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -155,7 +155,7 @@
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = BatchNorm(10, affine=true, track_stats=true),  # 20 parameters, plus 21
 )         # Total: 51_038 parameters,
-          #        plus 150 states.

Warning

Passing a batch size of 1, during training will result in an error.

See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.GroupNorm Type
julia
GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
+          #        plus 150 states.

Warning

Passing a batch size of 1, during training will result in an error.

See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.GroupNorm Type
julia
GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
           init_scale=ones32, affine=true, epsilon=1f-5)

Group Normalization layer.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

Use Lux.testmode during inference.

Example

julia
julia> Chain(Dense(784 => 64), GroupNorm(64, 4, relu), Dense(64 => 10), GroupNorm(10, 5))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -163,7 +163,7 @@
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = GroupNorm(10, 5, affine=true),  # 20 parameters
 )         # Total: 51_038 parameters,
-          #        plus 0 states.

See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.InstanceNorm Type
julia
InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+          #        plus 0 states.

See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.InstanceNorm Type
julia
InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
              affine=False(), track_stats=False(), epsilon=1f-5, momentum=0.1f0)

Instance Normalization. For details see [1].

Instance Normalization computes the mean and variance for each D1×...×DN−2×1×1` input slice and normalises the input accordingly.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

Use Lux.testmode during inference.

Example

julia
julia> Chain(Dense(784 => 64), InstanceNorm(64, relu; affine=true), Dense(64 => 10),
            InstanceNorm(10, relu; affine=true))
 Chain(
@@ -172,11 +172,11 @@
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = InstanceNorm(10, relu, affine=true, track_stats=false),  # 20 parameters, plus 1
 )         # Total: 51_038 parameters,
-          #        plus 2 states.

References

[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

source

Lux.LayerNorm Type
julia
LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
-          affine=true, init_bias=zeros32, init_scale=ones32)

Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

Given an input array x, this layer computes

y=x−E[x]Var[x]+ϵ∗γ+β

where γ & β are trainable parameters if affine=true.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Lux.WeightNorm Type
julia
WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
-           dims::Union{Tuple, Nothing}=nothing)

Applies weight normalization to a parameter in the given layer.

w=gv∥v∥

Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

Arguments

Inputs

Returns

Parameters

States

source

Upsampling ​

Lux.PixelShuffle Type
julia
PixelShuffle(r::Int)

Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

See NNlib.pixel_shuffle for more details.

Arguments

Inputs

Returns

source

Lux.Upsample Type
julia
Upsample(mode = :nearest; [scale, size, align_corners=false])
-Upsample(scale, mode = :nearest)

Upsampling Layer.

Layer Construction

Option 1

Exactly one of two keywords must be specified:

Option 2

Currently supported upsampling modes and corresponding NNlib's methods are:

Extended Help

Other Keyword Arguments

Inputs

Returns

source

- + # plus 2 states.

References

[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

source

Lux.LayerNorm Type
julia
LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
+          affine=true, init_bias=zeros32, init_scale=ones32)

Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

Given an input array x, this layer computes

y=x−E[x]Var[x]+ϵ∗γ+β

where γ & β are trainable parameters if affine=true.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Lux.WeightNorm Type
julia
WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
+           dims::Union{Tuple, Nothing}=nothing)

Applies weight normalization to a parameter in the given layer.

w=gv∥v∥

Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

Arguments

Inputs

Returns

Parameters

States

source

Upsampling ​

Lux.PixelShuffle Type
julia
PixelShuffle(r::Int)

Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

See NNlib.pixel_shuffle for more details.

Arguments

Inputs

Returns

source

Lux.Upsample Type
julia
Upsample(mode = :nearest; [scale, size, align_corners=false])
+Upsample(scale, mode = :nearest)

Upsampling Layer.

Layer Construction

Option 1

Exactly one of two keywords must be specified:

Option 2

Currently supported upsampling modes and corresponding NNlib's methods are:

Extended Help

Other Keyword Arguments

Inputs

Returns

source

+ \ No newline at end of file diff --git a/dev/api/Lux/utilities.html b/dev/api/Lux/utilities.html index 5dcdef90a7..9287b2dbd8 100644 --- a/dev/api/Lux/utilities.html +++ b/dev/api/Lux/utilities.html @@ -5,14 +5,14 @@ Utilities | Lux.jl Docs - - + + - + - + - + @@ -27,13 +27,13 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Utilities ​

Training API ​

Helper Functions making it easier to train Lux.jl models.

Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.

Lux.Training.TrainState Type
julia
TrainState

Training State containing:

  • model: Lux model.

  • parameters: Trainable Variables of the model.

  • states: Non-trainable Variables of the model.

  • optimizer: Optimizer from Optimisers.jl.

  • optimizer_state: Optimizer State.

  • step: Number of updates of the parameters made.

Internal fields:

  • cache: Cached values. Implementations are free to use this for whatever they want.

  • objective_function: Objective function might be cached.

Warning

Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.

source

Lux.Training.compute_gradients Function
julia
compute_gradients(ad::AbstractADType, objective_function::Function, data,
-    ts::TrainState)

Compute the gradients of the objective function wrt parameters stored in ts.

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl
AutoReverseDiff(; compile)ReverseDiff.jl
AutoTrackerTracker.jl
AutoEnzymeEnzyme.jl

Arguments

  • ad: Backend (from ADTypes.jl) used to compute the gradients.

  • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.

  • data: Data used to compute the gradients.

  • ts: Current Training State. See TrainState.

Return

A 4-Tuple containing:

  • grads: Computed Gradients.

  • loss: Loss from the objective function.

  • stats: Any computed statistics from the objective function.

  • ts: Updated Training State.

Known Limitations

  • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.

Aliased Gradients

grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.

source

Lux.Training.apply_gradients Function
julia
apply_gradients(ts::TrainState, grads)

Update the parameters stored in ts using the gradients grads.

Arguments

  • ts: TrainState object.

  • grads: Gradients of the loss function wrt ts.params.

Returns

Updated TrainState object.

source

Lux.Training.apply_gradients! Function
julia
apply_gradients!(ts::TrainState, grads)

Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.

Arguments

  • ts: TrainState object.

  • grads: Gradients of the loss function wrt ts.params.

Returns

Updated TrainState object.

source

Lux.Training.single_train_step Function
julia
single_train_step(backend, obj_fn::F, data, ts::TrainState)

Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.

In most cases you should use single_train_step! instead of this function.

Return

Returned values are the same as compute_gradients.

source

Lux.Training.single_train_step! Function
julia
single_train_step!(backend, obj_fn::F, data, ts::TrainState)

Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.

Return

Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).

source

Loss Functions ​

Loss Functions Objects take 2 forms of inputs:

  1. ŷ and y where ŷ is the predicted output and y is the target output.

  2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.

Warning

When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.

Lux.GenericLossFunction Type
julia
GenericLossFunction(loss_fn; agg = mean)

Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.

julia
julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
+    
Lux.jl Docs
Appearance

 Trusted by 
Scientific Computing
SciML.ai
Machine Learning
Utilities ​
Training API ​
Helper Functions making it easier to train Lux.jl models.
Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.
Lux.Training.TrainState Type
julia
TrainState
Training State containing:
  • model: Lux model.
  • parameters: Trainable Variables of the model.
  • states: Non-trainable Variables of the model.
  • optimizer: Optimizer from Optimisers.jl.
  • optimizer_state: Optimizer State.
  • step: Number of updates of the parameters made.
Internal fields:
  • cache: Cached values. Implementations are free to use this for whatever they want.
  • objective_function: Objective function might be cached.
Warning
Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.
source
Lux.Training.compute_gradients Function
julia
compute_gradients(ad::AbstractADType, objective_function::Function, data,
+    ts::TrainState)
Compute the gradients of the objective function wrt parameters stored in ts.
Backends & AD Packages
Supported BackendsPackages Needed
AutoZygoteZygote.jl
AutoReverseDiff(; compile)ReverseDiff.jl
AutoTrackerTracker.jl
AutoEnzymeEnzyme.jl
Arguments
  • ad: Backend (from ADTypes.jl) used to compute the gradients.
  • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.
  • data: Data used to compute the gradients.
  • ts: Current Training State. See TrainState.
Return
A 4-Tuple containing:
  • grads: Computed Gradients.
  • loss: Loss from the objective function.
  • stats: Any computed statistics from the objective function.
  • ts: Updated Training State.
Known Limitations
  • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.
Aliased Gradients
grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.
source
Lux.Training.apply_gradients Function
julia
apply_gradients(ts::TrainState, grads)
Update the parameters stored in ts using the gradients grads.
Arguments
  • ts: TrainState object.
  • grads: Gradients of the loss function wrt ts.params.
Returns
Updated TrainState object.
source
Lux.Training.apply_gradients! Function
julia
apply_gradients!(ts::TrainState, grads)
Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.
Arguments
  • ts: TrainState object.
  • grads: Gradients of the loss function wrt ts.params.
Returns
Updated TrainState object.
source
Lux.Training.single_train_step Function
julia
single_train_step(backend, obj_fn::F, data, ts::TrainState)
Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.
In most cases you should use single_train_step! instead of this function.
Return
Returned values are the same as compute_gradients.
source
Lux.Training.single_train_step! Function
julia
single_train_step!(backend, obj_fn::F, data, ts::TrainState)
Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.
Return
Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).
source
Loss Functions ​
Loss Functions Objects take 2 forms of inputs:
  1. ŷ and y where ŷ is the predicted output and y is the target output.
  2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.
Warning
When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.
Lux.GenericLossFunction Type
julia
GenericLossFunction(loss_fn; agg = mean)
Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.
julia
julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> mseloss(y_model, 1:3) ≈ 0.01
-true
Special Note
This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
source
Lux.BinaryCrossEntropyLoss Type
julia
BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
+true
Special Note
This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
source
Lux.BinaryCrossEntropyLoss Type
julia
BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
     label_smoothing::Union{Nothing, Real}=nothing,
     logits::Union{Bool, Val}=Val(false))
Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
Returns the binary cross entropy loss computed as:
  • If logits is either false or Val(false):
agg(−y~∗log⁡(y^+ϵ)−(1−y~)∗log⁡(1−y^+ϵ))
  • If logits is true or Val(true):
agg((1−y~)∗y^−logσ(y^))
The value of y~ is computed using label smoothing. If label_smoothing is nothing, then no label smoothing is applied. If label_smoothing is a real number ∈[0,1], then the value of y~ is:
y~=(1−α)∗y+α∗0.5
where α is the value of label_smoothing.
Extended Help
Example
julia
julia> bce = BinaryCrossEntropyLoss();
 
@@ -61,7 +61,7 @@
 julia> logitbce_ls = BinaryCrossEntropyLoss(label_smoothing=0.1, logits=Val(true));
 
 julia> logitbce_ls(y_model, y_bin) > logitbce(y_model, y_bin)
-true
source
Lux.BinaryFocalLoss Type
julia
BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
Return the binary focal loss [1]. The model input, y^, is expected to be normalized (i.e. softmax output).
For γ=0 this is equivalent to BinaryCrossEntropyLoss.
Example
julia
julia> y = [0  1  0
+true
source
Lux.BinaryFocalLoss Type
julia
BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
Return the binary focal loss [1]. The model input, y^, is expected to be normalized (i.e. softmax output).
For γ=0 this is equivalent to BinaryCrossEntropyLoss.
Example
julia
julia> y = [0  1  0
             1  0  1];
 
 julia> ŷ = [0.268941  0.5  0.268941
@@ -71,7 +71,7 @@
 true
 
 julia> BinaryFocalLoss(gamma=0)(ŷ, y) ≈ BinaryCrossEntropyLoss()(ŷ, y)
-true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.CrossEntropyLoss Type
julia
CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
+true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.CrossEntropyLoss Type
julia
CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
     label_smoothing::Union{Nothing, Real}=nothing)
Return the cross entropy loss which is used in multi-class classification tasks. The input, y^, is expected to be normalized (i.e. softmax output) if logits is false or Val(false).
The loss is calculated as:
agg(−∑y~log⁡(y^+ϵ))
where ϵ is added for numerical stability. The value of y~ is computed using label smoothing. If label_smoothing is nothing, then no label smoothing is applied. If label_smoothing is a real number ∈[0,1], then the value of y~ is calculated as:
y~=(1−α)∗y+α∗size along dim
where α is the value of label_smoothing.
Extended Help
Example
julia
julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
@@ -93,7 +93,7 @@
 true
 
 julia> CrossEntropyLoss(label_smoothing=0.15)(y_model, y) ≈ 1.5776052f0
-true
source
Lux.DiceCoeffLoss Type
julia
DiceCoeffLoss(; smooth = true, agg = mean)
Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
agg(1−2∑yy^+α∑y2+∑y^2+α)
where α is the smoothing factor (smooth).
Example
julia
julia> y_pred = [1.1, 2.1, 3.1];
+true
source
Lux.DiceCoeffLoss Type
julia
DiceCoeffLoss(; smooth = true, agg = mean)
Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
agg(1−2∑yy^+α∑y2+∑y^2+α)
where α is the smoothing factor (smooth).
Example
julia
julia> y_pred = [1.1, 2.1, 3.1];
 
 julia> DiceCoeffLoss()(y_pred, 1:3)  ≈ 0.000992391663909964
 true
@@ -102,7 +102,7 @@
 true
 
 julia> DiceCoeffLoss()(reshape(y_pred, 3, 1), reshape(1:3, 3, 1)) ≈ 0.000992391663909964
-true
References
[1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
source
Lux.FocalLoss Type
julia
FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, y^, is expected to be normalized (i.e. softmax output).
The modulating factor γ, controls the down-weighting strength. For γ=0 this is equivalent to CrossEntropyLoss.
Example
julia
julia> y = [1  0  0  0  1
+true
References
[1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
source
Lux.FocalLoss Type
julia
FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, y^, is expected to be normalized (i.e. softmax output).
The modulating factor γ, controls the down-weighting strength. For γ=0 this is equivalent to CrossEntropyLoss.
Example
julia
julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
 3×5 Matrix{Int64}:
@@ -117,20 +117,20 @@
  0.665241   0.665241   0.665241   0.665241   0.665241
 
 julia> FocalLoss()(ŷ, y) ≈ 1.1277556f0
-true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.HingeLoss Function
julia
HingeLoss(; agg = mean)
Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^))
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = HingeLoss();
+true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.HingeLoss Function
julia
HingeLoss(; agg = mean)
Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^))
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = HingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.55
-true
source
Lux.HuberLoss Function
julia
HuberLoss(; delta = 1, agg = mean)
Returns the Huber loss, calculated as:
L={0.5∗|y−y^|2if |y−y^|≤δδ∗(|y−y^|−0.5∗δ)otherwise
where δ is the delta parameter.
Example
julia
julia> y_model = [1.1, 2.1, 3.1];
+true
source
Lux.HuberLoss Function
julia
HuberLoss(; delta = 1, agg = mean)
Returns the Huber loss, calculated as:
L={0.5∗|y−y^|2if |y−y^|≤δδ∗(|y−y^|−0.5∗δ)otherwise
where δ is the delta parameter.
Example
julia
julia> y_model = [1.1, 2.1, 3.1];
 
 julia> HuberLoss()(y_model, 1:3) ≈ 0.005000000000000009
 true
 
 julia> HuberLoss(delta=0.05)(y_model, 1:3) ≈ 0.003750000000000005
-true
source
Lux.KLDivergenceLoss Type
julia
KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
Return the Kullback-Leibler Divergence loss between the predicted distribution y^ and the true distribution y:
The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
For epsilon and label_smoothing, see CrossEntropyLoss.
Example
julia
julia> p1 = [1 0; 0 1]
+true
source
Lux.KLDivergenceLoss Type
julia
KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
Return the Kullback-Leibler Divergence loss between the predicted distribution y^ and the true distribution y:
The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
For epsilon and label_smoothing, see CrossEntropyLoss.
Example
julia
julia> p1 = [1 0; 0 1]
 2×2 Matrix{Int64}:
  1  0
  0  1
@@ -150,44 +150,44 @@
 0.0
 
 julia> KLDivergenceLoss(; epsilon=0)(p1, p2)
-Inf
source
Lux.MAELoss Function
julia
MAELoss(; agg = mean)
Returns the loss corresponding to mean absolute error:
agg(|y^−y|)
Example
julia
julia> loss = MAELoss();
+Inf
source
Lux.MAELoss Function
julia
MAELoss(; agg = mean)
Returns the loss corresponding to mean absolute error:
agg(|y^−y|)
Example
julia
julia> loss = MAELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.1
-true
source
Lux.MSELoss Function
julia
MSELoss(; agg = mean)
Returns the loss corresponding to mean squared error:
agg((y^−y)2)
Example
julia
julia> loss = MSELoss();
+true
source
Lux.MSELoss Function
julia
MSELoss(; agg = mean)
Returns the loss corresponding to mean squared error:
agg((y^−y)2)
Example
julia
julia> loss = MSELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.01
-true
source
Lux.MSLELoss Function
julia
MSLELoss(; agg = mean, epsilon = nothing)
Returns the loss corresponding to mean squared logarithmic error:
agg((log⁡(y^+ϵ)−log⁡(y+ϵ))2)
epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
Example
julia
julia> loss = MSLELoss();
+true
source
Lux.MSLELoss Function
julia
MSLELoss(; agg = mean, epsilon = nothing)
Returns the loss corresponding to mean squared logarithmic error:
agg((log⁡(y^+ϵ)−log⁡(y+ϵ))2)
epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
Example
julia
julia> loss = MSLELoss();
 
 julia> loss(Float32[1.1, 2.2, 3.3], 1:3) ≈ 0.009084041f0
 true
 
 julia> loss(Float32[0.9, 1.8, 2.7], 1:3) ≈ 0.011100831f0
-true
source
Lux.PoissonLoss Function
julia
PoissonLoss(; agg = mean, epsilon = nothing)
Return how much the predicted distribution y^ diverges from the expected Poisson distribution y, calculated as:
agg(y^−y∗log⁡(y^))
Example
julia
julia> y_model = [1, 3, 3];  # data should only take integral values
+true
source
Lux.PoissonLoss Function
julia
PoissonLoss(; agg = mean, epsilon = nothing)
Return how much the predicted distribution y^ diverges from the expected Poisson distribution y, calculated as:
agg(y^−y∗log⁡(y^))
Example
julia
julia> y_model = [1, 3, 3];  # data should only take integral values
 
 julia> PoissonLoss()(y_model, 1:3) ≈ 0.502312852219817
-true
source
Lux.SiameseContrastiveLoss Function
julia
SiameseContrastiveLoss(; margin = true, agg = mean)
Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
agg((1−y)y^2+y∗max(0,margin−y^)2)
Specify margin to set the baseline for distance at which pairs are dissimilar.
Example
julia
julia> ŷ = [0.5, 1.5, 2.5];
+true
source
Lux.SiameseContrastiveLoss Function
julia
SiameseContrastiveLoss(; margin = true, agg = mean)
Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
agg((1−y)y^2+y∗max(0,margin−y^)2)
Specify margin to set the baseline for distance at which pairs are dissimilar.
Example
julia
julia> ŷ = [0.5, 1.5, 2.5];
 
 julia> SiameseContrastiveLoss()(ŷ, 1:3) ≈ -4.833333333333333
 true
 
 julia> SiameseContrastiveLoss(margin=2)(ŷ, 1:3) ≈ -4.0
-true
References
[1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
source
Lux.SquaredHingeLoss Function
julia
SquaredHingeLoss(; agg = mean)
Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^)2)
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = SquaredHingeLoss();
+true
References
[1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
source
Lux.SquaredHingeLoss Function
julia
SquaredHingeLoss(; agg = mean)
Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^)2)
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = SquaredHingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.625
-true
source
LuxOps Module ​
Lux.LuxOps Module
julia
LuxOps
This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
source
Lux.LuxOps.eachslice Function
julia
eachslice(x, dims::Val)
Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
source
Lux.LuxOps.foldl_init Function
julia
foldl_init(op, x)
-foldl_init(op, x, init)
Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
source
Lux.LuxOps.getproperty Function
julia
getproperty(x, ::Val{v})
-getproperty(x, ::StaticSymbol{v})
Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
source
Lux.LuxOps.xlogx Function
julia
xlogx(x::Number)
Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
source
Lux.LuxOps.xlogy Function
julia
xlogy(x::Number, y::Number)
Return x * log(y) for y > 0, and zero when x == 0.
source
Lux.LuxOps.istraining Function
julia
istraining(::Val{training})
+true
source
LuxOps Module ​
Lux.LuxOps Module
julia
LuxOps
This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
source
Lux.LuxOps.eachslice Function
julia
eachslice(x, dims::Val)
Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
source
Lux.LuxOps.foldl_init Function
julia
foldl_init(op, x)
+foldl_init(op, x, init)
Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
source
Lux.LuxOps.getproperty Function
julia
getproperty(x, ::Val{v})
+getproperty(x, ::StaticSymbol{v})
Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
source
Lux.LuxOps.xlogx Function
julia
xlogx(x::Number)
Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
source
Lux.LuxOps.xlogy Function
julia
xlogy(x::Number, y::Number)
Return x * log(y) for y > 0, and zero when x == 0.
source
Lux.LuxOps.istraining Function
julia
istraining(::Val{training})
 istraining(::StaticBool)
 istraining(::Bool)
-istraining(st::NamedTuple)
Returns true if training is true or if st contains a training field with value true. Else returns false.
source
Lux.LuxOps.multigate Function
julia
multigate(x::AbstractArray, ::Val{N})
Split up x into N equally sized chunks (along dimension 1).
source
Recursive Operations ​
Lux.recursive_map Function
julia
recursive_map(f, x, args...)
Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
How this works?
For the following types it directly defines recursion rules:
  1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
  2. Tuple/NamedTuple: We recurse on the values.
  3. Number/Val/Nothing: We directly apply f.
  4. For all other types, we recurse on the fields using Functors.fmap.
Note
In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
source
Lux.recursive_add!! Function
julia
recursive_add!!(x, y)
Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
Any leaves of x that are arrays and allow in-place addition will be modified in place.
source
Lux.recursive_copyto! Function
julia
recursive_copyto!(x, y)
Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
source
Lux.recursive_eltype Function
julia
recursive_eltype(x, unwrap_ad_types = Val(false))
Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
For ambiguous inputs like nothing and Val types we return Bool as the eltype.
If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
source
Lux.recursive_make_zero Function
julia
recursive_make_zero(x)
Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
See also Lux.recursive_make_zero!!.
source
Lux.recursive_make_zero!! Function
julia
recursive_make_zero!!(x)
Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
See also Lux.recursive_make_zero for fully out-of-place version.
source
Updating Floating Point Precision ​
By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.
Lux.f16 Function
julia
f16(m)
Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
source
Lux.f32 Function
julia
f32(m)
Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
source
Lux.f64 Function
julia
f64(m)
Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
source
Element Type Matching ​
Lux.match_eltype Function
julia
match_eltype(layer, ps, st, args...)
Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
Extended Help
Controlling the Behavior via Preferences
Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
  • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
  • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
  • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
  • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
Warning
We print the warning for type-mismatch only once.
Element Type Conversions
For "convert" only the following conversions are done:
Element Type of parameters/statesElement Type of args...Converted to
Float64IntegerFloat64
Float32Float64Float32
Float32IntegerFloat32
Float16Float64Float16
Float16Float32Float16
Float16IntegerFloat16
source
Stateful Layer ​
Lux.StatefulLuxLayer Type
julia
StatefulLuxLayer{FT}(model, ps, st)
Warning
This is not a Lux.AbstractLuxLayer
A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
Usecases
  • Internal implementation of @compact heavily uses this layer.
  • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
  • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
Static Parameters
  • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
  • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
Arguments
  • model: A Lux layer
  • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
  • st: The state of the layer
Inputs
  • x: The input to the layer
  • ps: The parameters of the layer. Optional, defaults to s.ps
Outputs
  • y: The output of the layer
source
Compact Layer ​
Lux.@compact Macro
julia
@compact(kw...) do x
+istraining(st::NamedTuple)
Returns true if training is true or if st contains a training field with value true. Else returns false.
source
Lux.LuxOps.multigate Function
julia
multigate(x::AbstractArray, ::Val{N})
Split up x into N equally sized chunks (along dimension 1).
source
Recursive Operations ​
Lux.recursive_map Function
julia
recursive_map(f, x, args...)
Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
How this works?
For the following types it directly defines recursion rules:
  1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
  2. Tuple/NamedTuple: We recurse on the values.
  3. Number/Val/Nothing: We directly apply f.
  4. For all other types, we recurse on the fields using Functors.fmap.
Note
In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
source
Lux.recursive_add!! Function
julia
recursive_add!!(x, y)
Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
Any leaves of x that are arrays and allow in-place addition will be modified in place.
source
Lux.recursive_copyto! Function
julia
recursive_copyto!(x, y)
Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
source
Lux.recursive_eltype Function
julia
recursive_eltype(x, unwrap_ad_types = Val(false))
Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
For ambiguous inputs like nothing and Val types we return Bool as the eltype.
If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
source
Lux.recursive_make_zero Function
julia
recursive_make_zero(x)
Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
See also Lux.recursive_make_zero!!.
source
Lux.recursive_make_zero!! Function
julia
recursive_make_zero!!(x)
Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
See also Lux.recursive_make_zero for fully out-of-place version.
source
Updating Floating Point Precision ​
By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.
Lux.f16 Function
julia
f16(m)
Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
source
Lux.f32 Function
julia
f32(m)
Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
source
Lux.f64 Function
julia
f64(m)
Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
source
Element Type Matching ​
Lux.match_eltype Function
julia
match_eltype(layer, ps, st, args...)
Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
Extended Help
Controlling the Behavior via Preferences
Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
  • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
  • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
  • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
  • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
Warning
We print the warning for type-mismatch only once.
Element Type Conversions
For "convert" only the following conversions are done:
Element Type of parameters/statesElement Type of args...Converted to
Float64IntegerFloat64
Float32Float64Float32
Float32IntegerFloat32
Float16Float64Float16
Float16Float32Float16
Float16IntegerFloat16
source
Stateful Layer ​
Lux.StatefulLuxLayer Type
julia
StatefulLuxLayer{FT}(model, ps, st)
Warning
This is not a Lux.AbstractLuxLayer
A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
Usecases
  • Internal implementation of @compact heavily uses this layer.
  • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
  • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
Static Parameters
  • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
  • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
Arguments
  • model: A Lux layer
  • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
  • st: The state of the layer
Inputs
  • x: The input to the layer
  • ps: The parameters of the layer. Optional, defaults to s.ps
Outputs
  • y: The output of the layer
source
Compact Layer ​
Lux.@compact Macro
julia
@compact(kw...) do x
     ...
     @return y # optional (but recommended for best performance)
 end
@@ -303,7 +303,7 @@
 true
You may also specify a name for the model, which will be used instead of the default printout, which gives a verbatim representation of the code used to construct the model:
julia
julia> model = @compact(w=rand(3), name="Linear(3 => 1)") do x
            @return sum(w .* x)
        end
-Linear(3 => 1)      # 3 parameters
This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
Type Stability
If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
Parameter Count
Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
source
Lux.@init_fn Macro
julia
@init_fn(fn, kind::Symbol = :parameter)
Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
Arguments
  • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
  • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
Examples
julia
julia> using Lux, Random
+Linear(3 => 1)      # 3 parameters
This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
Type Stability
If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
Parameter Count
Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
source
Lux.@init_fn Macro
julia
@init_fn(fn, kind::Symbol = :parameter)
Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
Arguments
  • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
  • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
Examples
julia
julia> using Lux, Random
 
 julia> r = @compact(w=@init_fn(rng->randn32(rng, 3, 2)),
            b=@init_fn(rng->randn32(rng, 3), :state)) do x
@@ -319,7 +319,7 @@
 (3,)
 
 julia> size(r([1, 2], ps, st)[1])
-(3,)
source
Lux.@non_trainable Macro
julia
@non_trainable(x)
Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
Arguments
  • x: The value to be marked as non-trainable.
Examples
julia
julia> using Lux, Random
+(3,)
source
Lux.@non_trainable Macro
julia
@non_trainable(x)
Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
Arguments
  • x: The value to be marked as non-trainable.
Examples
julia
julia> using Lux, Random
 
 julia> r = @compact(w=ones(3), w_fixed=@non_trainable(rand(3))) do x
            @return sum(w .* x .+ w_fixed)
@@ -339,9 +339,9 @@
 true
 
 julia> res isa Number
-true
source
Miscellaneous ​
Lux.set_dispatch_doctor_preferences! Function
julia
set_dispatch_doctor_preferences!(mode::String)
-set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")
Set the dispatch doctor preference for LuxCore and LuxLib packages.
mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.
If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.
source

-    
+true

source

Miscellaneous ​

Lux.set_dispatch_doctor_preferences! Function
julia
set_dispatch_doctor_preferences!(mode::String)
+set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")

Set the dispatch doctor preference for LuxCore and LuxLib packages.

mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.

If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.

source

+ \ No newline at end of file diff --git a/dev/api/Testing_Functionality/LuxTestUtils.html b/dev/api/Testing_Functionality/LuxTestUtils.html index 49081c5d58..51ac10baa4 100644 --- a/dev/api/Testing_Functionality/LuxTestUtils.html +++ b/dev/api/Testing_Functionality/LuxTestUtils.html @@ -5,12 +5,12 @@ LuxTestUtils | Lux.jl Docs - - + + - + - + @@ -39,7 +39,7 @@ julia> x = (; t=rand(10), x=(z=[2.0],)) julia> test_gradients(f, 1.0, x, nothing)

source

LuxTestUtils.@test_gradients Macro
julia
@test_gradients(f, args...; kwargs...)

See the documentation of test_gradients for more details. This macro provides correct line information for the failing tests.

source

Extensions to @test ​

LuxTestUtils.@test_softfail Macro
julia
@test_softfail expr

Evaluate expr and record a test result. If expr throws an exception, the test result will be recorded as an error. If expr returns a value, and it is not a boolean, the test result will be recorded as an error.

If the test result is false then the test will be recorded as a broken test, else it will be recorded as a pass.

source

- + \ No newline at end of file diff --git a/dev/assets/api_Lux_autodiff.md.C9onSrGW.js b/dev/assets/api_Lux_autodiff.md.CcfC5vAK.js similarity index 96% rename from dev/assets/api_Lux_autodiff.md.C9onSrGW.js rename to dev/assets/api_Lux_autodiff.md.CcfC5vAK.js index 97753023f5..37309f73eb 100644 --- a/dev/assets/api_Lux_autodiff.md.C9onSrGW.js +++ b/dev/assets/api_Lux_autodiff.md.CcfC5vAK.js @@ -1 +1 @@ -import{_ as d,c as i,j as t,a,G as l,a2 as s,B as r,o as n}from"./chunks/framework.CCjWn1F9.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),Q={name:"api/Lux/autodiff.md"},p={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return n(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",p,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",c,[(n(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(n(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

Returns

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/dev/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=d(Q,[["render",f]]);export{v as __pageData,L as default}; +import{_ as n,c as i,j as t,a,G as l,a2 as s,B as r,o as d}from"./chunks/framework.CCjWn1F9.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),c={name:"api/Lux/autodiff.md"},Q={class:"jldocstring custom-block"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return d(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",Q,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",p,[(d(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(d(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

Returns

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/dev/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=n(c,[["render",f]]);export{v as __pageData,L as default}; diff --git a/dev/assets/api_Lux_autodiff.md.C9onSrGW.lean.js b/dev/assets/api_Lux_autodiff.md.CcfC5vAK.lean.js similarity index 96% rename from dev/assets/api_Lux_autodiff.md.C9onSrGW.lean.js rename to dev/assets/api_Lux_autodiff.md.CcfC5vAK.lean.js index 97753023f5..37309f73eb 100644 --- a/dev/assets/api_Lux_autodiff.md.C9onSrGW.lean.js +++ b/dev/assets/api_Lux_autodiff.md.CcfC5vAK.lean.js @@ -1 +1 @@ -import{_ as d,c as i,j as t,a,G as l,a2 as s,B as r,o as n}from"./chunks/framework.CCjWn1F9.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),Q={name:"api/Lux/autodiff.md"},p={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return n(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",p,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",c,[(n(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(n(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

Returns

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/dev/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=d(Q,[["render",f]]);export{v as __pageData,L as default}; +import{_ as n,c as i,j as t,a,G as l,a2 as s,B as r,o as d}from"./chunks/framework.CCjWn1F9.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),c={name:"api/Lux/autodiff.md"},Q={class:"jldocstring custom-block"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return d(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",Q,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",p,[(d(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)
',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(d(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

Returns

source

',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

Returns

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/dev/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=n(c,[["render",f]]);export{v as __pageData,L as default}; diff --git a/dev/assets/api_Lux_contrib.md.BRGt7rCL.js b/dev/assets/api_Lux_contrib.md.BHxLoP1M.js similarity index 97% rename from dev/assets/api_Lux_contrib.md.BRGt7rCL.js rename to dev/assets/api_Lux_contrib.md.BHxLoP1M.js index 0874ab47b1..c2ab28acb5 100644 --- a/dev/assets/api_Lux_contrib.md.BRGt7rCL.js +++ b/dev/assets/api_Lux_contrib.md.BHxLoP1M.js @@ -1,6 +1,6 @@ -import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Experimental Features","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/contrib.md","filePath":"api/Lux/contrib.md","lastUpdated":null}'),r={name:"api/Lux/contrib.md"},d={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"};function F(m,s,b,C,f,D){const t=p("Badge");return k(),l("div",null,[s[21]||(s[21]=e('

Experimental Features ​

All features listed on this page are experimental which means:

  1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.

  2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.

  3. None of the features are exported.

Parameter Freezing ​

',4)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Lux.Experimental.FrozenLayer",href:"#Lux.Experimental.FrozenLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.FrozenLayer")],-1)),s[1]||(s[1]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[2]||(s[2]=e(`
julia
FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})

Freeze the parameters with name which_params of the layer l.

Use Lux.Experimental.freeze instead

It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.

No checks for which_params

There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.

Arguments

Extended Help

Parameters

States

Note on Internal Layer Implementation

The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.

Example

julia
julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
-FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

source

`,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

source

julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
-    which_params::Optional{Tuple} = nothing)

Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

source

`,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
julia
unfreeze(l::FrozenLayer)

Unfreezes the layer l.

source

julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)

Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

source

',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/dev/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

KeyPath provided to the function

The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

Call Signature for f

Extended Help

Example

julia
julia> using Lux, Random
+import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Experimental Features","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/contrib.md","filePath":"api/Lux/contrib.md","lastUpdated":null}'),r={name:"api/Lux/contrib.md"},d={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"};function F(m,s,C,b,D,f){const t=p("Badge");return k(),l("div",null,[s[21]||(s[21]=e('
Experimental Features ​
All features listed on this page are experimental which means:
  1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.
  2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.
  3. None of the features are exported.
Parameter Freezing ​
',4)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Lux.Experimental.FrozenLayer",href:"#Lux.Experimental.FrozenLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.FrozenLayer")],-1)),s[1]||(s[1]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[2]||(s[2]=e(`
julia
FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})
Freeze the parameters with name which_params of the layer l.
Use Lux.Experimental.freeze instead
It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.
No checks for which_params
There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.
Arguments
  • l: Lux AbstractLuxLayer.
  • which_params: Parameter Names to be Frozen. Can be set to nothing, in which case all parameters are frozen.
Extended Help
Parameters
  • Parameters of the layer l excluding which_params.
States
  • frozen_params: Parameters that are frozen, i.e., which_params.
  • states: The state of the inner layer l.
Note on Internal Layer Implementation
The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.
Example
julia
julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
+FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable
See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.
source
`,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)
Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.
source
julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
+    which_params::Optional{Tuple} = nothing)
Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.
source
`,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
julia
unfreeze(l::FrozenLayer)
Unfreezes the layer l.
source
julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)
Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.
source
',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/dev/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)
Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.
KeyPath provided to the function
The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).
Call Signature for f
  • Must take 4 inputs – AbstractLuxLayer, Corresponding Parameters, Corresponding States, and the Functors.KeyPath to the layer.
  • Must return a tuple of 3 elements – AbstractLuxLayer, new parameters and the new states.
Extended Help
Example
julia
julia> using Lux, Random
 
 julia> c = Parallel(
            +; chain=Chain(; dense_1=Dense(2 => 3), bn=BatchNorm(3), dense_2=Dense(3 => 5)),
@@ -27,10 +27,10 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
 julia> all(iszero, (ps_new.chain.dense_1.weight, ps_new.chain.dense_1.bias,
                     ps_new.chain.dense_2.weight, ps_new.chain.dense_2.bias,
                     ps_new.dense_3.weight, ps_new.dense_3.bias))
-true
source
`,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
julia
@debug_mode layer kwargs...
Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.
See Lux.Experimental.DebugLayer for details about the Keyword Arguments.
source
',4))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
julia
DebugLayer(layer::AbstractLuxLayer;
+true
source
`,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",c,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
julia
@debug_mode layer kwargs...
Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.
See Lux.Experimental.DebugLayer for details about the Keyword Arguments.
source
',4))]),i("details",y,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
julia
DebugLayer(layer::AbstractLuxLayer;
     nan_check::Union{Symbol, StaticSymbol, Val}=static(:both),
     error_check::Union{StaticBool, Bool, Val{true}, Val{false}}=True(),
-    location::KeyPath=KeyPath())
A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.
Arguments
  • layer: The layer to be wrapped.
Extended Help
Keyword Arguments
  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.
  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.
  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.
Input / Output
Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.
If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.
ChainRules Compatible Reverse Mode AD Tools
nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.
Disable After Debugging
This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.
See Lux.Experimental.@debug_mode to construct this layer.
source
`,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
julia
share_parameters(ps, sharing)
+    location::KeyPath=KeyPath())
A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.
Arguments
  • layer: The layer to be wrapped.
Extended Help
Keyword Arguments
  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.
  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.
  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.
Input / Output
Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.
If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.
ChainRules Compatible Reverse Mode AD Tools
nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.
Disable After Debugging
This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.
See Lux.Experimental.@debug_mode to construct this layer.
source
`,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
julia
share_parameters(ps, sharing)
 share_parameters(ps, sharing, new_parameters)
Updates the parameters in ps with a common set of parameters new_parameters that are shared between each list in the nested list sharing. (That was kind of a mouthful, the example should make it clear).
Arguments
  • ps: Original parameters.
  • sharing: A nested list of lists of accessors of ps which need to shate the parameters (See the example for details). (Each list in the list must be disjoint)
  • new_parameters: If passed the length of new_parameters must be equal to the length of sharing. For each vector in sharing the corresponding parameter in new_parameters will be used. (If not passed, the parameters corresponding to the first element of each vector in sharing will be used).
Returns
Updated Parameters having the same structure as ps.
Example
julia
julia> model = Chain(; d1=Dense(2 => 4, tanh),
            d3=Chain(; l1=Dense(4 => 2), l2=Dense(2 => 4)), d2=Dense(4 => 2))
 Chain(
@@ -52,4 +52,4 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
            ps.d3.l2.bias === ps.d1.bias &&
            ps.d2.weight === ps.d3.l1.weight &&
            ps.d2.bias === ps.d3.l1.bias
-true
ComponentArrays
ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.
source
`,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default};
+true

ComponentArrays

ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

source

`,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default}; diff --git a/dev/assets/api_Lux_contrib.md.BRGt7rCL.lean.js b/dev/assets/api_Lux_contrib.md.BHxLoP1M.lean.js similarity index 97% rename from dev/assets/api_Lux_contrib.md.BRGt7rCL.lean.js rename to dev/assets/api_Lux_contrib.md.BHxLoP1M.lean.js index 0874ab47b1..c2ab28acb5 100644 --- a/dev/assets/api_Lux_contrib.md.BRGt7rCL.lean.js +++ b/dev/assets/api_Lux_contrib.md.BHxLoP1M.lean.js @@ -1,6 +1,6 @@ -import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Experimental Features","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/contrib.md","filePath":"api/Lux/contrib.md","lastUpdated":null}'),r={name:"api/Lux/contrib.md"},d={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"};function F(m,s,b,C,f,D){const t=p("Badge");return k(),l("div",null,[s[21]||(s[21]=e('

Experimental Features ​

All features listed on this page are experimental which means:

  1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.

  2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.

  3. None of the features are exported.

Parameter Freezing ​

',4)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Lux.Experimental.FrozenLayer",href:"#Lux.Experimental.FrozenLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.FrozenLayer")],-1)),s[1]||(s[1]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[2]||(s[2]=e(`
julia
FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})

Freeze the parameters with name which_params of the layer l.

Use Lux.Experimental.freeze instead

It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.

No checks for which_params

There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.

Arguments

Extended Help

Parameters

States

Note on Internal Layer Implementation

The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.

Example

julia
julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
-FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

source

`,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

source

julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
-    which_params::Optional{Tuple} = nothing)

Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

source

`,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
julia
unfreeze(l::FrozenLayer)

Unfreezes the layer l.

source

julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)

Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

source

',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/dev/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

KeyPath provided to the function

The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

Call Signature for f

Extended Help

Example

julia
julia> using Lux, Random
+import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Experimental Features","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/contrib.md","filePath":"api/Lux/contrib.md","lastUpdated":null}'),r={name:"api/Lux/contrib.md"},d={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"};function F(m,s,C,b,D,f){const t=p("Badge");return k(),l("div",null,[s[21]||(s[21]=e('
Experimental Features ​
All features listed on this page are experimental which means:
  1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.
  2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.
  3. None of the features are exported.
Parameter Freezing ​
',4)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Lux.Experimental.FrozenLayer",href:"#Lux.Experimental.FrozenLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.FrozenLayer")],-1)),s[1]||(s[1]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[2]||(s[2]=e(`
julia
FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})
Freeze the parameters with name which_params of the layer l.
Use Lux.Experimental.freeze instead
It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.
No checks for which_params
There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.
Arguments
  • l: Lux AbstractLuxLayer.
  • which_params: Parameter Names to be Frozen. Can be set to nothing, in which case all parameters are frozen.
Extended Help
Parameters
  • Parameters of the layer l excluding which_params.
States
  • frozen_params: Parameters that are frozen, i.e., which_params.
  • states: The state of the inner layer l.
Note on Internal Layer Implementation
The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.
Example
julia
julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
+FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable
See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.
source
`,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)
Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.
source
julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
+    which_params::Optional{Tuple} = nothing)
Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.
source
`,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
julia
unfreeze(l::FrozenLayer)
Unfreezes the layer l.
source
julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)
Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.
source
',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/dev/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)
Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.
KeyPath provided to the function
The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).
Call Signature for f
  • Must take 4 inputs – AbstractLuxLayer, Corresponding Parameters, Corresponding States, and the Functors.KeyPath to the layer.
  • Must return a tuple of 3 elements – AbstractLuxLayer, new parameters and the new states.
Extended Help
Example
julia
julia> using Lux, Random
 
 julia> c = Parallel(
            +; chain=Chain(; dense_1=Dense(2 => 3), bn=BatchNorm(3), dense_2=Dense(3 => 5)),
@@ -27,10 +27,10 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
 julia> all(iszero, (ps_new.chain.dense_1.weight, ps_new.chain.dense_1.bias,
                     ps_new.chain.dense_2.weight, ps_new.chain.dense_2.bias,
                     ps_new.dense_3.weight, ps_new.dense_3.bias))
-true
source
`,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
julia
@debug_mode layer kwargs...
Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.
See Lux.Experimental.DebugLayer for details about the Keyword Arguments.
source
',4))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
julia
DebugLayer(layer::AbstractLuxLayer;
+true
source
`,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",c,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
julia
@debug_mode layer kwargs...
Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.
See Lux.Experimental.DebugLayer for details about the Keyword Arguments.
source
',4))]),i("details",y,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
julia
DebugLayer(layer::AbstractLuxLayer;
     nan_check::Union{Symbol, StaticSymbol, Val}=static(:both),
     error_check::Union{StaticBool, Bool, Val{true}, Val{false}}=True(),
-    location::KeyPath=KeyPath())
A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.
Arguments
  • layer: The layer to be wrapped.
Extended Help
Keyword Arguments
  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.
  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.
  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.
Input / Output
Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.
If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.
ChainRules Compatible Reverse Mode AD Tools
nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.
Disable After Debugging
This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.
See Lux.Experimental.@debug_mode to construct this layer.
source
`,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
julia
share_parameters(ps, sharing)
+    location::KeyPath=KeyPath())
A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.
Arguments
  • layer: The layer to be wrapped.
Extended Help
Keyword Arguments
  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.
  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.
  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.
Input / Output
Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.
If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.
ChainRules Compatible Reverse Mode AD Tools
nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.
Disable After Debugging
This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.
See Lux.Experimental.@debug_mode to construct this layer.
source
`,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
julia
share_parameters(ps, sharing)
 share_parameters(ps, sharing, new_parameters)
Updates the parameters in ps with a common set of parameters new_parameters that are shared between each list in the nested list sharing. (That was kind of a mouthful, the example should make it clear).
Arguments
  • ps: Original parameters.
  • sharing: A nested list of lists of accessors of ps which need to shate the parameters (See the example for details). (Each list in the list must be disjoint)
  • new_parameters: If passed the length of new_parameters must be equal to the length of sharing. For each vector in sharing the corresponding parameter in new_parameters will be used. (If not passed, the parameters corresponding to the first element of each vector in sharing will be used).
Returns
Updated Parameters having the same structure as ps.
Example
julia
julia> model = Chain(; d1=Dense(2 => 4, tanh),
            d3=Chain(; l1=Dense(4 => 2), l2=Dense(2 => 4)), d2=Dense(4 => 2))
 Chain(
@@ -52,4 +52,4 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
            ps.d3.l2.bias === ps.d1.bias &&
            ps.d2.weight === ps.d3.l1.weight &&
            ps.d2.bias === ps.d3.l1.bias
-true
ComponentArrays
ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.
source
`,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default};
+true

ComponentArrays

ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

source

`,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default}; diff --git a/dev/assets/api_Lux_distributed_utils.md.DGZwfoze.js b/dev/assets/api_Lux_distributed_utils.md.BqeyCLT9.js similarity index 93% rename from dev/assets/api_Lux_distributed_utils.md.DGZwfoze.js rename to dev/assets/api_Lux_distributed_utils.md.BqeyCLT9.js index d7dc3b4b7d..ec46676a92 100644 --- a/dev/assets/api_Lux_distributed_utils.md.DGZwfoze.js +++ b/dev/assets/api_Lux_distributed_utils.md.BqeyCLT9.js @@ -1,4 +1,4 @@ -import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.CCjWn1F9.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

',3)),s("details",c,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

',3))]),s("details",k,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

source

',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",f,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
-allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",m,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
-bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

`,3))]),s("details",E,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
-reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",C,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

source

',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

',4))])])}const z=n(o,[["render",F]]);export{I as __pageData,z as default}; +import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.CCjWn1F9.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

',3)),s("details",c,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

',3))]),s("details",k,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

source

',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",m,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
+allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",E,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
+bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

`,3))]),s("details",C,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
+reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",f,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

source

',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

',4))])])}const z=n(o,[["render",F]]);export{I as __pageData,z as default}; diff --git a/dev/assets/api_Lux_distributed_utils.md.DGZwfoze.lean.js b/dev/assets/api_Lux_distributed_utils.md.BqeyCLT9.lean.js similarity index 93% rename from dev/assets/api_Lux_distributed_utils.md.DGZwfoze.lean.js rename to dev/assets/api_Lux_distributed_utils.md.BqeyCLT9.lean.js index d7dc3b4b7d..ec46676a92 100644 --- a/dev/assets/api_Lux_distributed_utils.md.DGZwfoze.lean.js +++ b/dev/assets/api_Lux_distributed_utils.md.BqeyCLT9.lean.js @@ -1,4 +1,4 @@ -import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.CCjWn1F9.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

',3)),s("details",c,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

',3))]),s("details",k,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

source

',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",f,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
-allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",m,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
-bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

`,3))]),s("details",E,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
-reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",C,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

source

',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

',4))])])}const z=n(o,[["render",F]]);export{I as __pageData,z as default}; +import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.CCjWn1F9.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

',3)),s("details",c,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

',3))]),s("details",k,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

source

',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",m,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
+allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",E,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
+bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

`,3))]),s("details",C,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
+reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

`,4))]),s("details",f,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

source

',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

',4))])])}const z=n(o,[["render",F]]);export{I as __pageData,z as default}; diff --git a/dev/assets/api_Lux_interop.md.CseYUMhv.js b/dev/assets/api_Lux_interop.md.DN6M7ehc.js similarity index 96% rename from dev/assets/api_Lux_interop.md.CseYUMhv.js rename to dev/assets/api_Lux_interop.md.DN6M7ehc.js index 5cf12f30db..1c8b29e4fe 100644 --- a/dev/assets/api_Lux_interop.md.CseYUMhv.js +++ b/dev/assets/api_Lux_interop.md.DN6M7ehc.js @@ -1,4 +1,4 @@ -import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('

Interoperability between Lux and other packages ​

Switching from older frameworks ​

Flux Models to Lux Models ​

Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.

Tip

Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.

',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
julia
Adapt.adapt(from::FromFluxAdaptor, L)

Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.

source

',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)

Convert a Flux Model to Lux Model.

active field

This always ignores the active field of some of the Flux layers. This is almost never going to be supported.

Keyword Arguments

Example

julia
julia> import Flux
+import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('
Interoperability between Lux and other packages ​
Switching from older frameworks ​
Flux Models to Lux Models ​
Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.
Tip
Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.
',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
julia
Adapt.adapt(from::FromFluxAdaptor, L)
Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.
source
',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)
Convert a Flux Model to Lux Model.
active field
This always ignores the active field of some of the Flux layers. This is almost never going to be supported.
Keyword Arguments
  • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.
  • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.
Example
julia
julia> import Flux
 
 julia> using Adapt, Lux, Random
 
@@ -11,7 +11,7 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> ps, st = Lux.setup(Random.default_rng(), m2);
 
 julia> size(first(m2(x, ps, st)))
-(2, 32)
source
`,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
',9))]),s[19]||(s[19]=a('
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
',3))]),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
+(2, 32)
source
`,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
',9))]),s[19]||(s[19]=a('
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
',3))]),i("details",c,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
 
 julia> using Adapt, Lux, Random
 
@@ -28,5 +28,5 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> x = randn(Float32, 28, 28, 1, 1);
 
 julia> size(first(simple_chains_model(x, ps, st)))
-(10, 1)
source
`,9))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
-SimpleChainsLayer(layer, lux_layer, to_array)
Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.
ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.
Arguments
  • layer: SimpleChains layer
  • lux_layer: Potentially equivalent Lux layer that is used for printing
source
`,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default};
+(10, 1)

source

`,9))]),i("details",y,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
+SimpleChainsLayer(layer, lux_layer, to_array)

Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.

ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.

Arguments

source

`,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default}; diff --git a/dev/assets/api_Lux_interop.md.CseYUMhv.lean.js b/dev/assets/api_Lux_interop.md.DN6M7ehc.lean.js similarity index 96% rename from dev/assets/api_Lux_interop.md.CseYUMhv.lean.js rename to dev/assets/api_Lux_interop.md.DN6M7ehc.lean.js index 5cf12f30db..1c8b29e4fe 100644 --- a/dev/assets/api_Lux_interop.md.CseYUMhv.lean.js +++ b/dev/assets/api_Lux_interop.md.DN6M7ehc.lean.js @@ -1,4 +1,4 @@ -import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('

Interoperability between Lux and other packages ​

Switching from older frameworks ​

Flux Models to Lux Models ​

Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.

Tip

Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.

',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
julia
Adapt.adapt(from::FromFluxAdaptor, L)

Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.

source

',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)

Convert a Flux Model to Lux Model.

active field

This always ignores the active field of some of the Flux layers. This is almost never going to be supported.

Keyword Arguments

Example

julia
julia> import Flux
+import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.CCjWn1F9.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('
Interoperability between Lux and other packages ​
Switching from older frameworks ​
Flux Models to Lux Models ​
Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.
Tip
Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.
',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
julia
Adapt.adapt(from::FromFluxAdaptor, L)
Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.
source
',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)
Convert a Flux Model to Lux Model.
active field
This always ignores the active field of some of the Flux layers. This is almost never going to be supported.
Keyword Arguments
  • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.
  • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.
Example
julia
julia> import Flux
 
 julia> using Adapt, Lux, Random
 
@@ -11,7 +11,7 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> ps, st = Lux.setup(Random.default_rng(), m2);
 
 julia> size(first(m2(x, ps, st)))
-(2, 32)
source
`,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
',9))]),s[19]||(s[19]=a('
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
',3))]),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
+(2, 32)
source
`,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
',9))]),s[19]||(s[19]=a('
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
',3))]),i("details",c,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
 
 julia> using Adapt, Lux, Random
 
@@ -28,5 +28,5 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> x = randn(Float32, 28, 28, 1, 1);
 
 julia> size(first(simple_chains_model(x, ps, st)))
-(10, 1)
source
`,9))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
-SimpleChainsLayer(layer, lux_layer, to_array)
Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.
ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.
Arguments
  • layer: SimpleChains layer
  • lux_layer: Potentially equivalent Lux layer that is used for printing
source
`,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default};
+(10, 1)

source

`,9))]),i("details",y,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
+SimpleChainsLayer(layer, lux_layer, to_array)

Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.

ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.

Arguments

source

`,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default}; diff --git a/dev/assets/api_Lux_layers.md.DxMsNQ36.js b/dev/assets/api_Lux_layers.md.DEbtIDj-.js similarity index 99% rename from dev/assets/api_Lux_layers.md.DxMsNQ36.js rename to dev/assets/api_Lux_layers.md.DEbtIDj-.js index 3c8f80ab9d..b165b56f08 100644 --- a/dev/assets/api_Lux_layers.md.DxMsNQ36.js +++ b/dev/assets/api_Lux_layers.md.DEbtIDj-.js @@ -1,11 +1,11 @@ -import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/framework.CCjWn1F9.js";const o2=JSON.parse('{"title":"Built-In Layers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/layers.md","filePath":"api/Lux/layers.md","lastUpdated":null}'),o={name:"api/Lux/layers.md"},r={class:"jldocstring custom-block"},p={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},E={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},j={class:"jldocstring custom-block"},A={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},V={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},O={class:"jldocstring custom-block"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-5.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"51.473ex",height:"11.422ex",role:"img",focusable:"false",viewBox:"0 -2774.4 22750.9 5048.7","aria-hidden":"true"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"15.326ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 6774.2 1000","aria-hidden":"true"},U={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"16.435ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 7264.2 1000","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.831ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5229.2 1000","aria-hidden":"true"},$={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},t1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},a1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},s1={class:"jldocstring custom-block"},i1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},e1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-8.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.257ex",height:"17.424ex",role:"img",focusable:"false",viewBox:"0 -4100.7 17793.6 7701.4","aria-hidden":"true"},l1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},n1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},Q1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},d1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.018ex",height:"1.357ex",role:"img",focusable:"false",viewBox:"0 -442 1776.1 599.8","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"19.753ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 8730.9 1045","aria-hidden":"true"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},m1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.231ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 9384.3 1045","aria-hidden":"true"},g1={class:"jldocstring custom-block"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.65ex"},xmlns:"http://www.w3.org/2000/svg",width:"67.61ex",height:"2.347ex",role:"img",focusable:"false",viewBox:"0 -750 29883.5 1037.2","aria-hidden":"true"},u1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},L1={class:"jldocstring custom-block"},b1={class:"jldocstring custom-block"},x1={class:"jldocstring custom-block"},w1={class:"jldocstring custom-block"},H1={class:"jldocstring custom-block"},C1={class:"jldocstring custom-block"},F1={class:"jldocstring custom-block"},D1={class:"jldocstring custom-block"},M1={class:"jldocstring custom-block"},v1={class:"jldocstring custom-block"},Z1={class:"jldocstring custom-block"},j1={class:"jldocstring custom-block"},A1={class:"jldocstring custom-block"},B1={class:"jldocstring custom-block"},V1={class:"jldocstring custom-block"},N1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.059ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10192.1 891","aria-hidden":"true"},O1={class:"jldocstring custom-block"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},P1={class:"jldocstring custom-block"},_1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},W1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},X1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},U1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},$1={class:"jldocstring custom-block"},Y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t2={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.172ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.071ex",height:"4.704ex",role:"img",focusable:"false",viewBox:"0 -1119 4451.6 2079","aria-hidden":"true"},a2={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"};function i2(e2,a,l2,n2,Q2,T2){const e=d("Badge");return Q(),n("div",null,[a[269]||(a[269]=t("h1",{id:"Built-In-Layers",tabindex:"-1"},[s("Built-In Layers "),t("a",{class:"header-anchor",href:"#Built-In-Layers","aria-label":'Permalink to "Built-In Layers {#Built-In-Layers}"'},"​")],-1)),a[270]||(a[270]=t("h2",{id:"containers",tabindex:"-1"},[s("Containers "),t("a",{class:"header-anchor",href:"#containers","aria-label":'Permalink to "Containers"'},"​")],-1)),t("details",r,[t("summary",null,[a[0]||(a[0]=t("a",{id:"Lux.BranchLayer",href:"#Lux.BranchLayer"},[t("span",{class:"jlbinding"},"Lux.BranchLayer")],-1)),a[1]||(a[1]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[2]||(a[2]=i(`
julia
BranchLayer(layers...)
+import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/framework.CCjWn1F9.js";const o2=JSON.parse('{"title":"Built-In Layers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/layers.md","filePath":"api/Lux/layers.md","lastUpdated":null}'),o={name:"api/Lux/layers.md"},r={class:"jldocstring custom-block"},p={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},E={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},j={class:"jldocstring custom-block"},A={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},V={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},O={class:"jldocstring custom-block"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-5.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"51.473ex",height:"11.422ex",role:"img",focusable:"false",viewBox:"0 -2774.4 22750.9 5048.7","aria-hidden":"true"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"15.326ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 6774.2 1000","aria-hidden":"true"},U={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"16.435ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 7264.2 1000","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.831ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5229.2 1000","aria-hidden":"true"},$={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},t1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},a1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},s1={class:"jldocstring custom-block"},i1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},e1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-8.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.257ex",height:"17.424ex",role:"img",focusable:"false",viewBox:"0 -4100.7 17793.6 7701.4","aria-hidden":"true"},l1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},n1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},Q1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},d1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.018ex",height:"1.357ex",role:"img",focusable:"false",viewBox:"0 -442 1776.1 599.8","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"19.753ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 8730.9 1045","aria-hidden":"true"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},m1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.231ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 9384.3 1045","aria-hidden":"true"},g1={class:"jldocstring custom-block"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.65ex"},xmlns:"http://www.w3.org/2000/svg",width:"67.61ex",height:"2.347ex",role:"img",focusable:"false",viewBox:"0 -750 29883.5 1037.2","aria-hidden":"true"},u1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},L1={class:"jldocstring custom-block"},x1={class:"jldocstring custom-block"},b1={class:"jldocstring custom-block"},w1={class:"jldocstring custom-block"},H1={class:"jldocstring custom-block"},C1={class:"jldocstring custom-block"},F1={class:"jldocstring custom-block"},D1={class:"jldocstring custom-block"},M1={class:"jldocstring custom-block"},v1={class:"jldocstring custom-block"},Z1={class:"jldocstring custom-block"},j1={class:"jldocstring custom-block"},A1={class:"jldocstring custom-block"},B1={class:"jldocstring custom-block"},V1={class:"jldocstring custom-block"},N1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.059ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10192.1 891","aria-hidden":"true"},O1={class:"jldocstring custom-block"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},P1={class:"jldocstring custom-block"},_1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},W1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},X1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},U1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},$1={class:"jldocstring custom-block"},Y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t2={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.172ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.071ex",height:"4.704ex",role:"img",focusable:"false",viewBox:"0 -1119 4451.6 2079","aria-hidden":"true"},a2={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"};function i2(e2,a,l2,n2,Q2,T2){const e=d("Badge");return Q(),n("div",null,[a[269]||(a[269]=t("h1",{id:"Built-In-Layers",tabindex:"-1"},[s("Built-In Layers "),t("a",{class:"header-anchor",href:"#Built-In-Layers","aria-label":'Permalink to "Built-In Layers {#Built-In-Layers}"'},"​")],-1)),a[270]||(a[270]=t("h2",{id:"containers",tabindex:"-1"},[s("Containers "),t("a",{class:"header-anchor",href:"#containers","aria-label":'Permalink to "Containers"'},"​")],-1)),t("details",r,[t("summary",null,[a[0]||(a[0]=t("a",{id:"Lux.BranchLayer",href:"#Lux.BranchLayer"},[t("span",{class:"jlbinding"},"Lux.BranchLayer")],-1)),a[1]||(a[1]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[2]||(a[2]=i(`
julia
BranchLayer(layers...)
 BranchLayer(; name=nothing, layers...)
Takes an input x and passes it through all the layers and returns a tuple of the outputs.
Arguments
  • Layers can be specified in two formats: 
    • A list of N Lux layers
    • Specified as N keyword arguments.
Extended Help
Inputs
  • x: Will be directly passed to each of the layers
Returns
  • Tuple: (layer_1(x), layer_2(x), ..., layer_N(x)) (naming changes if using the kwargs API)
  • Updated state of the layers
Parameters
  • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
States
  • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
Comparison with Parallel
This is slightly different from Parallel(nothing, layers...)
  • If the input is a tuple, Parallel will pass each element individually to each layer.
  • BranchLayer essentially assumes 1 input comes in and is branched out into N outputs.
Example
An easy way to replicate an input to an NTuple is to do
julia
julia> BranchLayer(NoOpLayer(), NoOpLayer(), NoOpLayer())
 BranchLayer(
     layer_1 = NoOpLayer(),
     layer_2 = NoOpLayer(),
     layer_3 = NoOpLayer(),
 )         # Total: 0 parameters,
-          #        plus 0 states.
source
`,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
julia
Chain(layers...; name=nothing)
+          #        plus 0 states.
source
`,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
julia
Chain(layers...; name=nothing)
 Chain(; layers..., name=nothing)
Collects multiple layers / functions to be called in sequence on a given input.
Arguments
  • Layers can be specified in two formats: 
    • A list of N Lux layers
    • Specified as N keyword arguments.
Extended Help
Inputs
Input x is passed sequentially to each layer, and must conform to the input requirements of the internal layers.
Returns
  • Output after sequentially applying all the layers to x
  • Updated model states
Parameters
  • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
States
  • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
Miscellaneous Properties
  • Allows indexing and field access syntax. We can access the ith layer by m[i] or m.layer_i. We can also index using ranges or arrays.
Example
julia
julia> Chain(Dense(2, 3, relu), BatchNorm(3), Dense(3, 2))
 Chain(
     layer_1 = Dense(2 => 3, relu),      # 9 parameters
@@ -20,7 +20,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_2 = BatchNorm(3, affine=true, track_stats=true),  # 6 parameters, plus 7
     layer_3 = Dense(3 => 2),            # 8 parameters
 )         # Total: 23 parameters,
-          #        plus 7 states.
source
`,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
julia
PairwiseFusion(connection, layers...; name=nothing)
+          #        plus 7 states.
source
`,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
julia
PairwiseFusion(connection, layers...; name=nothing)
 PairwiseFusion(connection; name=nothing, layers...)
 PairwiseFusion(; connection, layers..., name=nothing)
x1 → layer1 → y1 ↘
                   connection → layer2 → y2 ↘
@@ -31,7 +31,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 end
  1. Any other kind of input
julia
y = x
 for i in 1:N
     y = connection(x, layers[i](y))
-end
Returns
  • See Inputs section for how the return value is computed
  • Updated model state for all the contained layers
Parameters
  • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
States
  • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
source
`,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
julia
Parallel(connection, layers...; name=nothing)
+end
Returns
  • See Inputs section for how the return value is computed
  • Updated model state for all the contained layers
Parameters
  • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
States
  • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
source
`,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
julia
Parallel(connection, layers...; name=nothing)
 Parallel(connection; name=nothing, layers...)
 Parallel(; connection, layers..., name=nothing)
Create a layer which passes an input to each path in layers, before reducing the output with connection.
Arguments
  • connection: An N-argument function that is called after passing the input through each layer. If connection = nothing, we return a tuple Parallel(nothing, f, g)(x, y) = (f(x), g(y))
  • Layers can be specified in two formats:
    • A list of N Lux layers
    • Specified as N keyword arguments.
Extended Help
Inputs
  • x: If x is not a tuple, then return is computed as connection([l(x) for l in layers]...). Else one is passed to each layer, thus Parallel(+, f, g)(x, y) = f(x) + g(y).
Returns
  • See the Inputs section for how the output is computed
  • Updated state of the layers
Parameters
  • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
States
  • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
See also SkipConnection which is Parallel with one identity.
Example
julia
julia> model = Parallel(nothing, Dense(2, 1), Dense(2, 1))
 Parallel(
@@ -47,37 +47,37 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x2 = randn(rng, Float32, 2);
 
 julia> size.(first(model((x1, x2), ps, st)))
-((1,), (1,))
source
`,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
julia
SkipConnection(layers, connection; name=nothing)
-SkipConnection(; layers, connection, name=nothing)
Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.
The simplest "ResNet"-type connection is just SkipConnection(layer, +).
Arguments
  • layer: Layer or Chain of layers to be applied to the input
  • connection:
    • A 2-argument function that takes layer(input) and the input OR
    • An AbstractLuxLayer that takes (layer(input), input) as input
Extended Help
Inputs
  • x: Will be passed directly to layer
Returns
  • Output of connection(layer(input), input)
  • Updated state of layer
Parameters
  • Parameters of layer OR
  • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
States
  • States of layer OR
  • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
See Parallel for a more general implementation.
source
`,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
julia
RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))
Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:
  • input_injection = Val(false)
julia
res = x
+((1,), (1,))
source
`,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
julia
SkipConnection(layers, connection; name=nothing)
+SkipConnection(; layers, connection, name=nothing)
Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.
The simplest "ResNet"-type connection is just SkipConnection(layer, +).
Arguments
  • layer: Layer or Chain of layers to be applied to the input
  • connection:
    • A 2-argument function that takes layer(input) and the input OR
    • An AbstractLuxLayer that takes (layer(input), input) as input
Extended Help
Inputs
  • x: Will be passed directly to layer
Returns
  • Output of connection(layer(input), input)
  • Updated state of layer
Parameters
  • Parameters of layer OR
  • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
States
  • States of layer OR
  • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
See Parallel for a more general implementation.
source
`,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
julia
RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))
Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:
  • input_injection = Val(false)
julia
res = x
 for i in 1:repeats
     res, st = model(res, ps, st)
 end
  • input_injection = Val(true)
julia
res = x
 for i in 1:repeats
     res, st = model((res, x), ps, st)
-end
It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.
Arguments
  • model must be an AbstractLuxLayer
Keyword Arguments
  • repeats: Number of times to apply the model
  • input_injection: If true, then the input is passed to the model along with the output
Extended Help
Inputs
  • x: Input as described above
Returns
  • Output is computed by as described above
  • Updated state of the model
Parameters
  • Parameters of model
States
  • State of model
source
`,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
julia
Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+end
It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.
Arguments
  • model must be an AbstractLuxLayer
Keyword Arguments
  • repeats: Number of times to apply the model
  • input_injection: If true, then the input is passed to the model along with the output
Extended Help
Inputs
  • x: Input as described above
Returns
  • Output is computed by as described above
  • Updated state of the model
Parameters
  • Parameters of model
States
  • State of model
source
`,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
julia
Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
      activation=identity; init_weight=nothing, init_bias=nothing, stride=1,
-     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())
Standard convolutional layer.
Conv 2D
Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.
Warning
Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.
Arguments
  • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
  • in_chs: Number of input channels
  • out_chs: Number of input and output channels
  • activation: Activation Function
Extended Help
Keyword Arguments
  • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
  • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    • Periodic padding can achieved by pre-empting the layer with a WrappedFunction(x -> NNlib.circular_pad(x, N_pad; dims=pad_dims))
  • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
  • use_bias: Trainable bias can be disabled entirely by setting this to false.
  • cross_correlation: If true, perform cross-correlation instead of convolution. Prior to v1, Lux used to have a CrossCor layer which performed cross-correlation. This was removed in v1 in favor of Conv with cross_correlation=true.
Inputs
  • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
Returns
  • Output of the convolution y of size (O_N, ..., O_1, C_out, N) where
`,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('
  • Empty NamedTuple()
Parameters
  • weight: Convolution kernel
  • bias: Bias (present if use_bias=true)
source
',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
julia
ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())
Standard convolutional layer.
Conv 2D
Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.
Warning
Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.
Arguments
  • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
  • in_chs: Number of input channels
  • out_chs: Number of input and output channels
  • activation: Activation Function
Extended Help
Keyword Arguments
  • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
  • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    • Periodic padding can achieved by pre-empting the layer with a WrappedFunction(x -> NNlib.circular_pad(x, N_pad; dims=pad_dims))
  • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
  • use_bias: Trainable bias can be disabled entirely by setting this to false.
  • cross_correlation: If true, perform cross-correlation instead of convolution. Prior to v1, Lux used to have a CrossCor layer which performed cross-correlation. This was removed in v1 in favor of Conv with cross_correlation=true.
Inputs
  • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
Returns
  • Output of the convolution y of size (O_N, ..., O_1, C_out, N) where
`,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('
  • Empty NamedTuple()
Parameters
  • weight: Convolution kernel
  • bias: Bias (present if use_bias=true)
source
',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
julia
ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
               activation=identity; init_weight=glorot_uniform, init_bias=zeros32,
               stride=1, pad=0, outpad=0, dilation=1, groups=1, use_bias=True(),
-              cross_correlation=False())
Standard convolutional transpose layer.
Arguments
  • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
  • in_chs: Number of input channels
  • out_chs: Number of input and output channels
  • activation: Activation Function
Keyword Arguments
  • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
  • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) * stride (possibly rounded) for each spatial dimension.
  • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
  • use_bias: Trainable bias can be disabled entirely by setting this to false.
  • cross_correlation: If true, perform transposed cross-correlation instead of transposed convolution.
  • outpad: To converse Conv inversability when stride > 1, outpad can be used to increase the size of the output in the desired dimensions. Whereas pad is used to zero-pad the input, outpad only affects the output shape.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
Returns
  • Output of the convolution transpose y of size (O_N, ..., O_1, C_out, N) where
  • Empty NamedTuple()
Parameters
  • weight: Convolution Transpose kernel
  • bias: Bias (present if use_bias=true)
source
`,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
julia
AlphaDropout(p::Real)
AlphaDropout layer.
Arguments
  • p: Probability of Dropout 
    • if p = 0 then NoOpLayer is returned.
    • if p = 1 then WrappedLayer(Base.Fix1(broadcast, zero)) is returned.
Inputs
  • x: Must be an AbstractArray
Returns
  • x with dropout mask applied if training=Val(true) else just x
  • State with updated rng
States
  • rng: Pseudo Random Number Generator
  • training: Used to check if training/inference mode
Call Lux.testmode to switch to test mode.
See also Dropout, VariationalHiddenDropout
source
',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
julia
Dropout(p; dims=:)
Dropout layer.
Arguments
  • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
Keyword Arguments
  • To apply dropout along certain dimension(s), specify the dims keyword. e.g. Dropout(p; dims = (3,4)) will randomly zero out entire channels on WHCN input (also called 2D dropout).
Inputs
  • x: Must be an AbstractArray
Returns
  • x with dropout mask applied if training=Val(true) else just x
  • State with updated rng
States
  • rng: Pseudo Random Number Generator
  • training: Used to check if training/inference mode
Call Lux.testmode to switch to test mode.
See also AlphaDropout, VariationalHiddenDropout
source
',15))]),t("details",b,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
julia
VariationalHiddenDropout(p; dims=:)
VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.
Arguments
  • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
Keyword Arguments
  • To apply dropout along certain dimension(s), specify the dims keyword. e.g. VariationalHiddenDropout(p; dims = 3) will randomly zero out entire channels on WHCN input (also called 2D dropout).
Inputs
  • x: Must be an AbstractArray
Returns
  • x with dropout mask applied if training=Val(true) else just x
  • State with updated rng
States
  • rng: Pseudo Random Number Generator
  • training: Used to check if training/inference mode
  • mask: Dropout mask. Initilly set to nothing. After every run, contains the mask applied in that call
  • update_mask: Stores whether new mask needs to be generated in the current call
Call Lux.testmode to switch to test mode.
See also AlphaDropout, Dropout
source
',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",x,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
julia
AdaptiveLPPool(output_size; p=2)
Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
Arguments
  • output_size: Size of the first N dimensions for the output
GPU Support
This layer is currently only supported on CPU.
Inputs
  • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
Returns
  • Output of size (out..., C, N)
  • Empty NamedTuple()
source
',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
julia
AdaptiveMaxPool(output_size)
Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
Arguments
  • output_size: Size of the first N dimensions for the output
Inputs
  • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
Returns
  • Output of size (out..., C, N)
  • Empty NamedTuple()
source
',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
julia
AdaptiveMeanPool(output_size)
Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
Arguments
  • output_size: Size of the first N dimensions for the output
Inputs
  • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
Returns
  • Output of size (out..., C, N)
  • Empty NamedTuple()
source
',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
julia
GlobalLPPool(; p=2)
Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
GPU Support
This layer is currently only supported on CPU.
Inputs
  • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (1, ..., 1, C, N)
  • Empty NamedTuple()
source
',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
julia
GlobalMaxPool()
Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
Inputs
  • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (1, ..., 1, C, N)
  • Empty NamedTuple()
source
',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
julia
GlobalMeanPool()
Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
Inputs
  • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (1, ..., 1, C, N)
  • Empty NamedTuple()
source
',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
julia
LPPool(window; stride=window, pad=0, dilation=1, p=2)
LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.
Arguments
  • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
Keyword Arguments
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
GPU Support
This layer is currently only supported on CPU.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (O_N, ..., O_1, C, N) where
',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
julia
MaxPool(window; stride=window, pad=0, dilation=1)
Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.
Arguments
  • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
Keyword Arguments
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (O_N, ..., O_1, C, N) where
',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
julia
MeanPool(window; stride=window, pad=0, dilation=1)
Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.
Arguments
  • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
Keyword Arguments
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (O_N, ..., O_1, C, N) where
',11)),t("mjx-container",N,[(Q(),n("svg",R,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
julia
GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
-        init_weight=nothing, init_bias=nothing, init_state=zeros32)
Gated Recurrent Unit (GRU) Cell
`,2)),t("mjx-container",z,[(Q(),n("svg",I,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("
Arguments
  • in_dims: Input Dimension
  • out_dims: Output (Hidden State) Dimension
  • use_bias: Set to false to deactivate bias
  • train_state: Trainable initial hidden state can be activated by setting this to true
  • init_bias: Initializer for bias. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
  • init_weight: Initializer for weight. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
  • init_state: Initializer for hidden state
Inputs
  • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
  • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
  • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
Returns
",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",S,[(Q(),n("svg",P,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
julia
LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
+              cross_correlation=False())
Standard convolutional transpose layer.
Arguments
  • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
  • in_chs: Number of input channels
  • out_chs: Number of input and output channels
  • activation: Activation Function
Keyword Arguments
  • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
  • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) * stride (possibly rounded) for each spatial dimension.
  • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
  • use_bias: Trainable bias can be disabled entirely by setting this to false.
  • cross_correlation: If true, perform transposed cross-correlation instead of transposed convolution.
  • outpad: To converse Conv inversability when stride > 1, outpad can be used to increase the size of the output in the desired dimensions. Whereas pad is used to zero-pad the input, outpad only affects the output shape.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
Returns
  • Output of the convolution transpose y of size (O_N, ..., O_1, C_out, N) where
  • Empty NamedTuple()
Parameters
  • weight: Convolution Transpose kernel
  • bias: Bias (present if use_bias=true)
source
`,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
julia
AlphaDropout(p::Real)
AlphaDropout layer.
Arguments
  • p: Probability of Dropout 
    • if p = 0 then NoOpLayer is returned.
    • if p = 1 then WrappedLayer(Base.Fix1(broadcast, zero)) is returned.
Inputs
  • x: Must be an AbstractArray
Returns
  • x with dropout mask applied if training=Val(true) else just x
  • State with updated rng
States
  • rng: Pseudo Random Number Generator
  • training: Used to check if training/inference mode
Call Lux.testmode to switch to test mode.
See also Dropout, VariationalHiddenDropout
source
',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
julia
Dropout(p; dims=:)
Dropout layer.
Arguments
  • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
Keyword Arguments
  • To apply dropout along certain dimension(s), specify the dims keyword. e.g. Dropout(p; dims = (3,4)) will randomly zero out entire channels on WHCN input (also called 2D dropout).
Inputs
  • x: Must be an AbstractArray
Returns
  • x with dropout mask applied if training=Val(true) else just x
  • State with updated rng
States
  • rng: Pseudo Random Number Generator
  • training: Used to check if training/inference mode
Call Lux.testmode to switch to test mode.
See also AlphaDropout, VariationalHiddenDropout
source
',15))]),t("details",x,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
julia
VariationalHiddenDropout(p; dims=:)
VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.
Arguments
  • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
Keyword Arguments
  • To apply dropout along certain dimension(s), specify the dims keyword. e.g. VariationalHiddenDropout(p; dims = 3) will randomly zero out entire channels on WHCN input (also called 2D dropout).
Inputs
  • x: Must be an AbstractArray
Returns
  • x with dropout mask applied if training=Val(true) else just x
  • State with updated rng
States
  • rng: Pseudo Random Number Generator
  • training: Used to check if training/inference mode
  • mask: Dropout mask. Initilly set to nothing. After every run, contains the mask applied in that call
  • update_mask: Stores whether new mask needs to be generated in the current call
Call Lux.testmode to switch to test mode.
See also AlphaDropout, Dropout
source
',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
julia
AdaptiveLPPool(output_size; p=2)
Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
Arguments
  • output_size: Size of the first N dimensions for the output
GPU Support
This layer is currently only supported on CPU.
Inputs
  • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
Returns
  • Output of size (out..., C, N)
  • Empty NamedTuple()
source
',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
julia
AdaptiveMaxPool(output_size)
Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
Arguments
  • output_size: Size of the first N dimensions for the output
Inputs
  • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
Returns
  • Output of size (out..., C, N)
  • Empty NamedTuple()
source
',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
julia
AdaptiveMeanPool(output_size)
Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
Arguments
  • output_size: Size of the first N dimensions for the output
Inputs
  • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
Returns
  • Output of size (out..., C, N)
  • Empty NamedTuple()
source
',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
julia
GlobalLPPool(; p=2)
Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
GPU Support
This layer is currently only supported on CPU.
Inputs
  • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (1, ..., 1, C, N)
  • Empty NamedTuple()
source
',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
julia
GlobalMaxPool()
Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
Inputs
  • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (1, ..., 1, C, N)
  • Empty NamedTuple()
source
',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
julia
GlobalMeanPool()
Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
Inputs
  • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (1, ..., 1, C, N)
  • Empty NamedTuple()
source
',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
julia
LPPool(window; stride=window, pad=0, dilation=1, p=2)
LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.
Arguments
  • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
Keyword Arguments
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
GPU Support
This layer is currently only supported on CPU.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (O_N, ..., O_1, C, N) where
',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
julia
MaxPool(window; stride=window, pad=0, dilation=1)
Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.
Arguments
  • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
Keyword Arguments
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (O_N, ..., O_1, C, N) where
',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
julia
MeanPool(window; stride=window, pad=0, dilation=1)
Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.
Arguments
  • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
Keyword Arguments
  • stride: Should each be either single integer, or a tuple with N integers
  • dilation: Should each be either single integer, or a tuple with N integers
  • pad: Specifies the number of elements added to the borders of the data array. It can be
    • a single integer for equal padding all around,
    • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
    • a tuple of 2*N integers, for asymmetric padding, or
    • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
Extended Help
Inputs
  • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
Returns
  • Output of the pooling y of size (O_N, ..., O_1, C, N) where
',11)),t("mjx-container",N,[(Q(),n("svg",R,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
julia
GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
+        init_weight=nothing, init_bias=nothing, init_state=zeros32)
Gated Recurrent Unit (GRU) Cell
`,2)),t("mjx-container",z,[(Q(),n("svg",I,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("
Arguments
  • in_dims: Input Dimension
  • out_dims: Output (Hidden State) Dimension
  • use_bias: Set to false to deactivate bias
  • train_state: Trainable initial hidden state can be activated by setting this to true
  • init_bias: Initializer for bias. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
  • init_weight: Initializer for weight. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
  • init_state: Initializer for hidden state
Inputs
  • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
  • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
  • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
Returns
",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",S,[(Q(),n("svg",P,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
julia
LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
          train_memory::Bool=false, init_weight=nothing, init_bias=nothing,
-         init_state=zeros32, init_memory=zeros32)
Long Short-Term (LSTM) Cell
`,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("
Arguments
  • in_dims: Input Dimension
  • out_dims: Output (Hidden State & Memory) Dimension
  • use_bias: Set to false to deactivate bias
  • train_state: Trainable initial hidden state can be activated by setting this to true
  • train_memory: Trainable initial memory can be activated by setting this to true
  • init_bias: Initializer for bias. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
  • init_weight: Initializer for weight. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
  • init_state: Initializer for hidden state
  • init_memory: Initializer for memory
Inputs
  • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to false - Creates a hidden state using init_state, hidden memory using init_memory and proceeds to Case 2.
  • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to false - Repeats hidden_state vector from the parameters to match the shape of x, creates hidden memory using init_memory and proceeds to Case 2.
  • Case 1c: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to true - Creates a hidden state using init_state, repeats the memory vector from parameters to match the shape of x and proceeds to Case 2.
  • Case 1d: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to true - Repeats the hidden state and memory vectors from the parameters to match the shape of x and proceeds to Case 2.
  • Case 2: Tuple (x, (h, c)) is provided, then the output and a tuple containing the updated hidden state and memory is returned.
Returns
",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("
  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)
  • hidden_state: Initial hidden state vector (not present if train_state=false)
  • memory: Initial memory vector (not present if train_memory=false)
    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
-    init_bias=nothing, init_weight=nothing, init_state=zeros32)
    An Elman RNNCell cell with activation (typically set to tanh or relu).
    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State) Dimension
    • activation: Activation function
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • init_bias: Initializer for bias. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('
    Parameters
    • weight_ih: Maps the input to the hidden state.
    • weight_hh: Maps the hidden state to the hidden state.
    • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
    • bias_hh: Bias vector for the hidden-hidden connection (not present if use_bias=false)
    • hidden_state: Initial hidden state vector (not present if train_state=false)
    States
    • rng: Controls the randomness (if any) in the initial state generation
    source
    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
+         init_state=zeros32, init_memory=zeros32)
    Long Short-Term (LSTM) Cell
    `,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State & Memory) Dimension
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • train_memory: Trainable initial memory can be activated by setting this to true
    • init_bias: Initializer for bias. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    • init_memory: Initializer for memory
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to false - Creates a hidden state using init_state, hidden memory using init_memory and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to false - Repeats hidden_state vector from the parameters to match the shape of x, creates hidden memory using init_memory and proceeds to Case 2.
    • Case 1c: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to true - Creates a hidden state using init_state, repeats the memory vector from parameters to match the shape of x and proceeds to Case 2.
    • Case 1d: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to true - Repeats the hidden state and memory vectors from the parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, c)) is provided, then the output and a tuple containing the updated hidden state and memory is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("
  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)
  • hidden_state: Initial hidden state vector (not present if train_state=false)
  • memory: Initial memory vector (not present if train_memory=false)
    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
+    init_bias=nothing, init_weight=nothing, init_state=zeros32)
    An Elman RNNCell cell with activation (typically set to tanh or relu).
    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State) Dimension
    • activation: Activation function
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • init_bias: Initializer for bias. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('
    Parameters
    • weight_ih: Maps the input to the hidden state.
    • weight_hh: Maps the hidden state to the hidden state.
    • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
    • bias_hh: Bias vector for the hidden-hidden connection (not present if use_bias=false)
    • hidden_state: Initial hidden state vector (not present if train_state=false)
    States
    • rng: Controls the randomness (if any) in the initial state generation
    source
    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
     ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex(),
-    return_sequence::Bool=false)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.
    Relation to Flux.Recur
    This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Keyword Arguments
    • return_sequence: If true returns the entire sequence of outputs, else returns only the last output. Defaults to false.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell.
    Tip
    Frameworks like Tensorflow have special implementation of MultiRNNCell to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.
    Chain(
+    return_sequence::Bool=false)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.
    Relation to Flux.Recur
    This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Keyword Arguments
    • return_sequence: If true returns the entire sequence of outputs, else returns only the last output. Defaults to false.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell.
    Tip
    Frameworks like Tensorflow have special implementation of StackedRNNCells to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.
    Chain(
     Recurrence(RNNCell(inputsize => latentsize); return_sequence=true),
     Recurrence(RNNCell(latentsize => latentsize); return_sequence=true),
     :
     x -> stack(x; dims=2)
-)
    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.
    source
    `,14))]),t("details",b1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.
    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Inputs
    • Input to the cell.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell and updated carry.
    States
    • NamedTuple containing: 
      • cell: Same as cell.
      • carry: The carry state of the cell.
    source
    ',12))]),t("details",x1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
+)
    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.
    source
    `,14))]),t("details",x1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.
    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Inputs
    • Input to the cell.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell and updated carry.
    States
    • NamedTuple containing: 
      • cell: Same as cell.
      • carry: The carry state of the cell.
    source
    ',12))]),t("details",b1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
     backward_cell::Union{AbstractRecurrentCell, Nothing}=nothing;
     merge_mode::Union{Function, Nothing}=vcat,
-    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())
    Bidirectional RNN wrapper.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    • backward_cell: A optional backward recurrent cell. If backward_cell is nothing, the rnn layer instance passed as the cell argument will be used to generate the backward layer automatically. in_dims of backward_cell should be consistent with in_dims of cell
    Keyword Arguments
    • merge_mode: Function by which outputs of the forward and backward RNNs will be combined. default value is vcat. If nothing, the outputs will not be combined.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Merged output of the cell and backward_cell for the entire sequence.
    • Update state of the cell and backward_cell.
    Parameters
    • NamedTuple with cell and backward_cell.
    States
    • Same as cell and backward_cell.
    source
    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
+    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())
    Bidirectional RNN wrapper.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    • backward_cell: A optional backward recurrent cell. If backward_cell is nothing, the rnn layer instance passed as the cell argument will be used to generate the backward layer automatically. in_dims of backward_cell should be consistent with in_dims of cell
    Keyword Arguments
    • merge_mode: Function by which outputs of the forward and backward RNNs will be combined. default value is vcat. If nothing, the outputs will not be combined.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Merged output of the cell and backward_cell for the entire sequence.
    • Update state of the cell and backward_cell.
    Parameters
    • NamedTuple with cell and backward_cell.
    States
    • Same as cell and backward_cell.
    source
    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
          init_bias=nothing, use_bias=True())
 Bilinear(in12_dims => out, activation=identity; init_weight=nothing,
-         init_bias=nothing, use_bias=True())
    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:
    z[i] = activation(x' * W[i, :, :] * y + bias[i])
    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.
    Arguments
    • in1_dims: number of input dimensions of x
    • in2_dims: number of input dimensions of y
    • in12_dims: If specified, then in1_dims = in2_dims = in12_dims
    • out: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in1_dims, in2_dims)). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • A 2-Tuple containing
      • x must be an AbstractArray with size(x, 1) == in1_dims
      • y must be an AbstractArray with size(y, 1) == in2_dims
    • If the input is an AbstractArray, then x = y
    Returns
    • AbstractArray with dimensions (out_dims, size(x, 2))
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in1_dims, in2_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
-      init_bias=nothing, use_bias=True())
    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)
    Arguments
    • in_dims: number of input dimensions
    • out_dims: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims)). If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an AbstractArray with size(x, 1) == in_dims
    Returns
    • AbstractArray with dimensions (out_dims, ...) where ... are the dimensions of x
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)
    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.
    This layer is often used to store word embeddings and retrieve them using indices.
    Arguments
    • in_dims: number(s) of input dimensions
    • out_dims: number of output dimensions
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims...))
    Input
    • Integer OR
    • Abstract Vector of Integers OR
    • Abstract Array of Integers OR
    • Tuple of Integers OR
    • Tuple of Abstract Vectors of Integers OR
    • Tuple of Abstract Arrays of Integers
    Returns
    • Returns the embedding corresponding to each index in the input. For an N dimensional input, an N + 1 dimensional output is returned.
    • Empty NamedTuple()
    source
    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())
    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)
    Arguments
    • dims: size of the learnable scale and bias parameters.
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims))
    • init_bias: initializer for the bias vector (ignored if use_bias=false)
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    Returns
    • Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    • Empty NamedTuple()
    Parameters
    • weight: Weight Array of size (dims...)
    • bias: Bias of size (dims...)
    source
    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)
    Flattens the passed array into a matrix.
    Keyword Arguments
    • N: Flatten the first N dimensions of the input array. If nothing, then all dimensions (except the last) are flattened. Note that the batch dimension is never flattened.
    Inputs
    • x: AbstractArray
    Returns
    • AbstractMatrix of size (:, size(x, ndims(x))) if N is nothing else the first N dimensions of the input array are flattened.
    • Empty NamedTuple()
    Example
    julia
    julia> model = FlattenLayer()
+         init_bias=nothing, use_bias=True())
    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:
    z[i] = activation(x' * W[i, :, :] * y + bias[i])
    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.
    Arguments
    • in1_dims: number of input dimensions of x
    • in2_dims: number of input dimensions of y
    • in12_dims: If specified, then in1_dims = in2_dims = in12_dims
    • out: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in1_dims, in2_dims)). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • A 2-Tuple containing
      • x must be an AbstractArray with size(x, 1) == in1_dims
      • y must be an AbstractArray with size(y, 1) == in2_dims
    • If the input is an AbstractArray, then x = y
    Returns
    • AbstractArray with dimensions (out_dims, size(x, 2))
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in1_dims, in2_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
+      init_bias=nothing, use_bias=True())
    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)
    Arguments
    • in_dims: number of input dimensions
    • out_dims: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims)). If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an AbstractArray with size(x, 1) == in_dims
    Returns
    • AbstractArray with dimensions (out_dims, ...) where ... are the dimensions of x
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)
    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.
    This layer is often used to store word embeddings and retrieve them using indices.
    Arguments
    • in_dims: number(s) of input dimensions
    • out_dims: number of output dimensions
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims...))
    Input
    • Integer OR
    • Abstract Vector of Integers OR
    • Abstract Array of Integers OR
    • Tuple of Integers OR
    • Tuple of Abstract Vectors of Integers OR
    • Tuple of Abstract Arrays of Integers
    Returns
    • Returns the embedding corresponding to each index in the input. For an N dimensional input, an N + 1 dimensional output is returned.
    • Empty NamedTuple()
    source
    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())
    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)
    Arguments
    • dims: size of the learnable scale and bias parameters.
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims))
    • init_bias: initializer for the bias vector (ignored if use_bias=false)
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    Returns
    • Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    • Empty NamedTuple()
    Parameters
    • weight: Weight Array of size (dims...)
    • bias: Bias of size (dims...)
    source
    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)
    Flattens the passed array into a matrix.
    Keyword Arguments
    • N: Flatten the first N dimensions of the input array. If nothing, then all dimensions (except the last) are flattened. Note that the batch dimension is never flattened.
    Inputs
    • x: AbstractArray
    Returns
    • AbstractMatrix of size (:, size(x, ndims(x))) if N is nothing else the first N dimensions of the input array are flattened.
    • Empty NamedTuple()
    Example
    julia
    julia> model = FlattenLayer()
 FlattenLayer{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -87,9 +87,9 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(8, 2)
    source
    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
+(8, 2)
    source
    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
 Maxout(; layers...)
-Maxout(f::Function, n_alts::Int)
    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.
    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].
    See also Parallel to reduce with other operators.
    Arguments
    • Layers can be specified in three formats: 
      • A list of N Lux layers
      • Specified as N keyword arguments.
      • A no argument function f and an integer n_alts which specifies the number of layers.
    Extended Help
    Inputs
    • x: Input that is passed to each of the layers
    Returns
    • Output is computed by taking elementwise max of the outputs of the individual layers.
    • Updated state of the layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    References
    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389
    source
    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()
    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.
    Example
    julia
    julia> model = NoOpLayer()
+Maxout(f::Function, n_alts::Int)
    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.
    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].
    See also Parallel to reduce with other operators.
    Arguments
    • Layers can be specified in three formats: 
      • A list of N Lux layers
      • Specified as N keyword arguments.
      • A no argument function f and an integer n_alts which specifies the number of layers.
    Extended Help
    Inputs
    • x: Input that is passed to each of the layers
    Returns
    • Output is computed by taking elementwise max of the outputs of the individual layers.
    • Updated state of the layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    References
    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389
    source
    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()
    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.
    Example
    julia
    julia> model = NoOpLayer()
 NoOpLayer()
 
 julia> rng = Random.default_rng();
@@ -99,7 +99,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 1
 
 julia> y, st_new = model(x, ps, st)
-(1, NamedTuple())
    source
    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)
    Reshapes the passed array to have a size of (dims..., :)
    Arguments
    • dims: The new dimensions of the array (excluding the last dimension).
    Inputs
    • x: AbstractArray of any shape which can be reshaped in (dims..., size(x, ndims(x)))
    Returns
    • AbstractArray of size (dims..., size(x, ndims(x)))
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReshapeLayer((2, 2))
+(1, NamedTuple())
    source
    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)
    Reshapes the passed array to have a size of (dims..., :)
    Arguments
    • dims: The new dimensions of the array (excluding the last dimension).
    Inputs
    • x: AbstractArray of any shape which can be reshaped in (dims..., size(x, ndims(x)))
    Returns
    • AbstractArray of size (dims..., size(x, ndims(x)))
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReshapeLayer((2, 2))
 ReshapeLayer(output_dims = (2, 2, :))
 
 julia> rng = Random.default_rng();
@@ -109,7 +109,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(2, 2, 3)
    source
    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)
    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.
    Arguments
    • dim: Dimension for indexing
    • i: Index for dimension dim
    Inputs
    • x: AbstractArray that can be indexed with view(x,:,:,...,i,:,:,...)
    Returns
    • view(x,:,:,...,i,:,:,...) where i is in position d
    • Empty NamedTuple()
    source
    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)
    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))
    Arguments
    • f: Some function.
    Inputs
    • x: will be directly passed to f
    Returns
    • Output of f(x)
    • Empty NamedTuple()
    source
    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)
    Reverse the specified dimension dims of the passed array
    Arguments
    • dim: Dimension that need to be reversed. If nothing, for AbstractVector{T} it reverses itself (dimension 1), for other arrays, reverse the dimension ndims(x) - 1.
    Inputs
    • x: AbstractArray.
    Returns
    • AbstractArray with the same dimensions as the input
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReverseSequence()
+(2, 2, 3)
    source
    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)
    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.
    Arguments
    • dim: Dimension for indexing
    • i: Index for dimension dim
    Inputs
    • x: AbstractArray that can be indexed with view(x,:,:,...,i,:,:,...)
    Returns
    • view(x,:,:,...,i,:,:,...) where i is in position d
    • Empty NamedTuple()
    source
    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)
    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))
    Arguments
    • f: Some function.
    Inputs
    • x: will be directly passed to f
    Returns
    • Output of f(x)
    • Empty NamedTuple()
    source
    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)
    Reverse the specified dimension dims of the passed array
    Arguments
    • dim: Dimension that need to be reversed. If nothing, for AbstractVector{T} it reverses itself (dimension 1), for other arrays, reverse the dimension ndims(x) - 1.
    Inputs
    • x: AbstractArray.
    Returns
    • AbstractArray with the same dimensions as the input
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReverseSequence()
 ReverseSequence{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -118,7 +118,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x = [1.0, 2.0, 3.0];
 
 julia> y, st_new = model(x, ps, st)
-([3.0, 2.0, 1.0], NamedTuple())
    source
    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+([3.0, 2.0, 1.0], NamedTuple())
    source
    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
           affine=True(), track_stats=True(), epsilon=1f-5, momentum=0.1f0)
    Batch Normalization layer.
    `,2)),t("p",null,[a[222]||(a[222]=t("code",null,"BatchNorm",-1)),a[223]||(a[223]=s(" computes the mean and variance for each ")),t("mjx-container",N1,[(Q(),n("svg",R1,a[220]||(a[220]=[i('',1)]))),a[221]||(a[221]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mi",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mi",null,"×"),t("mn",null,"1"),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),a[224]||(a[224]=s(" input slice and normalises the input accordingly."))]),a[226]||(a[226]=i(`
    Arguments
    • chs: Size of the channel dimension in your data. Given an array with N dimensions, call the N-1th the channel dimension. For a batch of feature vectors this is just the data dimension, for WHCN images it's the usual channel dimension.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • If track_stats=true, accumulates mean and variance statistics in training phase that will be used to renormalize the input in test phase.
    • epsilon: a value added to the denominator for numerical stability
    • momentum: the value used for the running_mean and running_var computation
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-channel bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: Array where size(x, N - 1) = chs
    Returns
    • y: Normalized Array
    • Update model state
    Parameters
    • affine=true
      • bias: Bias of shape (chs,)
      • scale: Scale of shape (chs,)
    • affine=false - Empty NamedTuple()
    States
    • Statistics if track_stats=true
      • running_mean: Running mean of shape (chs,)
      • running_var: Running variance of shape (chs,)
    • Statistics if track_stats=false
      • running_mean: nothing
      • running_var: nothing
    • training: Used to check if training/inference mode
    Use Lux.testmode during inference.
    Example
    julia
    julia> Chain(Dense(784 => 64), BatchNorm(64, relu), Dense(64 => 10), BatchNorm(10))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -126,7 +126,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = BatchNorm(10, affine=true, track_stats=true),  # 20 parameters, plus 21
 )         # Total: 51_038 parameters,
-          #        plus 150 states.
    Warning
    Passing a batch size of 1, during training will result in an error.
    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
+          #        plus 150 states.
    Warning
    Passing a batch size of 1, during training will result in an error.
    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
           init_scale=ones32, affine=true, epsilon=1f-5)
    Group Normalization layer.
    Arguments
    • chs: Size of the channel dimension in your data. Given an array with N dimensions, call the N-1th the channel dimension. For a batch of feature vectors this is just the data dimension, for WHCN images it's the usual channel dimension.
    • groups is the number of groups along which the statistics are computed. The number of channels must be an integer multiple of the number of groups.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • epsilon: a value added to the denominator for numerical stability
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-channel bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: Array where size(x, N - 1) = chs and ndims(x) > 2
    Returns
    • y: Normalized Array
    • Update model state
    Parameters
    • affine=true
      • bias: Bias of shape (chs,)
      • scale: Scale of shape (chs,)
    • affine=false - Empty NamedTuple()
    States
    • training: Used to check if training/inference mode
    Use Lux.testmode during inference.
    Example
    julia
    julia> Chain(Dense(784 => 64), GroupNorm(64, 4, relu), Dense(64 => 10), GroupNorm(10, 5))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -134,7 +134,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = GroupNorm(10, 5, affine=true),  # 20 parameters
 )         # Total: 51_038 parameters,
-          #        plus 0 states.
    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+          #        plus 0 states.
    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
              affine=False(), track_stats=False(), epsilon=1f-5, momentum=0.1f0)
    Instance Normalization. For details see [1].
    `,2)),t("p",null,[a[234]||(a[234]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",I1,[(Q(),n("svg",S1,a[232]||(a[232]=[i('',1)]))),a[233]||(a[233]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),a[235]||(a[235]=s("` input slice and normalises the input accordingly."))]),a[237]||(a[237]=i(`
    Arguments
    • chs: Size of the channel dimension in your data. Given an array with N dimensions, call the N-1th the channel dimension. For a batch of feature vectors this is just the data dimension, for WHCN images it's the usual channel dimension.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • If track_stats=true, accumulates mean and variance statistics in training phase that will be used to renormalize the input in test phase.
    • epsilon: a value added to the denominator for numerical stability
    • momentum: the value used for the running_mean and running_var computation
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-channel bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: Array where size(x, N - 1) = chs and ndims(x) > 2
    Returns
    • y: Normalized Array
    • Update model state
    Parameters
    • affine=true
      • bias: Bias of shape (chs,)
      • scale: Scale of shape (chs,)
    • affine=false - Empty NamedTuple()
    States
    • Statistics if track_stats=true
      • running_mean: Running mean of shape (chs,)
      • running_var: Running variance of shape (chs,)
    • Statistics if track_stats=false
      • running_mean: nothing
      • running_var: nothing
    • training: Used to check if training/inference mode
    Use Lux.testmode during inference.
    Example
    julia
    julia> Chain(Dense(784 => 64), InstanceNorm(64, relu; affine=true), Dense(64 => 10),
            InstanceNorm(10, relu; affine=true))
 Chain(
@@ -143,7 +143,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = InstanceNorm(10, relu, affine=true, track_stats=false),  # 20 parameters, plus 1
 )         # Total: 51_038 parameters,
-          #        plus 2 states.
    References
    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).
    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm
    source
    `,20))]),t("details",P1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
-          affine=true, init_bias=zeros32, init_scale=ones32)
    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.
    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('
    Arguments
    • shape: Broadcastable shape of input array excluding the batch dimension.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • epsilon: a value added to the denominator for numerical stability.
    • dims: Dimensions to normalize the array over.
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-element bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: AbstractArray
    Returns
    • y: Normalized Array
    • Empty NamedTuple()
    Parameters
    • affine=false: Empty NamedTuple()
    • affine=true
      • bias: Bias of shape (shape..., 1)
      • scale: Scale of shape (shape..., 1)
    source
    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
-           dims::Union{Tuple, Nothing}=nothing)
    Applies weight normalization to a parameter in the given layer.
    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('
    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).
    Arguments
    • layer whose parameters are being reparameterized
    • which_params: parameter names for the parameters being reparameterized
    • By default, a norm over the entire array is computed. Pass dims to modify the dimension.
    Inputs
    • x: Should be of valid type for input to layer
    Returns
    • Output from layer
    • Updated model state of layer
    Parameters
    • normalized: Parameters of layer that are being normalized
    • unnormalized: Parameters of layer that are not being normalized
    States
    • Same as that of layer
    source
    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)
    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.
    See NNlib.pixel_shuffle for more details.
    Arguments
    • r: Upscale factor
    Inputs
    • x: For 4D-arrays representing N images, the operation converts input size(x) == (W, H, r² x C, N) to output of size (r x W, r x H, C, N). For D-dimensional data, it expects ndims(x) == D + 2 with channel and batch dimensions, and divides the number of channels by rᴰ.
    Returns
    • Output of size (r x W, r x H, C, N) for 4D-arrays, and (r x W, r x H, ..., C, N) for D-dimensional data, where D = ndims(x) - 2
    source
    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
-Upsample(scale, mode = :nearest)
    Upsampling Layer.
    Layer Construction
    Option 1
    • mode: Set to :nearest, :linear, :bilinear or :trilinear
    Exactly one of two keywords must be specified:
    • If scale is a number, this applies to all but the last two dimensions (channel and batch) of the input. It may also be a tuple, to control dimensions individually.
    • Alternatively, keyword size accepts a tuple, to directly specify the leading dimensions of the output.
    Option 2
    • If scale is a number, this applies to all but the last two dimensions (channel and batch) of the input. It may also be a tuple, to control dimensions individually.
    • mode: Set to :nearest, :bilinear or :trilinear
    Currently supported upsampling modes and corresponding NNlib's methods are:
    • :nearest -> NNlib.upsample_nearest
    • :bilinear -> NNlib.upsample_bilinear
    • :trilinear -> NNlib.upsample_trilinear
    Extended Help
    Other Keyword Arguments
    • align_corners: If true, the corner pixels of the input and output tensors are aligned, and thus preserving the values at those pixels. This only has effect when mode is one of :bilinear or :trilinear.
    Inputs
    • x: For the input dimensions look into the documentation for the corresponding NNlib function 
      • As a rule of thumb, :nearest should work with arrays of arbitrary dimensions
      • :bilinear works with 4D Arrays
      • :trilinear works with 5D Arrays
    Returns
    • Upsampled Input of size size or of size (I_1 x scale[1], ..., I_N x scale[N], C, N)
    • Empty NamedTuple()
    source
    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default};
+          #        plus 2 states.

    References

    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",P1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
+          affine=true, init_bias=zeros32, init_scale=ones32)

    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
+           dims::Union{Tuple, Nothing}=nothing)

    Applies weight normalization to a parameter in the given layer.

    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('

    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

    Arguments

    Inputs

    Returns

    Parameters

    States

    source

    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)

    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

    See NNlib.pixel_shuffle for more details.

    Arguments

    Inputs

    Returns

    source

    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
+Upsample(scale, mode = :nearest)

    Upsampling Layer.

    Layer Construction

    Option 1

    Exactly one of two keywords must be specified:

    Option 2

    Currently supported upsampling modes and corresponding NNlib's methods are:

    Extended Help

    Other Keyword Arguments

    Inputs

    Returns

    source

    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default}; diff --git a/dev/assets/api_Lux_layers.md.DxMsNQ36.lean.js b/dev/assets/api_Lux_layers.md.DEbtIDj-.lean.js similarity index 99% rename from dev/assets/api_Lux_layers.md.DxMsNQ36.lean.js rename to dev/assets/api_Lux_layers.md.DEbtIDj-.lean.js index 3c8f80ab9d..b165b56f08 100644 --- a/dev/assets/api_Lux_layers.md.DxMsNQ36.lean.js +++ b/dev/assets/api_Lux_layers.md.DEbtIDj-.lean.js @@ -1,11 +1,11 @@ -import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/framework.CCjWn1F9.js";const o2=JSON.parse('{"title":"Built-In Layers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/layers.md","filePath":"api/Lux/layers.md","lastUpdated":null}'),o={name:"api/Lux/layers.md"},r={class:"jldocstring custom-block"},p={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},E={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},j={class:"jldocstring custom-block"},A={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},V={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},O={class:"jldocstring custom-block"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-5.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"51.473ex",height:"11.422ex",role:"img",focusable:"false",viewBox:"0 -2774.4 22750.9 5048.7","aria-hidden":"true"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"15.326ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 6774.2 1000","aria-hidden":"true"},U={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"16.435ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 7264.2 1000","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.831ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5229.2 1000","aria-hidden":"true"},$={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},t1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},a1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},s1={class:"jldocstring custom-block"},i1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},e1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-8.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.257ex",height:"17.424ex",role:"img",focusable:"false",viewBox:"0 -4100.7 17793.6 7701.4","aria-hidden":"true"},l1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},n1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},Q1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},d1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.018ex",height:"1.357ex",role:"img",focusable:"false",viewBox:"0 -442 1776.1 599.8","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"19.753ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 8730.9 1045","aria-hidden":"true"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},m1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.231ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 9384.3 1045","aria-hidden":"true"},g1={class:"jldocstring custom-block"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.65ex"},xmlns:"http://www.w3.org/2000/svg",width:"67.61ex",height:"2.347ex",role:"img",focusable:"false",viewBox:"0 -750 29883.5 1037.2","aria-hidden":"true"},u1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},L1={class:"jldocstring custom-block"},b1={class:"jldocstring custom-block"},x1={class:"jldocstring custom-block"},w1={class:"jldocstring custom-block"},H1={class:"jldocstring custom-block"},C1={class:"jldocstring custom-block"},F1={class:"jldocstring custom-block"},D1={class:"jldocstring custom-block"},M1={class:"jldocstring custom-block"},v1={class:"jldocstring custom-block"},Z1={class:"jldocstring custom-block"},j1={class:"jldocstring custom-block"},A1={class:"jldocstring custom-block"},B1={class:"jldocstring custom-block"},V1={class:"jldocstring custom-block"},N1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.059ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10192.1 891","aria-hidden":"true"},O1={class:"jldocstring custom-block"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},P1={class:"jldocstring custom-block"},_1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},W1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},X1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},U1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},$1={class:"jldocstring custom-block"},Y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t2={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.172ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.071ex",height:"4.704ex",role:"img",focusable:"false",viewBox:"0 -1119 4451.6 2079","aria-hidden":"true"},a2={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"};function i2(e2,a,l2,n2,Q2,T2){const e=d("Badge");return Q(),n("div",null,[a[269]||(a[269]=t("h1",{id:"Built-In-Layers",tabindex:"-1"},[s("Built-In Layers "),t("a",{class:"header-anchor",href:"#Built-In-Layers","aria-label":'Permalink to "Built-In Layers {#Built-In-Layers}"'},"​")],-1)),a[270]||(a[270]=t("h2",{id:"containers",tabindex:"-1"},[s("Containers "),t("a",{class:"header-anchor",href:"#containers","aria-label":'Permalink to "Containers"'},"​")],-1)),t("details",r,[t("summary",null,[a[0]||(a[0]=t("a",{id:"Lux.BranchLayer",href:"#Lux.BranchLayer"},[t("span",{class:"jlbinding"},"Lux.BranchLayer")],-1)),a[1]||(a[1]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[2]||(a[2]=i(`
    julia
    BranchLayer(layers...)
+import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/framework.CCjWn1F9.js";const o2=JSON.parse('{"title":"Built-In Layers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/layers.md","filePath":"api/Lux/layers.md","lastUpdated":null}'),o={name:"api/Lux/layers.md"},r={class:"jldocstring custom-block"},p={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},E={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},j={class:"jldocstring custom-block"},A={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},V={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"45.995ex",height:"5.686ex",role:"img",focusable:"false",viewBox:"0 -1563.5 20329.9 2513","aria-hidden":"true"},O={class:"jldocstring custom-block"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-5.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"51.473ex",height:"11.422ex",role:"img",focusable:"false",viewBox:"0 -2774.4 22750.9 5048.7","aria-hidden":"true"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"15.326ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 6774.2 1000","aria-hidden":"true"},U={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"16.435ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 7264.2 1000","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.831ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5229.2 1000","aria-hidden":"true"},$={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},t1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},a1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.939ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 5719.2 1000","aria-hidden":"true"},s1={class:"jldocstring custom-block"},i1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},e1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-8.146ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.257ex",height:"17.424ex",role:"img",focusable:"false",viewBox:"0 -4100.7 17793.6 7701.4","aria-hidden":"true"},l1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},n1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},Q1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},d1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.018ex",height:"1.357ex",role:"img",focusable:"false",viewBox:"0 -442 1776.1 599.8","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"19.753ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 8730.9 1045","aria-hidden":"true"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},m1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.667ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.231ex",height:"2.364ex",role:"img",focusable:"false",viewBox:"0 -750 9384.3 1045","aria-hidden":"true"},g1={class:"jldocstring custom-block"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.65ex"},xmlns:"http://www.w3.org/2000/svg",width:"67.61ex",height:"2.347ex",role:"img",focusable:"false",viewBox:"0 -750 29883.5 1037.2","aria-hidden":"true"},u1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"4.342ex",height:"1.927ex",role:"img",focusable:"false",viewBox:"0 -694 1919.1 851.8","aria-hidden":"true"},L1={class:"jldocstring custom-block"},x1={class:"jldocstring custom-block"},b1={class:"jldocstring custom-block"},w1={class:"jldocstring custom-block"},H1={class:"jldocstring custom-block"},C1={class:"jldocstring custom-block"},F1={class:"jldocstring custom-block"},D1={class:"jldocstring custom-block"},M1={class:"jldocstring custom-block"},v1={class:"jldocstring custom-block"},Z1={class:"jldocstring custom-block"},j1={class:"jldocstring custom-block"},A1={class:"jldocstring custom-block"},B1={class:"jldocstring custom-block"},V1={class:"jldocstring custom-block"},N1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.059ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10192.1 891","aria-hidden":"true"},O1={class:"jldocstring custom-block"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},P1={class:"jldocstring custom-block"},_1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},G1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},W1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},X1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},U1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},$1={class:"jldocstring custom-block"},Y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t2={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.172ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.071ex",height:"4.704ex",role:"img",focusable:"false",viewBox:"0 -1119 4451.6 2079","aria-hidden":"true"},a2={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"};function i2(e2,a,l2,n2,Q2,T2){const e=d("Badge");return Q(),n("div",null,[a[269]||(a[269]=t("h1",{id:"Built-In-Layers",tabindex:"-1"},[s("Built-In Layers "),t("a",{class:"header-anchor",href:"#Built-In-Layers","aria-label":'Permalink to "Built-In Layers {#Built-In-Layers}"'},"​")],-1)),a[270]||(a[270]=t("h2",{id:"containers",tabindex:"-1"},[s("Containers "),t("a",{class:"header-anchor",href:"#containers","aria-label":'Permalink to "Containers"'},"​")],-1)),t("details",r,[t("summary",null,[a[0]||(a[0]=t("a",{id:"Lux.BranchLayer",href:"#Lux.BranchLayer"},[t("span",{class:"jlbinding"},"Lux.BranchLayer")],-1)),a[1]||(a[1]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[2]||(a[2]=i(`
    julia
    BranchLayer(layers...)
 BranchLayer(; name=nothing, layers...)
    Takes an input x and passes it through all the layers and returns a tuple of the outputs.
    Arguments
    • Layers can be specified in two formats: 
      • A list of N Lux layers
      • Specified as N keyword arguments.
    Extended Help
    Inputs
    • x: Will be directly passed to each of the layers
    Returns
    • Tuple: (layer_1(x), layer_2(x), ..., layer_N(x)) (naming changes if using the kwargs API)
    • Updated state of the layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    Comparison with Parallel
    This is slightly different from Parallel(nothing, layers...)
    • If the input is a tuple, Parallel will pass each element individually to each layer.
    • BranchLayer essentially assumes 1 input comes in and is branched out into N outputs.
    Example
    An easy way to replicate an input to an NTuple is to do
    julia
    julia> BranchLayer(NoOpLayer(), NoOpLayer(), NoOpLayer())
 BranchLayer(
     layer_1 = NoOpLayer(),
     layer_2 = NoOpLayer(),
     layer_3 = NoOpLayer(),
 )         # Total: 0 parameters,
-          #        plus 0 states.
    source
    `,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
    julia
    Chain(layers...; name=nothing)
+          #        plus 0 states.
    source
    `,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
    julia
    Chain(layers...; name=nothing)
 Chain(; layers..., name=nothing)
    Collects multiple layers / functions to be called in sequence on a given input.
    Arguments
    • Layers can be specified in two formats: 
      • A list of N Lux layers
      • Specified as N keyword arguments.
    Extended Help
    Inputs
    Input x is passed sequentially to each layer, and must conform to the input requirements of the internal layers.
    Returns
    • Output after sequentially applying all the layers to x
    • Updated model states
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    Miscellaneous Properties
    • Allows indexing and field access syntax. We can access the ith layer by m[i] or m.layer_i. We can also index using ranges or arrays.
    Example
    julia
    julia> Chain(Dense(2, 3, relu), BatchNorm(3), Dense(3, 2))
 Chain(
     layer_1 = Dense(2 => 3, relu),      # 9 parameters
@@ -20,7 +20,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_2 = BatchNorm(3, affine=true, track_stats=true),  # 6 parameters, plus 7
     layer_3 = Dense(3 => 2),            # 8 parameters
 )         # Total: 23 parameters,
-          #        plus 7 states.
    source
    `,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
    julia
    PairwiseFusion(connection, layers...; name=nothing)
+          #        plus 7 states.
    source
    `,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
    julia
    PairwiseFusion(connection, layers...; name=nothing)
 PairwiseFusion(connection; name=nothing, layers...)
 PairwiseFusion(; connection, layers..., name=nothing)
    x1 → layer1 → y1 ↘
                   connection → layer2 → y2 ↘
@@ -31,7 +31,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 end
    1. Any other kind of input
    julia
    y = x
 for i in 1:N
     y = connection(x, layers[i](y))
-end
    Returns
    • See Inputs section for how the return value is computed
    • Updated model state for all the contained layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    source
    `,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
    julia
    Parallel(connection, layers...; name=nothing)
+end
    Returns
    • See Inputs section for how the return value is computed
    • Updated model state for all the contained layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    source
    `,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
    julia
    Parallel(connection, layers...; name=nothing)
 Parallel(connection; name=nothing, layers...)
 Parallel(; connection, layers..., name=nothing)
    Create a layer which passes an input to each path in layers, before reducing the output with connection.
    Arguments
    • connection: An N-argument function that is called after passing the input through each layer. If connection = nothing, we return a tuple Parallel(nothing, f, g)(x, y) = (f(x), g(y))
    • Layers can be specified in two formats:
      • A list of N Lux layers
      • Specified as N keyword arguments.
    Extended Help
    Inputs
    • x: If x is not a tuple, then return is computed as connection([l(x) for l in layers]...). Else one is passed to each layer, thus Parallel(+, f, g)(x, y) = f(x) + g(y).
    Returns
    • See the Inputs section for how the output is computed
    • Updated state of the layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    See also SkipConnection which is Parallel with one identity.
    Example
    julia
    julia> model = Parallel(nothing, Dense(2, 1), Dense(2, 1))
 Parallel(
@@ -47,37 +47,37 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x2 = randn(rng, Float32, 2);
 
 julia> size.(first(model((x1, x2), ps, st)))
-((1,), (1,))
    source
    `,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
    julia
    SkipConnection(layers, connection; name=nothing)
-SkipConnection(; layers, connection, name=nothing)
    Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.
    The simplest "ResNet"-type connection is just SkipConnection(layer, +).
    Arguments
    • layer: Layer or Chain of layers to be applied to the input
    • connection:
      • A 2-argument function that takes layer(input) and the input OR
      • An AbstractLuxLayer that takes (layer(input), input) as input
    Extended Help
    Inputs
    • x: Will be passed directly to layer
    Returns
    • Output of connection(layer(input), input)
    • Updated state of layer
    Parameters
    • Parameters of layer OR
    • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
    States
    • States of layer OR
    • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
    See Parallel for a more general implementation.
    source
    `,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
    julia
    RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))
    Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:
    • input_injection = Val(false)
    julia
    res = x
+((1,), (1,))
    source
    `,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
    julia
    SkipConnection(layers, connection; name=nothing)
+SkipConnection(; layers, connection, name=nothing)
    Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.
    The simplest "ResNet"-type connection is just SkipConnection(layer, +).
    Arguments
    • layer: Layer or Chain of layers to be applied to the input
    • connection:
      • A 2-argument function that takes layer(input) and the input OR
      • An AbstractLuxLayer that takes (layer(input), input) as input
    Extended Help
    Inputs
    • x: Will be passed directly to layer
    Returns
    • Output of connection(layer(input), input)
    • Updated state of layer
    Parameters
    • Parameters of layer OR
    • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
    States
    • States of layer OR
    • If connection is an AbstractLuxLayer, then NamedTuple with fields :layers and :connection
    See Parallel for a more general implementation.
    source
    `,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
    julia
    RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))
    Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:
    • input_injection = Val(false)
    julia
    res = x
 for i in 1:repeats
     res, st = model(res, ps, st)
 end
    • input_injection = Val(true)
    julia
    res = x
 for i in 1:repeats
     res, st = model((res, x), ps, st)
-end
    It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.
    Arguments
    • model must be an AbstractLuxLayer
    Keyword Arguments
    • repeats: Number of times to apply the model
    • input_injection: If true, then the input is passed to the model along with the output
    Extended Help
    Inputs
    • x: Input as described above
    Returns
    • Output is computed by as described above
    • Updated state of the model
    Parameters
    • Parameters of model
    States
    • State of model
    source
    `,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
    julia
    Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+end
    It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.
    Arguments
    • model must be an AbstractLuxLayer
    Keyword Arguments
    • repeats: Number of times to apply the model
    • input_injection: If true, then the input is passed to the model along with the output
    Extended Help
    Inputs
    • x: Input as described above
    Returns
    • Output is computed by as described above
    • Updated state of the model
    Parameters
    • Parameters of model
    States
    • State of model
    source
    `,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
    julia
    Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
      activation=identity; init_weight=nothing, init_bias=nothing, stride=1,
-     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())
    Standard convolutional layer.
    Conv 2D
    Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.
    Warning
    Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.
    Arguments
    • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
    • in_chs: Number of input channels
    • out_chs: Number of input and output channels
    • activation: Activation Function
    Extended Help
    Keyword Arguments
    • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
      • Periodic padding can achieved by pre-empting the layer with a WrappedFunction(x -> NNlib.circular_pad(x, N_pad; dims=pad_dims))
    • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
    • use_bias: Trainable bias can be disabled entirely by setting this to false.
    • cross_correlation: If true, perform cross-correlation instead of convolution. Prior to v1, Lux used to have a CrossCor layer which performed cross-correlation. This was removed in v1 in favor of Conv with cross_correlation=true.
    Inputs
    • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
    Returns
    • Output of the convolution y of size (O_N, ..., O_1, C_out, N) where
    `,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('
    • Empty NamedTuple()
    Parameters
    • weight: Convolution kernel
    • bias: Bias (present if use_bias=true)
    source
    ',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
    julia
    ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())
    Standard convolutional layer.
    Conv 2D
    Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.
    Warning
    Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.
    Arguments
    • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
    • in_chs: Number of input channels
    • out_chs: Number of input and output channels
    • activation: Activation Function
    Extended Help
    Keyword Arguments
    • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
      • Periodic padding can achieved by pre-empting the layer with a WrappedFunction(x -> NNlib.circular_pad(x, N_pad; dims=pad_dims))
    • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
    • use_bias: Trainable bias can be disabled entirely by setting this to false.
    • cross_correlation: If true, perform cross-correlation instead of convolution. Prior to v1, Lux used to have a CrossCor layer which performed cross-correlation. This was removed in v1 in favor of Conv with cross_correlation=true.
    Inputs
    • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
    Returns
    • Output of the convolution y of size (O_N, ..., O_1, C_out, N) where
    `,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('
    • Empty NamedTuple()
    Parameters
    • weight: Convolution kernel
    • bias: Bias (present if use_bias=true)
    source
    ',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
    julia
    ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
               activation=identity; init_weight=glorot_uniform, init_bias=zeros32,
               stride=1, pad=0, outpad=0, dilation=1, groups=1, use_bias=True(),
-              cross_correlation=False())
    Standard convolutional transpose layer.
    Arguments
    • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
    • in_chs: Number of input channels
    • out_chs: Number of input and output channels
    • activation: Activation Function
    Keyword Arguments
    • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) * stride (possibly rounded) for each spatial dimension.
    • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
    • use_bias: Trainable bias can be disabled entirely by setting this to false.
    • cross_correlation: If true, perform transposed cross-correlation instead of transposed convolution.
    • outpad: To converse Conv inversability when stride > 1, outpad can be used to increase the size of the output in the desired dimensions. Whereas pad is used to zero-pad the input, outpad only affects the output shape.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
    Returns
    • Output of the convolution transpose y of size (O_N, ..., O_1, C_out, N) where
    • Empty NamedTuple()
    Parameters
    • weight: Convolution Transpose kernel
    • bias: Bias (present if use_bias=true)
    source
    `,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
    julia
    AlphaDropout(p::Real)
    AlphaDropout layer.
    Arguments
    • p: Probability of Dropout 
      • if p = 0 then NoOpLayer is returned.
      • if p = 1 then WrappedLayer(Base.Fix1(broadcast, zero)) is returned.
    Inputs
    • x: Must be an AbstractArray
    Returns
    • x with dropout mask applied if training=Val(true) else just x
    • State with updated rng
    States
    • rng: Pseudo Random Number Generator
    • training: Used to check if training/inference mode
    Call Lux.testmode to switch to test mode.
    See also Dropout, VariationalHiddenDropout
    source
    ',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
    julia
    Dropout(p; dims=:)
    Dropout layer.
    Arguments
    • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
    Keyword Arguments
    • To apply dropout along certain dimension(s), specify the dims keyword. e.g. Dropout(p; dims = (3,4)) will randomly zero out entire channels on WHCN input (also called 2D dropout).
    Inputs
    • x: Must be an AbstractArray
    Returns
    • x with dropout mask applied if training=Val(true) else just x
    • State with updated rng
    States
    • rng: Pseudo Random Number Generator
    • training: Used to check if training/inference mode
    Call Lux.testmode to switch to test mode.
    See also AlphaDropout, VariationalHiddenDropout
    source
    ',15))]),t("details",b,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
    julia
    VariationalHiddenDropout(p; dims=:)
    VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.
    Arguments
    • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
    Keyword Arguments
    • To apply dropout along certain dimension(s), specify the dims keyword. e.g. VariationalHiddenDropout(p; dims = 3) will randomly zero out entire channels on WHCN input (also called 2D dropout).
    Inputs
    • x: Must be an AbstractArray
    Returns
    • x with dropout mask applied if training=Val(true) else just x
    • State with updated rng
    States
    • rng: Pseudo Random Number Generator
    • training: Used to check if training/inference mode
    • mask: Dropout mask. Initilly set to nothing. After every run, contains the mask applied in that call
    • update_mask: Stores whether new mask needs to be generated in the current call
    Call Lux.testmode to switch to test mode.
    See also AlphaDropout, Dropout
    source
    ',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",x,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
    julia
    AdaptiveLPPool(output_size; p=2)
    Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
    Arguments
    • output_size: Size of the first N dimensions for the output
    GPU Support
    This layer is currently only supported on CPU.
    Inputs
    • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
    Returns
    • Output of size (out..., C, N)
    • Empty NamedTuple()
    source
    ',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
    julia
    AdaptiveMaxPool(output_size)
    Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
    Arguments
    • output_size: Size of the first N dimensions for the output
    Inputs
    • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
    Returns
    • Output of size (out..., C, N)
    • Empty NamedTuple()
    source
    ',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
    julia
    AdaptiveMeanPool(output_size)
    Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
    Arguments
    • output_size: Size of the first N dimensions for the output
    Inputs
    • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
    Returns
    • Output of size (out..., C, N)
    • Empty NamedTuple()
    source
    ',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
    julia
    GlobalLPPool(; p=2)
    Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
    GPU Support
    This layer is currently only supported on CPU.
    Inputs
    • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (1, ..., 1, C, N)
    • Empty NamedTuple()
    source
    ',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
    julia
    GlobalMaxPool()
    Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
    Inputs
    • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (1, ..., 1, C, N)
    • Empty NamedTuple()
    source
    ',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
    julia
    GlobalMeanPool()
    Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
    Inputs
    • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (1, ..., 1, C, N)
    • Empty NamedTuple()
    source
    ',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
    julia
    LPPool(window; stride=window, pad=0, dilation=1, p=2)
    LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.
    Arguments
    • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
    Keyword Arguments
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    GPU Support
    This layer is currently only supported on CPU.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (O_N, ..., O_1, C, N) where
    ',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
    julia
    MaxPool(window; stride=window, pad=0, dilation=1)
    Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.
    Arguments
    • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
    Keyword Arguments
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (O_N, ..., O_1, C, N) where
    ',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
    julia
    MeanPool(window; stride=window, pad=0, dilation=1)
    Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.
    Arguments
    • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
    Keyword Arguments
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (O_N, ..., O_1, C, N) where
    ',11)),t("mjx-container",N,[(Q(),n("svg",R,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
    julia
    GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
-        init_weight=nothing, init_bias=nothing, init_state=zeros32)
    Gated Recurrent Unit (GRU) Cell
    `,2)),t("mjx-container",z,[(Q(),n("svg",I,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State) Dimension
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • init_bias: Initializer for bias. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",S,[(Q(),n("svg",P,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
    julia
    LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
+              cross_correlation=False())
    Standard convolutional transpose layer.
    Arguments
    • k: Tuple of integers specifying the size of the convolutional kernel. Eg, for 2D convolutions length(k) == 2
    • in_chs: Number of input channels
    • out_chs: Number of input and output channels
    • activation: Activation Function
    Keyword Arguments
    • init_weight: Controls the initialization of the weight parameter. If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: Controls the initialization of the bias parameter. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(fan_in)).
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) * stride (possibly rounded) for each spatial dimension.
    • groups: Expected to be an Int. It specifies the number of groups to divide a convolution into (set groups = in_chs for Depthwise Convolutions). in_chs and out_chs must be divisible by groups.
    • use_bias: Trainable bias can be disabled entirely by setting this to false.
    • cross_correlation: If true, perform transposed cross-correlation instead of transposed convolution.
    • outpad: To converse Conv inversability when stride > 1, outpad can be used to increase the size of the output in the desired dimensions. Whereas pad is used to zero-pad the input, outpad only affects the output shape.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2 && size(x, N - 1) == in_chs, i.e. size(x) = (I_N, ..., I_1, C_in, N)
    Returns
    • Output of the convolution transpose y of size (O_N, ..., O_1, C_out, N) where
    • Empty NamedTuple()
    Parameters
    • weight: Convolution Transpose kernel
    • bias: Bias (present if use_bias=true)
    source
    `,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
    julia
    AlphaDropout(p::Real)
    AlphaDropout layer.
    Arguments
    • p: Probability of Dropout 
      • if p = 0 then NoOpLayer is returned.
      • if p = 1 then WrappedLayer(Base.Fix1(broadcast, zero)) is returned.
    Inputs
    • x: Must be an AbstractArray
    Returns
    • x with dropout mask applied if training=Val(true) else just x
    • State with updated rng
    States
    • rng: Pseudo Random Number Generator
    • training: Used to check if training/inference mode
    Call Lux.testmode to switch to test mode.
    See also Dropout, VariationalHiddenDropout
    source
    ',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
    julia
    Dropout(p; dims=:)
    Dropout layer.
    Arguments
    • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
    Keyword Arguments
    • To apply dropout along certain dimension(s), specify the dims keyword. e.g. Dropout(p; dims = (3,4)) will randomly zero out entire channels on WHCN input (also called 2D dropout).
    Inputs
    • x: Must be an AbstractArray
    Returns
    • x with dropout mask applied if training=Val(true) else just x
    • State with updated rng
    States
    • rng: Pseudo Random Number Generator
    • training: Used to check if training/inference mode
    Call Lux.testmode to switch to test mode.
    See also AlphaDropout, VariationalHiddenDropout
    source
    ',15))]),t("details",x,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
    julia
    VariationalHiddenDropout(p; dims=:)
    VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.
    Arguments
    • p: Probability of Dropout (if p = 0 then NoOpLayer is returned)
    Keyword Arguments
    • To apply dropout along certain dimension(s), specify the dims keyword. e.g. VariationalHiddenDropout(p; dims = 3) will randomly zero out entire channels on WHCN input (also called 2D dropout).
    Inputs
    • x: Must be an AbstractArray
    Returns
    • x with dropout mask applied if training=Val(true) else just x
    • State with updated rng
    States
    • rng: Pseudo Random Number Generator
    • training: Used to check if training/inference mode
    • mask: Dropout mask. Initilly set to nothing. After every run, contains the mask applied in that call
    • update_mask: Stores whether new mask needs to be generated in the current call
    Call Lux.testmode to switch to test mode.
    See also AlphaDropout, Dropout
    source
    ',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
    julia
    AdaptiveLPPool(output_size; p=2)
    Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
    Arguments
    • output_size: Size of the first N dimensions for the output
    GPU Support
    This layer is currently only supported on CPU.
    Inputs
    • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
    Returns
    • Output of size (out..., C, N)
    • Empty NamedTuple()
    source
    ',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
    julia
    AdaptiveMaxPool(output_size)
    Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
    Arguments
    • output_size: Size of the first N dimensions for the output
    Inputs
    • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
    Returns
    • Output of size (out..., C, N)
    • Empty NamedTuple()
    source
    ',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
    julia
    AdaptiveMeanPool(output_size)
    Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.
    Arguments
    • output_size: Size of the first N dimensions for the output
    Inputs
    • x: Expects as input an array with ndims(x) == N + 2, i.e. channel and batch dimensions, after the N feature dimensions, where N = length(output_size).
    Returns
    • Output of size (out..., C, N)
    • Empty NamedTuple()
    source
    ',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
    julia
    GlobalLPPool(; p=2)
    Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
    GPU Support
    This layer is currently only supported on CPU.
    Inputs
    • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (1, ..., 1, C, N)
    • Empty NamedTuple()
    source
    ',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
    julia
    GlobalMaxPool()
    Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
    Inputs
    • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (1, ..., 1, C, N)
    • Empty NamedTuple()
    source
    ',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
    julia
    GlobalMeanPool()
    Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.
    Inputs
    • x: Data satisfying ndims(x) > 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (1, ..., 1, C, N)
    • Empty NamedTuple()
    source
    ',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
    julia
    LPPool(window; stride=window, pad=0, dilation=1, p=2)
    LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.
    Arguments
    • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
    Keyword Arguments
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    GPU Support
    This layer is currently only supported on CPU.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (O_N, ..., O_1, C, N) where
    ',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
    julia
    MaxPool(window; stride=window, pad=0, dilation=1)
    Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.
    Arguments
    • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
    Keyword Arguments
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (O_N, ..., O_1, C, N) where
    ',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
    julia
    MeanPool(window; stride=window, pad=0, dilation=1)
    Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.
    Arguments
    • window: Tuple of integers specifying the size of the window. Eg, for 2D pooling length(window) == 2
    Keyword Arguments
    • stride: Should each be either single integer, or a tuple with N integers
    • dilation: Should each be either single integer, or a tuple with N integers
    • pad: Specifies the number of elements added to the borders of the data array. It can be
      • a single integer for equal padding all around,
      • a tuple of N integers, to apply the same padding at begin/end of each spatial dimension,
      • a tuple of 2*N integers, for asymmetric padding, or
      • the singleton SamePad(), to calculate padding such that size(output,d) == size(x,d) / stride (possibly rounded) for each spatial dimension.
    Extended Help
    Inputs
    • x: Data satisfying ndims(x) == N + 2, i.e. size(x) = (I_N, ..., I_1, C, N)
    Returns
    • Output of the pooling y of size (O_N, ..., O_1, C, N) where
    ',11)),t("mjx-container",N,[(Q(),n("svg",R,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
    julia
    GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
+        init_weight=nothing, init_bias=nothing, init_state=zeros32)
    Gated Recurrent Unit (GRU) Cell
    `,2)),t("mjx-container",z,[(Q(),n("svg",I,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State) Dimension
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • init_bias: Initializer for bias. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. Must be a tuple containing 3 functions. If a single value is passed, it is copied into a 3 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",S,[(Q(),n("svg",P,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
    julia
    LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
          train_memory::Bool=false, init_weight=nothing, init_bias=nothing,
-         init_state=zeros32, init_memory=zeros32)
    Long Short-Term (LSTM) Cell
    `,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State & Memory) Dimension
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • train_memory: Trainable initial memory can be activated by setting this to true
    • init_bias: Initializer for bias. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    • init_memory: Initializer for memory
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to false - Creates a hidden state using init_state, hidden memory using init_memory and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to false - Repeats hidden_state vector from the parameters to match the shape of x, creates hidden memory using init_memory and proceeds to Case 2.
    • Case 1c: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to true - Creates a hidden state using init_state, repeats the memory vector from parameters to match the shape of x and proceeds to Case 2.
    • Case 1d: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to true - Repeats the hidden state and memory vectors from the parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, c)) is provided, then the output and a tuple containing the updated hidden state and memory is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("
  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)
  • hidden_state: Initial hidden state vector (not present if train_state=false)
  • memory: Initial memory vector (not present if train_memory=false)
    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/cb92a563392f7c74ee2072fa765c31bb9b017fc7/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
-    init_bias=nothing, init_weight=nothing, init_state=zeros32)
    An Elman RNNCell cell with activation (typically set to tanh or relu).
    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State) Dimension
    • activation: Activation function
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • init_bias: Initializer for bias. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('
    Parameters
    • weight_ih: Maps the input to the hidden state.
    • weight_hh: Maps the hidden state to the hidden state.
    • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
    • bias_hh: Bias vector for the hidden-hidden connection (not present if use_bias=false)
    • hidden_state: Initial hidden state vector (not present if train_state=false)
    States
    • rng: Controls the randomness (if any) in the initial state generation
    source
    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
+         init_state=zeros32, init_memory=zeros32)
    Long Short-Term (LSTM) Cell
    `,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State & Memory) Dimension
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • train_memory: Trainable initial memory can be activated by setting this to true
    • init_bias: Initializer for bias. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. Must be a tuple containing 4 functions. If a single value is passed, it is copied into a 4 element tuple. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    • init_memory: Initializer for memory
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to false - Creates a hidden state using init_state, hidden memory using init_memory and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to false - Repeats hidden_state vector from the parameters to match the shape of x, creates hidden memory using init_memory and proceeds to Case 2.
    • Case 1c: Only a single input x of shape (in_dims, batch_size), train_state is set to false, train_memory is set to true - Creates a hidden state using init_state, repeats the memory vector from parameters to match the shape of x and proceeds to Case 2.
    • Case 1d: Only a single input x of shape (in_dims, batch_size), train_state is set to true, train_memory is set to true - Repeats the hidden state and memory vectors from the parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, c)) is provided, then the output and a tuple containing the updated hidden state and memory is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("
  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)
  • hidden_state: Initial hidden state vector (not present if train_state=false)
  • memory: Initial memory vector (not present if train_memory=false)
    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/699c8d8fd1112cb1d7801f7073c85c51c96cac1d/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
+    init_bias=nothing, init_weight=nothing, init_state=zeros32)
    An Elman RNNCell cell with activation (typically set to tanh or relu).
    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("
    Arguments
    • in_dims: Input Dimension
    • out_dims: Output (Hidden State) Dimension
    • activation: Activation function
    • use_bias: Set to false to deactivate bias
    • train_state: Trainable initial hidden state can be activated by setting this to true
    • init_bias: Initializer for bias. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_weight: Initializer for weight. If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(out_dims)).
    • init_state: Initializer for hidden state
    Inputs
    • Case 1a: Only a single input x of shape (in_dims, batch_size), train_state is set to false - Creates a hidden state using init_state and proceeds to Case 2.
    • Case 1b: Only a single input x of shape (in_dims, batch_size), train_state is set to true - Repeats hidden_state from parameters to match the shape of x and proceeds to Case 2.
    • Case 2: Tuple (x, (h, )) is provided, then the output and a tuple containing the updated hidden state is returned.
    Returns
    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('
    Parameters
    • weight_ih: Maps the input to the hidden state.
    • weight_hh: Maps the hidden state to the hidden state.
    • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)
    • bias_hh: Bias vector for the hidden-hidden connection (not present if use_bias=false)
    • hidden_state: Initial hidden state vector (not present if train_state=false)
    States
    • rng: Controls the randomness (if any) in the initial state generation
    source
    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
     ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex(),
-    return_sequence::Bool=false)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.
    Relation to Flux.Recur
    This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Keyword Arguments
    • return_sequence: If true returns the entire sequence of outputs, else returns only the last output. Defaults to false.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell.
    Tip
    Frameworks like Tensorflow have special implementation of MultiRNNCell to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.
    Chain(
+    return_sequence::Bool=false)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.
    Relation to Flux.Recur
    This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Keyword Arguments
    • return_sequence: If true returns the entire sequence of outputs, else returns only the last output. Defaults to false.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell.
    Tip
    Frameworks like Tensorflow have special implementation of StackedRNNCells to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.
    Chain(
     Recurrence(RNNCell(inputsize => latentsize); return_sequence=true),
     Recurrence(RNNCell(latentsize => latentsize); return_sequence=true),
     :
     x -> stack(x; dims=2)
-)
    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.
    source
    `,14))]),t("details",b1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.
    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Inputs
    • Input to the cell.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell and updated carry.
    States
    • NamedTuple containing: 
      • cell: Same as cell.
      • carry: The carry state of the cell.
    source
    ',12))]),t("details",x1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
+)
    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.
    source
    `,14))]),t("details",x1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)
    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.
    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    Inputs
    • Input to the cell.
    Returns
    • Output of the cell for the entire sequence.
    • Update state of the cell and updated carry.
    States
    • NamedTuple containing: 
      • cell: Same as cell.
      • carry: The carry state of the cell.
    source
    ',12))]),t("details",b1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
     backward_cell::Union{AbstractRecurrentCell, Nothing}=nothing;
     merge_mode::Union{Function, Nothing}=vcat,
-    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())
    Bidirectional RNN wrapper.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    • backward_cell: A optional backward recurrent cell. If backward_cell is nothing, the rnn layer instance passed as the cell argument will be used to generate the backward layer automatically. in_dims of backward_cell should be consistent with in_dims of cell
    Keyword Arguments
    • merge_mode: Function by which outputs of the forward and backward RNNs will be combined. default value is vcat. If nothing, the outputs will not be combined.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Merged output of the cell and backward_cell for the entire sequence.
    • Update state of the cell and backward_cell.
    Parameters
    • NamedTuple with cell and backward_cell.
    States
    • Same as cell and backward_cell.
    source
    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
+    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())
    Bidirectional RNN wrapper.
    Arguments
    • cell: A recurrent cell. See RNNCell, LSTMCell, GRUCell, for how the inputs/outputs of a recurrent cell must be structured.
    • backward_cell: A optional backward recurrent cell. If backward_cell is nothing, the rnn layer instance passed as the cell argument will be used to generate the backward layer automatically. in_dims of backward_cell should be consistent with in_dims of cell
    Keyword Arguments
    • merge_mode: Function by which outputs of the forward and backward RNNs will be combined. default value is vcat. If nothing, the outputs will not be combined.
    • ordering: The ordering of the batch and time dimensions in the input. Defaults to BatchLastIndex(). Alternatively can be set to TimeLastIndex().
    Extended Help
    Inputs
    • If x is a 
      • Tuple or Vector: Each element is fed to the cell sequentially.
      • Array (except a Vector): It is spliced along the penultimate dimension and each slice is fed to the cell sequentially.
    Returns
    • Merged output of the cell and backward_cell for the entire sequence.
    • Update state of the cell and backward_cell.
    Parameters
    • NamedTuple with cell and backward_cell.
    States
    • Same as cell and backward_cell.
    source
    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
          init_bias=nothing, use_bias=True())
 Bilinear(in12_dims => out, activation=identity; init_weight=nothing,
-         init_bias=nothing, use_bias=True())
    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:
    z[i] = activation(x' * W[i, :, :] * y + bias[i])
    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.
    Arguments
    • in1_dims: number of input dimensions of x
    • in2_dims: number of input dimensions of y
    • in12_dims: If specified, then in1_dims = in2_dims = in12_dims
    • out: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in1_dims, in2_dims)). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • A 2-Tuple containing
      • x must be an AbstractArray with size(x, 1) == in1_dims
      • y must be an AbstractArray with size(y, 1) == in2_dims
    • If the input is an AbstractArray, then x = y
    Returns
    • AbstractArray with dimensions (out_dims, size(x, 2))
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in1_dims, in2_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
-      init_bias=nothing, use_bias=True())
    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)
    Arguments
    • in_dims: number of input dimensions
    • out_dims: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims)). If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an AbstractArray with size(x, 1) == in_dims
    Returns
    • AbstractArray with dimensions (out_dims, ...) where ... are the dimensions of x
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)
    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.
    This layer is often used to store word embeddings and retrieve them using indices.
    Arguments
    • in_dims: number(s) of input dimensions
    • out_dims: number of output dimensions
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims...))
    Input
    • Integer OR
    • Abstract Vector of Integers OR
    • Abstract Array of Integers OR
    • Tuple of Integers OR
    • Tuple of Abstract Vectors of Integers OR
    • Tuple of Abstract Arrays of Integers
    Returns
    • Returns the embedding corresponding to each index in the input. For an N dimensional input, an N + 1 dimensional output is returned.
    • Empty NamedTuple()
    source
    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())
    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)
    Arguments
    • dims: size of the learnable scale and bias parameters.
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims))
    • init_bias: initializer for the bias vector (ignored if use_bias=false)
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    Returns
    • Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    • Empty NamedTuple()
    Parameters
    • weight: Weight Array of size (dims...)
    • bias: Bias of size (dims...)
    source
    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)
    Flattens the passed array into a matrix.
    Keyword Arguments
    • N: Flatten the first N dimensions of the input array. If nothing, then all dimensions (except the last) are flattened. Note that the batch dimension is never flattened.
    Inputs
    • x: AbstractArray
    Returns
    • AbstractMatrix of size (:, size(x, ndims(x))) if N is nothing else the first N dimensions of the input array are flattened.
    • Empty NamedTuple()
    Example
    julia
    julia> model = FlattenLayer()
+         init_bias=nothing, use_bias=True())
    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:
    z[i] = activation(x' * W[i, :, :] * y + bias[i])
    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.
    Arguments
    • in1_dims: number of input dimensions of x
    • in2_dims: number of input dimensions of y
    • in12_dims: If specified, then in1_dims = in2_dims = in12_dims
    • out: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in1_dims, in2_dims)). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in1_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • A 2-Tuple containing
      • x must be an AbstractArray with size(x, 1) == in1_dims
      • y must be an AbstractArray with size(y, 1) == in2_dims
    • If the input is an AbstractArray, then x = y
    Returns
    • AbstractArray with dimensions (out_dims, size(x, 2))
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in1_dims, in2_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
+      init_bias=nothing, use_bias=True())
    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)
    Arguments
    • in_dims: number of input dimensions
    • out_dims: number of output dimensions
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims)). If nothing, then we use kaiming_uniform with gain computed on the basis of the activation function (taken from Pytorch nn.init.calculate_gain).
    • init_bias: initializer for the bias vector (ignored if use_bias=false). If nothing, then we use uniform distribution with bounds -bound and bound where bound = inv(sqrt(in_dims)).
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an AbstractArray with size(x, 1) == in_dims
    Returns
    • AbstractArray with dimensions (out_dims, ...) where ... are the dimensions of x
    • Empty NamedTuple()
    Parameters
    • weight: Weight Matrix of size (out_dims, in_dims)
    • bias: Bias of size (out_dims, 1) (present if use_bias=true)
    source
    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)
    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.
    This layer is often used to store word embeddings and retrieve them using indices.
    Arguments
    • in_dims: number(s) of input dimensions
    • out_dims: number of output dimensions
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims...))
    Input
    • Integer OR
    • Abstract Vector of Integers OR
    • Abstract Array of Integers OR
    • Tuple of Integers OR
    • Tuple of Abstract Vectors of Integers OR
    • Tuple of Abstract Arrays of Integers
    Returns
    • Returns the embedding corresponding to each index in the input. For an N dimensional input, an N + 1 dimensional output is returned.
    • Empty NamedTuple()
    source
    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())
    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)
    Arguments
    • dims: size of the learnable scale and bias parameters.
    • activation: activation function
    Keyword Arguments
    • init_weight: initializer for the weight matrix (weight = init_weight(rng, out_dims, in_dims))
    • init_bias: initializer for the bias vector (ignored if use_bias=false)
    • use_bias: Trainable bias can be disabled entirely by setting this to false
    Input
    • x must be an Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    Returns
    • Array of size (dims..., B) or (dims...[0], ..., dims[k]) for k ≤ size(dims)
    • Empty NamedTuple()
    Parameters
    • weight: Weight Array of size (dims...)
    • bias: Bias of size (dims...)
    source
    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)
    Flattens the passed array into a matrix.
    Keyword Arguments
    • N: Flatten the first N dimensions of the input array. If nothing, then all dimensions (except the last) are flattened. Note that the batch dimension is never flattened.
    Inputs
    • x: AbstractArray
    Returns
    • AbstractMatrix of size (:, size(x, ndims(x))) if N is nothing else the first N dimensions of the input array are flattened.
    • Empty NamedTuple()
    Example
    julia
    julia> model = FlattenLayer()
 FlattenLayer{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -87,9 +87,9 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(8, 2)
    source
    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
+(8, 2)
    source
    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
 Maxout(; layers...)
-Maxout(f::Function, n_alts::Int)
    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.
    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].
    See also Parallel to reduce with other operators.
    Arguments
    • Layers can be specified in three formats: 
      • A list of N Lux layers
      • Specified as N keyword arguments.
      • A no argument function f and an integer n_alts which specifies the number of layers.
    Extended Help
    Inputs
    • x: Input that is passed to each of the layers
    Returns
    • Output is computed by taking elementwise max of the outputs of the individual layers.
    • Updated state of the layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    References
    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389
    source
    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()
    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.
    Example
    julia
    julia> model = NoOpLayer()
+Maxout(f::Function, n_alts::Int)
    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.
    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].
    See also Parallel to reduce with other operators.
    Arguments
    • Layers can be specified in three formats: 
      • A list of N Lux layers
      • Specified as N keyword arguments.
      • A no argument function f and an integer n_alts which specifies the number of layers.
    Extended Help
    Inputs
    • x: Input that is passed to each of the layers
    Returns
    • Output is computed by taking elementwise max of the outputs of the individual layers.
    • Updated state of the layers
    Parameters
    • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    States
    • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N (naming changes if using the kwargs API)
    References
    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389
    source
    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()
    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.
    Example
    julia
    julia> model = NoOpLayer()
 NoOpLayer()
 
 julia> rng = Random.default_rng();
@@ -99,7 +99,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 1
 
 julia> y, st_new = model(x, ps, st)
-(1, NamedTuple())
    source
    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)
    Reshapes the passed array to have a size of (dims..., :)
    Arguments
    • dims: The new dimensions of the array (excluding the last dimension).
    Inputs
    • x: AbstractArray of any shape which can be reshaped in (dims..., size(x, ndims(x)))
    Returns
    • AbstractArray of size (dims..., size(x, ndims(x)))
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReshapeLayer((2, 2))
+(1, NamedTuple())
    source
    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)
    Reshapes the passed array to have a size of (dims..., :)
    Arguments
    • dims: The new dimensions of the array (excluding the last dimension).
    Inputs
    • x: AbstractArray of any shape which can be reshaped in (dims..., size(x, ndims(x)))
    Returns
    • AbstractArray of size (dims..., size(x, ndims(x)))
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReshapeLayer((2, 2))
 ReshapeLayer(output_dims = (2, 2, :))
 
 julia> rng = Random.default_rng();
@@ -109,7 +109,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(2, 2, 3)
    source
    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)
    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.
    Arguments
    • dim: Dimension for indexing
    • i: Index for dimension dim
    Inputs
    • x: AbstractArray that can be indexed with view(x,:,:,...,i,:,:,...)
    Returns
    • view(x,:,:,...,i,:,:,...) where i is in position d
    • Empty NamedTuple()
    source
    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)
    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))
    Arguments
    • f: Some function.
    Inputs
    • x: will be directly passed to f
    Returns
    • Output of f(x)
    • Empty NamedTuple()
    source
    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)
    Reverse the specified dimension dims of the passed array
    Arguments
    • dim: Dimension that need to be reversed. If nothing, for AbstractVector{T} it reverses itself (dimension 1), for other arrays, reverse the dimension ndims(x) - 1.
    Inputs
    • x: AbstractArray.
    Returns
    • AbstractArray with the same dimensions as the input
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReverseSequence()
+(2, 2, 3)
    source
    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)
    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.
    Arguments
    • dim: Dimension for indexing
    • i: Index for dimension dim
    Inputs
    • x: AbstractArray that can be indexed with view(x,:,:,...,i,:,:,...)
    Returns
    • view(x,:,:,...,i,:,:,...) where i is in position d
    • Empty NamedTuple()
    source
    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)
    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))
    Arguments
    • f: Some function.
    Inputs
    • x: will be directly passed to f
    Returns
    • Output of f(x)
    • Empty NamedTuple()
    source
    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)
    Reverse the specified dimension dims of the passed array
    Arguments
    • dim: Dimension that need to be reversed. If nothing, for AbstractVector{T} it reverses itself (dimension 1), for other arrays, reverse the dimension ndims(x) - 1.
    Inputs
    • x: AbstractArray.
    Returns
    • AbstractArray with the same dimensions as the input
    • Empty NamedTuple()
    Example
    julia
    julia> model = ReverseSequence()
 ReverseSequence{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -118,7 +118,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x = [1.0, 2.0, 3.0];
 
 julia> y, st_new = model(x, ps, st)
-([3.0, 2.0, 1.0], NamedTuple())
    source
    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+([3.0, 2.0, 1.0], NamedTuple())
    source
    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
           affine=True(), track_stats=True(), epsilon=1f-5, momentum=0.1f0)
    Batch Normalization layer.
    `,2)),t("p",null,[a[222]||(a[222]=t("code",null,"BatchNorm",-1)),a[223]||(a[223]=s(" computes the mean and variance for each ")),t("mjx-container",N1,[(Q(),n("svg",R1,a[220]||(a[220]=[i('',1)]))),a[221]||(a[221]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mi",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mi",null,"×"),t("mn",null,"1"),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),a[224]||(a[224]=s(" input slice and normalises the input accordingly."))]),a[226]||(a[226]=i(`
    Arguments
    • chs: Size of the channel dimension in your data. Given an array with N dimensions, call the N-1th the channel dimension. For a batch of feature vectors this is just the data dimension, for WHCN images it's the usual channel dimension.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • If track_stats=true, accumulates mean and variance statistics in training phase that will be used to renormalize the input in test phase.
    • epsilon: a value added to the denominator for numerical stability
    • momentum: the value used for the running_mean and running_var computation
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-channel bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: Array where size(x, N - 1) = chs
    Returns
    • y: Normalized Array
    • Update model state
    Parameters
    • affine=true
      • bias: Bias of shape (chs,)
      • scale: Scale of shape (chs,)
    • affine=false - Empty NamedTuple()
    States
    • Statistics if track_stats=true
      • running_mean: Running mean of shape (chs,)
      • running_var: Running variance of shape (chs,)
    • Statistics if track_stats=false
      • running_mean: nothing
      • running_var: nothing
    • training: Used to check if training/inference mode
    Use Lux.testmode during inference.
    Example
    julia
    julia> Chain(Dense(784 => 64), BatchNorm(64, relu), Dense(64 => 10), BatchNorm(10))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -126,7 +126,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = BatchNorm(10, affine=true, track_stats=true),  # 20 parameters, plus 21
 )         # Total: 51_038 parameters,
-          #        plus 150 states.
    Warning
    Passing a batch size of 1, during training will result in an error.
    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
+          #        plus 150 states.
    Warning
    Passing a batch size of 1, during training will result in an error.
    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
           init_scale=ones32, affine=true, epsilon=1f-5)
    Group Normalization layer.
    Arguments
    • chs: Size of the channel dimension in your data. Given an array with N dimensions, call the N-1th the channel dimension. For a batch of feature vectors this is just the data dimension, for WHCN images it's the usual channel dimension.
    • groups is the number of groups along which the statistics are computed. The number of channels must be an integer multiple of the number of groups.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • epsilon: a value added to the denominator for numerical stability
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-channel bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: Array where size(x, N - 1) = chs and ndims(x) > 2
    Returns
    • y: Normalized Array
    • Update model state
    Parameters
    • affine=true
      • bias: Bias of shape (chs,)
      • scale: Scale of shape (chs,)
    • affine=false - Empty NamedTuple()
    States
    • training: Used to check if training/inference mode
    Use Lux.testmode during inference.
    Example
    julia
    julia> Chain(Dense(784 => 64), GroupNorm(64, 4, relu), Dense(64 => 10), GroupNorm(10, 5))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -134,7 +134,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = GroupNorm(10, 5, affine=true),  # 20 parameters
 )         # Total: 51_038 parameters,
-          #        plus 0 states.
    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+          #        plus 0 states.
    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm
    source
    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
              affine=False(), track_stats=False(), epsilon=1f-5, momentum=0.1f0)
    Instance Normalization. For details see [1].
    `,2)),t("p",null,[a[234]||(a[234]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",I1,[(Q(),n("svg",S1,a[232]||(a[232]=[i('',1)]))),a[233]||(a[233]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),a[235]||(a[235]=s("` input slice and normalises the input accordingly."))]),a[237]||(a[237]=i(`
    Arguments
    • chs: Size of the channel dimension in your data. Given an array with N dimensions, call the N-1th the channel dimension. For a batch of feature vectors this is just the data dimension, for WHCN images it's the usual channel dimension.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • If track_stats=true, accumulates mean and variance statistics in training phase that will be used to renormalize the input in test phase.
    • epsilon: a value added to the denominator for numerical stability
    • momentum: the value used for the running_mean and running_var computation
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-channel bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: Array where size(x, N - 1) = chs and ndims(x) > 2
    Returns
    • y: Normalized Array
    • Update model state
    Parameters
    • affine=true
      • bias: Bias of shape (chs,)
      • scale: Scale of shape (chs,)
    • affine=false - Empty NamedTuple()
    States
    • Statistics if track_stats=true
      • running_mean: Running mean of shape (chs,)
      • running_var: Running variance of shape (chs,)
    • Statistics if track_stats=false
      • running_mean: nothing
      • running_var: nothing
    • training: Used to check if training/inference mode
    Use Lux.testmode during inference.
    Example
    julia
    julia> Chain(Dense(784 => 64), InstanceNorm(64, relu; affine=true), Dense(64 => 10),
            InstanceNorm(10, relu; affine=true))
 Chain(
@@ -143,7 +143,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = InstanceNorm(10, relu, affine=true, track_stats=false),  # 20 parameters, plus 1
 )         # Total: 51_038 parameters,
-          #        plus 2 states.
    References
    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).
    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm
    source
    `,20))]),t("details",P1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
-          affine=true, init_bias=zeros32, init_scale=ones32)
    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.
    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('
    Arguments
    • shape: Broadcastable shape of input array excluding the batch dimension.
    • activation: After normalization, elementwise activation activation is applied.
    Keyword Arguments
    • epsilon: a value added to the denominator for numerical stability.
    • dims: Dimensions to normalize the array over.
    • If affine=true, it also applies a shift and a rescale to the input through to learnable per-element bias and scale parameters.
      • init_bias: Controls how the bias is initialized
      • init_scale: Controls how the scale is initialized
    Extended Help
    Inputs
    • x: AbstractArray
    Returns
    • y: Normalized Array
    • Empty NamedTuple()
    Parameters
    • affine=false: Empty NamedTuple()
    • affine=true
      • bias: Bias of shape (shape..., 1)
      • scale: Scale of shape (shape..., 1)
    source
    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
-           dims::Union{Tuple, Nothing}=nothing)
    Applies weight normalization to a parameter in the given layer.
    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('
    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).
    Arguments
    • layer whose parameters are being reparameterized
    • which_params: parameter names for the parameters being reparameterized
    • By default, a norm over the entire array is computed. Pass dims to modify the dimension.
    Inputs
    • x: Should be of valid type for input to layer
    Returns
    • Output from layer
    • Updated model state of layer
    Parameters
    • normalized: Parameters of layer that are being normalized
    • unnormalized: Parameters of layer that are not being normalized
    States
    • Same as that of layer
    source
    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)
    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.
    See NNlib.pixel_shuffle for more details.
    Arguments
    • r: Upscale factor
    Inputs
    • x: For 4D-arrays representing N images, the operation converts input size(x) == (W, H, r² x C, N) to output of size (r x W, r x H, C, N). For D-dimensional data, it expects ndims(x) == D + 2 with channel and batch dimensions, and divides the number of channels by rᴰ.
    Returns
    • Output of size (r x W, r x H, C, N) for 4D-arrays, and (r x W, r x H, ..., C, N) for D-dimensional data, where D = ndims(x) - 2
    source
    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
-Upsample(scale, mode = :nearest)
    Upsampling Layer.
    Layer Construction
    Option 1
    • mode: Set to :nearest, :linear, :bilinear or :trilinear
    Exactly one of two keywords must be specified:
    • If scale is a number, this applies to all but the last two dimensions (channel and batch) of the input. It may also be a tuple, to control dimensions individually.
    • Alternatively, keyword size accepts a tuple, to directly specify the leading dimensions of the output.
    Option 2
    • If scale is a number, this applies to all but the last two dimensions (channel and batch) of the input. It may also be a tuple, to control dimensions individually.
    • mode: Set to :nearest, :bilinear or :trilinear
    Currently supported upsampling modes and corresponding NNlib's methods are:
    • :nearest -> NNlib.upsample_nearest
    • :bilinear -> NNlib.upsample_bilinear
    • :trilinear -> NNlib.upsample_trilinear
    Extended Help
    Other Keyword Arguments
    • align_corners: If true, the corner pixels of the input and output tensors are aligned, and thus preserving the values at those pixels. This only has effect when mode is one of :bilinear or :trilinear.
    Inputs
    • x: For the input dimensions look into the documentation for the corresponding NNlib function 
      • As a rule of thumb, :nearest should work with arrays of arbitrary dimensions
      • :bilinear works with 4D Arrays
      • :trilinear works with 5D Arrays
    Returns
    • Upsampled Input of size size or of size (I_1 x scale[1], ..., I_N x scale[N], C, N)
    • Empty NamedTuple()
    source
    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default};
+          #        plus 2 states.

    References

    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",P1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
+          affine=true, init_bias=zeros32, init_scale=ones32)

    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
+           dims::Union{Tuple, Nothing}=nothing)

    Applies weight normalization to a parameter in the given layer.

    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('

    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

    Arguments

    Inputs

    Returns

    Parameters

    States

    source

    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)

    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

    See NNlib.pixel_shuffle for more details.

    Arguments

    Inputs

    Returns

    source

    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
+Upsample(scale, mode = :nearest)

    Upsampling Layer.

    Layer Construction

    Option 1

    Exactly one of two keywords must be specified:

    Option 2

    Currently supported upsampling modes and corresponding NNlib's methods are:

    Extended Help

    Other Keyword Arguments

    Inputs

    Returns

    source

    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default}; diff --git a/dev/assets/api_Lux_utilities.md.BJScatXv.js b/dev/assets/api_Lux_utilities.md.BeOts_Il.js similarity index 98% rename from dev/assets/api_Lux_utilities.md.BJScatXv.js rename to dev/assets/api_Lux_utilities.md.BeOts_Il.js index 33fcd97789..f4e1c36a73 100644 --- a/dev/assets/api_Lux_utilities.md.BJScatXv.js +++ b/dev/assets/api_Lux_utilities.md.BeOts_Il.js @@ -1,10 +1,10 @@ -import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.CCjWn1F9.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},P1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(c2,i,E2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState

    Training State containing:

    Internal fields:

    Warning

    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.

    source

    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
-    ts::TrainState)

    Compute the gradients of the objective function wrt parameters stored in ts.

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl

    Arguments

    Return

    A 4-Tuple containing:

    Known Limitations

    Aliased Gradients

    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.

    source

    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.

    In most cases you should use single_train_step! instead of this function.

    Return

    Returned values are the same as compute_gradients.

    source

    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.

    Return

    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).

    source

    ',5))]),i[290]||(i[290]=t('

    Loss Functions ​

    Loss Functions Objects take 2 forms of inputs:

    1. ŷ and y where ŷ is the predicted output and y is the target output.

    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.

    Warning

    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.

    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)

    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.

    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
+import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.CCjWn1F9.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},P1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(c2,i,E2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState
    Training State containing:
    • model: Lux model.
    • parameters: Trainable Variables of the model.
    • states: Non-trainable Variables of the model.
    • optimizer: Optimizer from Optimisers.jl.
    • optimizer_state: Optimizer State.
    • step: Number of updates of the parameters made.
    Internal fields:
    • cache: Cached values. Implementations are free to use this for whatever they want.
    • objective_function: Objective function might be cached.
    Warning
    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.
    source
    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
+    ts::TrainState)
    Compute the gradients of the objective function wrt parameters stored in ts.
    Backends & AD Packages
    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl
    Arguments
    • ad: Backend (from ADTypes.jl) used to compute the gradients.
    • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.
    • data: Data used to compute the gradients.
    • ts: Current Training State. See TrainState.
    Return
    A 4-Tuple containing:
    • grads: Computed Gradients.
    • loss: Loss from the objective function.
    • stats: Any computed statistics from the objective function.
    • ts: Updated Training State.
    Known Limitations
    • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.
    Aliased Gradients
    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.
    source
    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.
    In most cases you should use single_train_step! instead of this function.
    Return
    Returned values are the same as compute_gradients.
    source
    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.
    Return
    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).
    source
    ',5))]),i[290]||(i[290]=t('
    Loss Functions ​
    Loss Functions Objects take 2 forms of inputs:
    1. ŷ and y where ŷ is the predicted output and y is the target output.
    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.
    Warning
    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.
    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)
    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.
    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> mseloss(y_model, 1:3) ≈ 0.01
-true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",c,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
+true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",c,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
     label_smoothing::Union{Nothing, Real}=nothing,
     logits::Union{Bool, Val}=Val(false))
    Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
    Returns the binary cross entropy loss computed as:
    • If logits is either false or Val(false):
    `,4)),s("mjx-container",E,[(h(),n("svg",u,i[23]||(i[23]=[t('',1)]))),i[24]||(i[24]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[52]||(i[52]=s("ul",null,[s("li",null,[a("If "),s("code",null,"logits"),a(" is "),s("code",null,"true"),a(" or "),s("code",null,"Val(true)"),a(":")])],-1)),s("mjx-container",F,[(h(),n("svg",x,i[25]||(i[25]=[t('',1)]))),i[26]||(i[26]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"l"),s("mi",null,"o"),s("mi",null,"g"),s("mi",null,"σ"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[33]||(i[33]=a("The value of ")),s("mjx-container",C,[(h(),n("svg",L,i[27]||(i[27]=[t('',1)]))),i[28]||(i[28]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[34]||(i[34]=a(" is computed using label smoothing. If ")),i[35]||(i[35]=s("code",null,"label_smoothing",-1)),i[36]||(i[36]=a(" is ")),i[37]||(i[37]=s("code",null,"nothing",-1)),i[38]||(i[38]=a(", then no label smoothing is applied. If ")),i[39]||(i[39]=s("code",null,"label_smoothing",-1)),i[40]||(i[40]=a(" is a real number ")),s("mjx-container",f,[(h(),n("svg",b,i[29]||(i[29]=[t('',1)]))),i[30]||(i[30]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[41]||(i[41]=a(", then the value of ")),s("mjx-container",w,[(h(),n("svg",H,i[31]||(i[31]=[t('',1)]))),i[32]||(i[32]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[42]||(i[42]=a(" is:"))]),s("mjx-container",v,[(h(),n("svg",j,i[43]||(i[43]=[t('',1)]))),i[44]||(i[44]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mn",null,"0.5")])],-1))]),s("p",null,[i[47]||(i[47]=a("where ")),s("mjx-container",D,[(h(),n("svg",B,i[45]||(i[45]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[46]||(i[46]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[48]||(i[48]=a(" is the value of ")),i[49]||(i[49]=s("code",null,"label_smoothing",-1)),i[50]||(i[50]=a("."))]),i[53]||(i[53]=t(`
    Extended Help
    Example
    julia
    julia> bce = BinaryCrossEntropyLoss();
 
@@ -32,7 +32,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 julia> logitbce_ls = BinaryCrossEntropyLoss(label_smoothing=0.1, logits=Val(true));
 
 julia> logitbce_ls(y_model, y_bin) > logitbce(y_model, y_bin)
-true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/dev/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
+true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/dev/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
             1  0  1];
 
 julia> ŷ = [0.268941  0.5  0.268941
@@ -42,7 +42,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> BinaryFocalLoss(gamma=0)(ŷ, y) ≈ BinaryCrossEntropyLoss()(ŷ, y)
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
     label_smoothing::Union{Nothing, Real}=nothing)
    `,1)),s("p",null,[i[72]||(i[72]=a("Return the cross entropy loss which is used in multi-class classification tasks. The input, ")),s("mjx-container",S,[(h(),n("svg",N,i[70]||(i[70]=[t('',1)]))),i[71]||(i[71]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[73]||(i[73]=a(", is expected to be normalized (i.e. ")),i[74]||(i[74]=s("code",null,"softmax",-1)),i[75]||(i[75]=a(" output) if ")),i[76]||(i[76]=s("code",null,"logits",-1)),i[77]||(i[77]=a(" is ")),i[78]||(i[78]=s("code",null,"false",-1)),i[79]||(i[79]=a(" or ")),i[80]||(i[80]=s("code",null,"Val(false)",-1)),i[81]||(i[81]=a("."))]),i[112]||(i[112]=s("p",null,"The loss is calculated as:",-1)),s("mjx-container",z,[(h(),n("svg",P,i[82]||(i[82]=[t('',1)]))),i[83]||(i[83]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[92]||(i[92]=a("where ")),s("mjx-container",I,[(h(),n("svg",q,i[84]||(i[84]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D716",d:"M227 -11Q149 -11 95 41T40 174Q40 262 87 322Q121 367 173 396T287 430Q289 431 329 431H367Q382 426 382 411Q382 385 341 385H325H312Q191 385 154 277L150 265H327Q340 256 340 246Q340 228 320 219H138V217Q128 187 128 143Q128 77 160 52T231 26Q258 26 284 36T326 57T343 68Q350 68 354 58T358 39Q358 36 357 35Q354 31 337 21T289 0T227 -11Z",style:{"stroke-width":"3"}})])])],-1)]))),i[85]||(i[85]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"ϵ")])],-1))]),i[93]||(i[93]=a(" is added for numerical stability. The value of ")),s("mjx-container",G,[(h(),n("svg",X,i[86]||(i[86]=[t('',1)]))),i[87]||(i[87]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[94]||(i[94]=a(" is computed using label smoothing. If ")),i[95]||(i[95]=s("code",null,"label_smoothing",-1)),i[96]||(i[96]=a(" is ")),i[97]||(i[97]=s("code",null,"nothing",-1)),i[98]||(i[98]=a(", then no label smoothing is applied. If ")),i[99]||(i[99]=s("code",null,"label_smoothing",-1)),i[100]||(i[100]=a(" is a real number ")),s("mjx-container",J,[(h(),n("svg",U,i[88]||(i[88]=[t('',1)]))),i[89]||(i[89]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[101]||(i[101]=a(", then the value of ")),s("mjx-container",W,[(h(),n("svg",K,i[90]||(i[90]=[t('',1)]))),i[91]||(i[91]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[102]||(i[102]=a(" is calculated as:"))]),s("mjx-container",Y,[(h(),n("svg",$,i[103]||(i[103]=[t('',1)]))),i[104]||(i[104]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mtext",null,"size along dim")])],-1))]),s("p",null,[i[107]||(i[107]=a("where ")),s("mjx-container",_,[(h(),n("svg",s1,i[105]||(i[105]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[106]||(i[106]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[108]||(i[108]=a(" is the value of ")),i[109]||(i[109]=s("code",null,"label_smoothing",-1)),i[110]||(i[110]=a("."))]),i[113]||(i[113]=t(`
    Extended Help
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
@@ -64,7 +64,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> CrossEntropyLoss(label_smoothing=0.15)(y_model, y) ≈ 1.5776052f0
-true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
 
 julia> DiceCoeffLoss()(y_pred, 1:3)  ≈ 0.000992391663909964
 true
@@ -73,7 +73,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> DiceCoeffLoss()(reshape(y_pred, 3, 1), reshape(1:3, 3, 1)) ≈ 0.000992391663909964
-true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/dev/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
+true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/dev/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
 3×5 Matrix{Int64}:
@@ -88,20 +88,20 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
  0.665241   0.665241   0.665241   0.665241   0.665241
 
 julia> FocalLoss()(ŷ, y) ≈ 1.1277556f0
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.55
-true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",c1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",E1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",c1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",E1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
 
 julia> HuberLoss()(y_model, 1:3) ≈ 0.005000000000000009
 true
 
 julia> HuberLoss(delta=0.05)(y_model, 1:3) ≈ 0.003750000000000005
-true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
+true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
 2×2 Matrix{Int64}:
  1  0
  0  1
@@ -121,44 +121,44 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 0.0
 
 julia> KLDivergenceLoss(; epsilon=0)(p1, p2)
-Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
+Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.1
-true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
+true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.01
-true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
+true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
 
 julia> loss(Float32[1.1, 2.2, 3.3], 1:3) ≈ 0.009084041f0
 true
 
 julia> loss(Float32[0.9, 1.8, 2.7], 1:3) ≈ 0.011100831f0
-true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",z1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
+true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",z1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
 
 julia> PoissonLoss()(y_model, 1:3) ≈ 0.502312852219817
-true
    source
    `,3))]),s("details",P1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
+true
    source
    `,3))]),s("details",P1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
 
 julia> SiameseContrastiveLoss()(ŷ, 1:3) ≈ -4.833333333333333
 true
 
 julia> SiameseContrastiveLoss(margin=2)(ŷ, 1:3) ≈ -4.0
-true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
+true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.625
-true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
-foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
-getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
+true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
+foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
+getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
 istraining(::StaticBool)
 istraining(::Bool)
-istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
+istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
     ...
     @return y # optional (but recommended for best performance)
 end
@@ -274,7 +274,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
    You may also specify a name for the model, which will be used instead of the default printout, which gives a verbatim representation of the code used to construct the model:
    julia
    julia> model = @compact(w=rand(3), name="Linear(3 => 1)") do x
            @return sum(w .* x)
        end
-Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
+Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=@init_fn(rng->randn32(rng, 3, 2)),
            b=@init_fn(rng->randn32(rng, 3), :state)) do x
@@ -290,7 +290,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 (3,)
 
 julia> size(r([1, 2], ps, st)[1])
-(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
+(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=ones(3), w_fixed=@non_trainable(rand(3))) do x
            @return sum(w .* x .+ w_fixed)
@@ -310,5 +310,5 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> res isa Number
-true
    source
    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
-set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")
    Set the dispatch doctor preference for LuxCore and LuxLib packages.
    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.
    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.
    source
    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default};
+true

    source

    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
+set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")

    Set the dispatch doctor preference for LuxCore and LuxLib packages.

    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.

    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.

    source

    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default}; diff --git a/dev/assets/api_Lux_utilities.md.BJScatXv.lean.js b/dev/assets/api_Lux_utilities.md.BeOts_Il.lean.js similarity index 98% rename from dev/assets/api_Lux_utilities.md.BJScatXv.lean.js rename to dev/assets/api_Lux_utilities.md.BeOts_Il.lean.js index 33fcd97789..f4e1c36a73 100644 --- a/dev/assets/api_Lux_utilities.md.BJScatXv.lean.js +++ b/dev/assets/api_Lux_utilities.md.BeOts_Il.lean.js @@ -1,10 +1,10 @@ -import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.CCjWn1F9.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},P1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(c2,i,E2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState

    Training State containing:

    Internal fields:

    Warning

    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.

    source

    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
-    ts::TrainState)

    Compute the gradients of the objective function wrt parameters stored in ts.

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl

    Arguments

    Return

    A 4-Tuple containing:

    Known Limitations

    Aliased Gradients

    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.

    source

    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.

    In most cases you should use single_train_step! instead of this function.

    Return

    Returned values are the same as compute_gradients.

    source

    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.

    Return

    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).

    source

    ',5))]),i[290]||(i[290]=t('

    Loss Functions ​

    Loss Functions Objects take 2 forms of inputs:

    1. ŷ and y where ŷ is the predicted output and y is the target output.

    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.

    Warning

    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.

    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)

    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.

    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
+import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.CCjWn1F9.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},P1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(c2,i,E2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState
    Training State containing:
    • model: Lux model.
    • parameters: Trainable Variables of the model.
    • states: Non-trainable Variables of the model.
    • optimizer: Optimizer from Optimisers.jl.
    • optimizer_state: Optimizer State.
    • step: Number of updates of the parameters made.
    Internal fields:
    • cache: Cached values. Implementations are free to use this for whatever they want.
    • objective_function: Objective function might be cached.
    Warning
    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.
    source
    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
+    ts::TrainState)
    Compute the gradients of the objective function wrt parameters stored in ts.
    Backends & AD Packages
    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl
    Arguments
    • ad: Backend (from ADTypes.jl) used to compute the gradients.
    • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.
    • data: Data used to compute the gradients.
    • ts: Current Training State. See TrainState.
    Return
    A 4-Tuple containing:
    • grads: Computed Gradients.
    • loss: Loss from the objective function.
    • stats: Any computed statistics from the objective function.
    • ts: Updated Training State.
    Known Limitations
    • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.
    Aliased Gradients
    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.
    source
    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.
    In most cases you should use single_train_step! instead of this function.
    Return
    Returned values are the same as compute_gradients.
    source
    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.
    Return
    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).
    source
    ',5))]),i[290]||(i[290]=t('
    Loss Functions ​
    Loss Functions Objects take 2 forms of inputs:
    1. ŷ and y where ŷ is the predicted output and y is the target output.
    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.
    Warning
    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.
    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)
    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.
    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> mseloss(y_model, 1:3) ≈ 0.01
-true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",c,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
+true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",c,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
     label_smoothing::Union{Nothing, Real}=nothing,
     logits::Union{Bool, Val}=Val(false))
    Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
    Returns the binary cross entropy loss computed as:
    • If logits is either false or Val(false):
    `,4)),s("mjx-container",E,[(h(),n("svg",u,i[23]||(i[23]=[t('',1)]))),i[24]||(i[24]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[52]||(i[52]=s("ul",null,[s("li",null,[a("If "),s("code",null,"logits"),a(" is "),s("code",null,"true"),a(" or "),s("code",null,"Val(true)"),a(":")])],-1)),s("mjx-container",F,[(h(),n("svg",x,i[25]||(i[25]=[t('',1)]))),i[26]||(i[26]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"l"),s("mi",null,"o"),s("mi",null,"g"),s("mi",null,"σ"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[33]||(i[33]=a("The value of ")),s("mjx-container",C,[(h(),n("svg",L,i[27]||(i[27]=[t('',1)]))),i[28]||(i[28]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[34]||(i[34]=a(" is computed using label smoothing. If ")),i[35]||(i[35]=s("code",null,"label_smoothing",-1)),i[36]||(i[36]=a(" is ")),i[37]||(i[37]=s("code",null,"nothing",-1)),i[38]||(i[38]=a(", then no label smoothing is applied. If ")),i[39]||(i[39]=s("code",null,"label_smoothing",-1)),i[40]||(i[40]=a(" is a real number ")),s("mjx-container",f,[(h(),n("svg",b,i[29]||(i[29]=[t('',1)]))),i[30]||(i[30]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[41]||(i[41]=a(", then the value of ")),s("mjx-container",w,[(h(),n("svg",H,i[31]||(i[31]=[t('',1)]))),i[32]||(i[32]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[42]||(i[42]=a(" is:"))]),s("mjx-container",v,[(h(),n("svg",j,i[43]||(i[43]=[t('',1)]))),i[44]||(i[44]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mn",null,"0.5")])],-1))]),s("p",null,[i[47]||(i[47]=a("where ")),s("mjx-container",D,[(h(),n("svg",B,i[45]||(i[45]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[46]||(i[46]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[48]||(i[48]=a(" is the value of ")),i[49]||(i[49]=s("code",null,"label_smoothing",-1)),i[50]||(i[50]=a("."))]),i[53]||(i[53]=t(`
    Extended Help
    Example
    julia
    julia> bce = BinaryCrossEntropyLoss();
 
@@ -32,7 +32,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 julia> logitbce_ls = BinaryCrossEntropyLoss(label_smoothing=0.1, logits=Val(true));
 
 julia> logitbce_ls(y_model, y_bin) > logitbce(y_model, y_bin)
-true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/dev/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
+true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/dev/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
             1  0  1];
 
 julia> ŷ = [0.268941  0.5  0.268941
@@ -42,7 +42,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> BinaryFocalLoss(gamma=0)(ŷ, y) ≈ BinaryCrossEntropyLoss()(ŷ, y)
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
     label_smoothing::Union{Nothing, Real}=nothing)
    `,1)),s("p",null,[i[72]||(i[72]=a("Return the cross entropy loss which is used in multi-class classification tasks. The input, ")),s("mjx-container",S,[(h(),n("svg",N,i[70]||(i[70]=[t('',1)]))),i[71]||(i[71]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[73]||(i[73]=a(", is expected to be normalized (i.e. ")),i[74]||(i[74]=s("code",null,"softmax",-1)),i[75]||(i[75]=a(" output) if ")),i[76]||(i[76]=s("code",null,"logits",-1)),i[77]||(i[77]=a(" is ")),i[78]||(i[78]=s("code",null,"false",-1)),i[79]||(i[79]=a(" or ")),i[80]||(i[80]=s("code",null,"Val(false)",-1)),i[81]||(i[81]=a("."))]),i[112]||(i[112]=s("p",null,"The loss is calculated as:",-1)),s("mjx-container",z,[(h(),n("svg",P,i[82]||(i[82]=[t('',1)]))),i[83]||(i[83]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[92]||(i[92]=a("where ")),s("mjx-container",I,[(h(),n("svg",q,i[84]||(i[84]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D716",d:"M227 -11Q149 -11 95 41T40 174Q40 262 87 322Q121 367 173 396T287 430Q289 431 329 431H367Q382 426 382 411Q382 385 341 385H325H312Q191 385 154 277L150 265H327Q340 256 340 246Q340 228 320 219H138V217Q128 187 128 143Q128 77 160 52T231 26Q258 26 284 36T326 57T343 68Q350 68 354 58T358 39Q358 36 357 35Q354 31 337 21T289 0T227 -11Z",style:{"stroke-width":"3"}})])])],-1)]))),i[85]||(i[85]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"ϵ")])],-1))]),i[93]||(i[93]=a(" is added for numerical stability. The value of ")),s("mjx-container",G,[(h(),n("svg",X,i[86]||(i[86]=[t('',1)]))),i[87]||(i[87]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[94]||(i[94]=a(" is computed using label smoothing. If ")),i[95]||(i[95]=s("code",null,"label_smoothing",-1)),i[96]||(i[96]=a(" is ")),i[97]||(i[97]=s("code",null,"nothing",-1)),i[98]||(i[98]=a(", then no label smoothing is applied. If ")),i[99]||(i[99]=s("code",null,"label_smoothing",-1)),i[100]||(i[100]=a(" is a real number ")),s("mjx-container",J,[(h(),n("svg",U,i[88]||(i[88]=[t('',1)]))),i[89]||(i[89]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[101]||(i[101]=a(", then the value of ")),s("mjx-container",W,[(h(),n("svg",K,i[90]||(i[90]=[t('',1)]))),i[91]||(i[91]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[102]||(i[102]=a(" is calculated as:"))]),s("mjx-container",Y,[(h(),n("svg",$,i[103]||(i[103]=[t('',1)]))),i[104]||(i[104]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mtext",null,"size along dim")])],-1))]),s("p",null,[i[107]||(i[107]=a("where ")),s("mjx-container",_,[(h(),n("svg",s1,i[105]||(i[105]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[106]||(i[106]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[108]||(i[108]=a(" is the value of ")),i[109]||(i[109]=s("code",null,"label_smoothing",-1)),i[110]||(i[110]=a("."))]),i[113]||(i[113]=t(`
    Extended Help
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
@@ -64,7 +64,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> CrossEntropyLoss(label_smoothing=0.15)(y_model, y) ≈ 1.5776052f0
-true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
 
 julia> DiceCoeffLoss()(y_pred, 1:3)  ≈ 0.000992391663909964
 true
@@ -73,7 +73,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> DiceCoeffLoss()(reshape(y_pred, 3, 1), reshape(1:3, 3, 1)) ≈ 0.000992391663909964
-true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/dev/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
+true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/dev/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
 3×5 Matrix{Int64}:
@@ -88,20 +88,20 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
  0.665241   0.665241   0.665241   0.665241   0.665241
 
 julia> FocalLoss()(ŷ, y) ≈ 1.1277556f0
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.55
-true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",c1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",E1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",c1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",E1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
 
 julia> HuberLoss()(y_model, 1:3) ≈ 0.005000000000000009
 true
 
 julia> HuberLoss(delta=0.05)(y_model, 1:3) ≈ 0.003750000000000005
-true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
+true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
 2×2 Matrix{Int64}:
  1  0
  0  1
@@ -121,44 +121,44 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 0.0
 
 julia> KLDivergenceLoss(; epsilon=0)(p1, p2)
-Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
+Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.1
-true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
+true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.01
-true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
+true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
 
 julia> loss(Float32[1.1, 2.2, 3.3], 1:3) ≈ 0.009084041f0
 true
 
 julia> loss(Float32[0.9, 1.8, 2.7], 1:3) ≈ 0.011100831f0
-true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",z1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
+true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",z1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
 
 julia> PoissonLoss()(y_model, 1:3) ≈ 0.502312852219817
-true
    source
    `,3))]),s("details",P1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
+true
    source
    `,3))]),s("details",P1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
 
 julia> SiameseContrastiveLoss()(ŷ, 1:3) ≈ -4.833333333333333
 true
 
 julia> SiameseContrastiveLoss(margin=2)(ŷ, 1:3) ≈ -4.0
-true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
+true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.625
-true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
-foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
-getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
+true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
+foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
+getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
 istraining(::StaticBool)
 istraining(::Bool)
-istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
+istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
     ...
     @return y # optional (but recommended for best performance)
 end
@@ -274,7 +274,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
    You may also specify a name for the model, which will be used instead of the default printout, which gives a verbatim representation of the code used to construct the model:
    julia
    julia> model = @compact(w=rand(3), name="Linear(3 => 1)") do x
            @return sum(w .* x)
        end
-Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
+Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=@init_fn(rng->randn32(rng, 3, 2)),
            b=@init_fn(rng->randn32(rng, 3), :state)) do x
@@ -290,7 +290,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 (3,)
 
 julia> size(r([1, 2], ps, st)[1])
-(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
+(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=ones(3), w_fixed=@non_trainable(rand(3))) do x
            @return sum(w .* x .+ w_fixed)
@@ -310,5 +310,5 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> res isa Number
-true
    source
    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
-set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")
    Set the dispatch doctor preference for LuxCore and LuxLib packages.
    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.
    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.
    source
    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default};
+true

    source

    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
+set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")

    Set the dispatch doctor preference for LuxCore and LuxLib packages.

    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.

    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.

    source

    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default}; diff --git a/dev/assets/app.CHKBgYXT.js b/dev/assets/app.CiNod0R0.js similarity index 95% rename from dev/assets/app.CHKBgYXT.js rename to dev/assets/app.CiNod0R0.js index cedfd97554..579c1f406b 100644 --- a/dev/assets/app.CHKBgYXT.js +++ b/dev/assets/app.CiNod0R0.js @@ -1 +1 @@ -import{R as p}from"./chunks/theme.C74glUlb.js";import{R as o,a6 as u,a7 as c,a8 as l,a9 as f,aa as d,ab as m,ac as h,ad as g,ae as A,af as v,d as P,u as R,v as w,s as y,ag as C,ah as b,ai as E,a5 as S}from"./chunks/framework.CCjWn1F9.js";function i(e){if(e.extends){const a=i(e.extends);return{...a,...e,async enhanceApp(t){a.enhanceApp&&await a.enhanceApp(t),e.enhanceApp&&await e.enhanceApp(t)}}}return e}const s=i(p),T=P({name:"VitePressApp",setup(){const{site:e,lang:a,dir:t}=R();return w(()=>{y(()=>{document.documentElement.lang=a.value,document.documentElement.dir=t.value})}),e.value.router.prefetchLinks&&C(),b(),E(),s.setup&&s.setup(),()=>S(s.Layout)}});async function D(){globalThis.__VITEPRESS__=!0;const e=j(),a=_();a.provide(c,e);const t=l(e.route);return a.provide(f,t),a.component("Content",d),a.component("ClientOnly",m),Object.defineProperties(a.config.globalProperties,{$frontmatter:{get(){return t.frontmatter.value}},$params:{get(){return t.page.value.params}}}),s.enhanceApp&&await s.enhanceApp({app:a,router:e,siteData:h}),{app:a,router:e,data:t}}function _(){return g(T)}function j(){let e=o,a;return A(t=>{let n=v(t),r=null;return n&&(e&&(a=n),(e||a===n)&&(n=n.replace(/\.js$/,".lean.js")),r=import(n)),o&&(e=!1),r},s.NotFound)}o&&D().then(({app:e,router:a,data:t})=>{a.go().then(()=>{u(a.route,t.site),e.mount("#app")})});export{D as createApp}; +import{R as p}from"./chunks/theme.DgYiELuP.js";import{R as o,a6 as u,a7 as c,a8 as l,a9 as f,aa as d,ab as m,ac as h,ad as g,ae as A,af as v,d as P,u as R,v as w,s as y,ag as C,ah as b,ai as E,a5 as S}from"./chunks/framework.CCjWn1F9.js";function i(e){if(e.extends){const a=i(e.extends);return{...a,...e,async enhanceApp(t){a.enhanceApp&&await a.enhanceApp(t),e.enhanceApp&&await e.enhanceApp(t)}}}return e}const s=i(p),T=P({name:"VitePressApp",setup(){const{site:e,lang:a,dir:t}=R();return w(()=>{y(()=>{document.documentElement.lang=a.value,document.documentElement.dir=t.value})}),e.value.router.prefetchLinks&&C(),b(),E(),s.setup&&s.setup(),()=>S(s.Layout)}});async function D(){globalThis.__VITEPRESS__=!0;const e=j(),a=_();a.provide(c,e);const t=l(e.route);return a.provide(f,t),a.component("Content",d),a.component("ClientOnly",m),Object.defineProperties(a.config.globalProperties,{$frontmatter:{get(){return t.frontmatter.value}},$params:{get(){return t.page.value.params}}}),s.enhanceApp&&await s.enhanceApp({app:a,router:e,siteData:h}),{app:a,router:e,data:t}}function _(){return g(T)}function j(){let e=o,a;return A(t=>{let n=v(t),r=null;return n&&(e&&(a=n),(e||a===n)&&(n=n.replace(/\.js$/,".lean.js")),r=import(n)),o&&(e=!1),r},s.NotFound)}o&&D().then(({app:e,router:a,data:t})=>{a.go().then(()=>{u(a.route,t.site),e.mount("#app")})});export{D as createApp}; diff --git a/dev/assets/chunks/@localSearchIndexroot.Bf_mqVD5.js b/dev/assets/chunks/@localSearchIndexroot.Bf_mqVD5.js deleted file mode 100644 index d144b3e33a..0000000000 --- a/dev/assets/chunks/@localSearchIndexroot.Bf_mqVD5.js +++ /dev/null @@ -1 +0,0 @@ -const e='{"documentCount":261,"nextId":261,"documentIds":{"0":"/dev/api/Accelerator_Support/MLDataDevices#MLDataDevices-API","1":"/dev/api/Accelerator_Support/MLDataDevices#preferences","2":"/dev/api/Accelerator_Support/MLDataDevices#Data-Transfer","3":"/dev/api/Accelerator_Support/MLDataDevices#miscellaneous","4":"/dev/api/Accelerator_Support/MLDataDevices#Multi-GPU-Support","5":"/dev/api/Accelerator_Support/MLDataDevices#iteration","6":"/dev/api/Building_Blocks/LuxCore#luxcore","7":"/dev/api/Building_Blocks/LuxCore#Abstract-Types","8":"/dev/api/Building_Blocks/LuxCore#general","9":"/dev/api/Building_Blocks/LuxCore#parameters","10":"/dev/api/Building_Blocks/LuxCore#states","11":"/dev/api/Building_Blocks/LuxCore#Layer-size","12":"/dev/api/Building_Blocks/LuxLib#LuxLib-API","13":"/dev/api/Building_Blocks/LuxLib#Apply-Activation","14":"/dev/api/Building_Blocks/LuxLib#Batched-Operations","15":"/dev/api/Building_Blocks/LuxLib#Bias-Activation","16":"/dev/api/Building_Blocks/LuxLib#Convolutional-Layers","17":"/dev/api/Building_Blocks/LuxLib#dropout","18":"/dev/api/Building_Blocks/LuxLib#Fully-Connected-Layers","19":"/dev/api/Building_Blocks/LuxLib#normalization","20":"/dev/api/Building_Blocks/LuxLib#Helper-Functions","21":"/dev/api/Building_Blocks/WeightInitializers#WeightInitializers-API","22":"/dev/api/Building_Blocks/WeightInitializers#Supported-RNG-Types-WeightInit","23":"/dev/api/Building_Blocks/WeightInitializers#API-Reference","24":"/dev/api/Building_Blocks/WeightInitializers#Main-Functions","25":"/dev/api/Building_Blocks/WeightInitializers#Other-Convenience-Functions","26":"/dev/api/Lux/autodiff#autodiff-lux-helpers","27":"/dev/api/Lux/autodiff#JVP-and-VJP-Wrappers","28":"/dev/api/Lux/autodiff#Batched-AD","29":"/dev/api/Lux/autodiff#Nested-2nd-Order-AD","30":"/dev/api/Lux/contrib#Experimental-Features","31":"/dev/api/Lux/contrib#Parameter-Freezing","32":"/dev/api/Lux/contrib#Map-over-Layer","33":"/dev/api/Lux/contrib#Debugging-Functionality","34":"/dev/api/Lux/contrib#Tied-Parameters","35":"/dev/api/Lux/interop#Interoperability-between-Lux-and-other-packages","36":"/dev/api/Lux/interop#Switching-from-older-frameworks","37":"/dev/api/Lux/interop#flux-to-lux-migrate-api","38":"/dev/api/Lux/interop#Using-a-different-backend-for-Lux","39":"/dev/api/Lux/interop#Lux-Models-to-Simple-Chains","40":"/dev/api/Lux/distributed_utils#Distributed-Utils","41":"/dev/api/Lux/distributed_utils#communication-backends","42":"/dev/api/Lux/distributed_utils#initialization","43":"/dev/api/Lux/distributed_utils#Helper-Functions","44":"/dev/api/Lux/distributed_utils#Communication-Primitives","45":"/dev/api/Lux/distributed_utils#Optimizers.jl-Integration","46":"/dev/api/Lux/distributed_utils#MLUtils.jl-Integration","47":"/dev/api/Testing_Functionality/LuxTestUtils#luxtestutils","48":"/dev/api/Testing_Functionality/LuxTestUtils#Testing-using-JET.jl","49":"/dev/api/Testing_Functionality/LuxTestUtils#Gradient-Correctness","50":"/dev/api/Testing_Functionality/LuxTestUtils#Extensions-to-@test","51":"/dev/api/Lux/utilities#utilities","52":"/dev/api/Lux/utilities#Training-API","53":"/dev/api/Lux/utilities#Loss-Functions","54":"/dev/api/Lux/utilities#LuxOps-Module","55":"/dev/api/Lux/utilities#Recursive-Operations","56":"/dev/api/Lux/utilities#Updating-Floating-Point-Precision","57":"/dev/api/Lux/utilities#Element-Type-Matching","58":"/dev/api/Lux/utilities#Stateful-Layer","59":"/dev/api/Lux/utilities#Compact-Layer","60":"/dev/api/Lux/utilities#miscellaneous","61":"/dev/api/Lux/layers#Built-In-Layers","62":"/dev/api/Lux/layers#containers","63":"/dev/api/Lux/layers#Convolutional-Layers","64":"/dev/api/Lux/layers#Dropout-Layers","65":"/dev/api/Lux/layers#Pooling-Layers","66":"/dev/api/Lux/layers#Recurrent-Layers","67":"/dev/api/Lux/layers#Linear-Layers","68":"/dev/api/Lux/layers#Misc.-Helper-Layers","69":"/dev/api/Lux/layers#Normalization-Layers","70":"/dev/api/Lux/layers#upsampling","71":"/dev/#How-to-Install-Lux.jl?","72":"/dev/#Want-GPU-Support?","73":"/dev/#Want-XLA-Support?","74":"/dev/introduction/citation#citation","75":"/dev/introduction/#getting-started","76":"/dev/introduction/#installation","77":"/dev/introduction/#quickstart","78":"/dev/introduction/#Defining-Custom-Layers","79":"/dev/introduction/#Additional-Packages","80":"/dev/introduction/#GPU-Support","81":"/dev/introduction/overview#Why-we-wrote-Lux?","82":"/dev/introduction/overview#Design-Principles","83":"/dev/introduction/overview#Why-use-Lux-over-Flux?","84":"/dev/introduction/resources#Resources-to-Get-Started","85":"/dev/manual/autodiff#autodiff-lux","86":"/dev/manual/autodiff#overview","87":"/dev/manual/autodiff#autodiff-recommendations","88":"/dev/manual/autodiff#Support-Class","89":"/dev/manual/autodiff#footnotes","90":"/dev/introduction/updating_to_v1#updating-to-v1","91":"/dev/introduction/updating_to_v1#LuxLib.jl","92":"/dev/introduction/updating_to_v1#Breaking-Changes","93":"/dev/introduction/updating_to_v1#New-Major-Features","94":"/dev/introduction/updating_to_v1#LuxCore.jl","95":"/dev/introduction/updating_to_v1#Breaking-Changes-2","96":"/dev/introduction/updating_to_v1#New-Major-Features-2","97":"/dev/introduction/updating_to_v1#WeightInitializers.jl","98":"/dev/introduction/updating_to_v1#MLDataDevices.jl","99":"/dev/introduction/updating_to_v1#Breaking-Changes-3","100":"/dev/introduction/updating_to_v1#New-Major-Features-3","101":"/dev/introduction/updating_to_v1#Lux.jl","102":"/dev/introduction/updating_to_v1#Breaking-Changes-(Removed-Functionality)","103":"/dev/introduction/updating_to_v1#Breaking-Changes-(Moved-Functionality)","104":"/dev/introduction/updating_to_v1#Breaking-Changes-(Changes-in-Defaults)","105":"/dev/introduction/updating_to_v1#New-Features","106":"/dev/manual/compiling_lux_models#reactant-compilation","107":"/dev/manual/compiling_lux_models#compile_lux_model_trainstate","108":"/dev/manual/debugging#debug-lux-layers","109":"/dev/manual/debugging#Incorrect-Model-Specification:-Dimension-Mismatch-Problems","110":"/dev/manual/debugging#Tracking-down-NaNs","111":"/dev/manual/debugging#conclusion","112":"/dev/manual/distributed_utils#Distributed-Data-Parallel-Training","113":"/dev/manual/distributed_utils#Guide-to-Integrating-DistributedUtils-into-your-code","114":"/dev/manual/distributed_utils#Migration-Guide-from-FluxMPI.jl","115":"/dev/manual/distributed_utils#Removed-Functionality","116":"/dev/manual/distributed_utils#Key-Differences","117":"/dev/manual/distributed_utils#Known-Shortcomings","118":"/dev/manual/dispatch_custom_input#Dispatching-on-Custom-Input-Types","119":"/dev/manual/dispatch_custom_input#Which-function-should-participate-in-dispatch?","120":"/dev/manual/dispatch_custom_input#Concrete-Example","121":"/dev/manual/dispatch_custom_input#Time-Dependent-Chain-Implementation","122":"/dev/manual/dispatch_custom_input#Running-the-TDChain","123":"/dev/manual/dispatch_custom_input#Writing-the-Correct-Dispatch-Rules","124":"/dev/manual/dispatch_custom_input#Using-the-Same-Input-for-Non-TD-Models","125":"/dev/manual/freezing_model_parameters#freezing-model-parameters","126":"/dev/manual/freezing_model_parameters#Freezing-Layers-of-a-Particular-Kind","127":"/dev/manual/freezing_model_parameters#Freezing-by-Layer-Name","128":"/dev/manual/freezing_model_parameters#Freezing-Part-of-the-Parameters","129":"/dev/manual/freezing_model_parameters#Freezing-Part-of-a-Chain","130":"/dev/manual/gpu_management#GPU-Management","131":"/dev/manual/gpu_management#Automatic-Backend-Management-(Recommended-Approach)","132":"/dev/manual/gpu_management#Manual-Backend-Management","133":"/dev/manual/migrate_from_flux#migrate-from-flux","134":"/dev/manual/migrate_from_flux#Implementing-Custom-Layers","135":"/dev/manual/migrate_from_flux#Certain-Important-Implementation-Details","136":"/dev/manual/migrate_from_flux#Training/Inference-Mode","137":"/dev/manual/migrate_from_flux#Can-we-still-use-Flux-Layers?","138":"/dev/manual/interface#lux-interface","139":"/dev/manual/interface#Layer-Interface","140":"/dev/manual/interface#Singular-Layer","141":"/dev/manual/interface#Container-Layer","142":"/dev/manual/interface#Parameter-Interface","143":"/dev/manual/interface#State-Interface","144":"/dev/manual/nested_autodiff#nested_autodiff","145":"/dev/manual/nested_autodiff#Loss-Function-containing-Jacobian-Computation","146":"/dev/manual/nested_autodiff#Using-Batched-Jacobian-for-Multiple-Inputs","147":"/dev/manual/nested_autodiff#Loss-Function-contains-Gradient-Computation","148":"/dev/manual/nested_autodiff#Loss-Function-computing-the-Jacobian-of-the-Parameters","149":"/dev/manual/nested_autodiff#Hutchinson-Trace-Estimation","150":"/dev/manual/nested_autodiff#Computing-using-the-Vector-Jacobian-Product","151":"/dev/manual/nested_autodiff#Computing-using-the-Jacobian-Vector-Product","152":"/dev/manual/nested_autodiff#Computing-using-the-Full-Jacobian","153":"/dev/manual/nn_inside_gpu_kernels#Neural-Networks-Inside-GPU-Kernels","154":"/dev/manual/performance_pitfalls#Performance-Pitfalls-and-How-to-Catch-Them","155":"/dev/manual/performance_pitfalls#Spurious-Type-Promotion","156":"/dev/manual/performance_pitfalls#Scalar-Indexing-on-GPU-Arrays","157":"/dev/manual/performance_pitfalls#Type-Instabilities","158":"/dev/manual/performance_pitfalls#Faster-Primitives","159":"/dev/manual/performance_pitfalls#Optional-Dependencies-for-Performance","160":"/dev/manual/performance_pitfalls#Data-Loading-and-Device-Transfer","161":"/dev/manual/preferences#Preferences-for-Lux.jl","162":"/dev/manual/preferences#Nested-Automatic-Differentiation","163":"/dev/manual/preferences#gpu-aware-mpi-preferences","164":"/dev/manual/preferences#GPU-Backend-Selection","165":"/dev/manual/preferences#automatic-eltypes-preference","166":"/dev/manual/preferences#dispatch-doctor-preference","167":"/dev/manual/preferences#disable_loop_vectorization","168":"/dev/manual/weight_initializers#Initializing-Weights","169":"/dev/manual/weight_initializers#Quick-examples","170":"/dev/tutorials/beginner/1_Basics#Julia-and-Lux-for-the-Uninitiated","171":"/dev/tutorials/beginner/1_Basics#arrays","172":"/dev/tutorials/beginner/1_Basics#CUDA-Arrays","173":"/dev/tutorials/beginner/1_Basics#im-mutability","174":"/dev/tutorials/beginner/1_Basics#Managing-Randomness","175":"/dev/tutorials/beginner/1_Basics#Automatic-Differentiation","176":"/dev/tutorials/beginner/1_Basics#gradients","177":"/dev/tutorials/beginner/1_Basics#Jacobian-Vector-Product","178":"/dev/tutorials/beginner/1_Basics#Vector-Jacobian-Product","179":"/dev/tutorials/beginner/1_Basics#Linear-Regression","180":"/dev/tutorials/beginner/1_Basics#appendix","181":"/dev/tutorials/beginner/3_SimpleRNN#Training-a-Simple-LSTM","182":"/dev/tutorials/beginner/3_SimpleRNN#Package-Imports","183":"/dev/tutorials/beginner/3_SimpleRNN#dataset","184":"/dev/tutorials/beginner/3_SimpleRNN#Creating-a-Classifier","185":"/dev/tutorials/beginner/3_SimpleRNN#Using-the-@compact-API","186":"/dev/tutorials/beginner/3_SimpleRNN#Defining-Accuracy,-Loss-and-Optimiser","187":"/dev/tutorials/beginner/3_SimpleRNN#Training-the-Model","188":"/dev/tutorials/beginner/3_SimpleRNN#Saving-the-Model","189":"/dev/tutorials/beginner/3_SimpleRNN#appendix","190":"/dev/tutorials/beginner/4_SimpleChains#MNIST-Classification-with-SimpleChains","191":"/dev/tutorials/beginner/4_SimpleChains#Package-Imports","192":"/dev/tutorials/beginner/4_SimpleChains#Loading-MNIST","193":"/dev/tutorials/beginner/4_SimpleChains#Define-the-Model","194":"/dev/tutorials/beginner/4_SimpleChains#Helper-Functions","195":"/dev/tutorials/beginner/4_SimpleChains#Define-the-Training-Loop","196":"/dev/tutorials/beginner/4_SimpleChains#Finally-Training-the-Model","197":"/dev/tutorials/beginner/4_SimpleChains#appendix","198":"/dev/tutorials/#tutorials","199":"/dev/tutorials/#beginner-tutorials","200":"/dev/tutorials/#intermediate-tutorials","201":"/dev/tutorials/#advanced-tutorials","202":"/dev/tutorials/#larger-models","203":"/dev/tutorials/#selected-3rd-party-tutorials","204":"/dev/tutorials/beginner/5_OptimizationIntegration#Optimization-Lux-Tutorial","205":"/dev/tutorials/beginner/5_OptimizationIntegration#Imports-packages","206":"/dev/tutorials/beginner/5_OptimizationIntegration#Generate-some-training-data","207":"/dev/tutorials/beginner/5_OptimizationIntegration#Define-the-DataLoader","208":"/dev/tutorials/beginner/5_OptimizationIntegration#Training-the-model","209":"/dev/tutorials/beginner/5_OptimizationIntegration#Plotting-the-results","210":"/dev/tutorials/beginner/5_OptimizationIntegration#appendix","211":"/dev/tutorials/intermediate/1_NeuralODE#MNIST-Classification-using-Neural-ODEs","212":"/dev/tutorials/intermediate/1_NeuralODE#Package-Imports","213":"/dev/tutorials/intermediate/1_NeuralODE#Loading-MNIST","214":"/dev/tutorials/intermediate/1_NeuralODE#Define-the-Neural-ODE-Layer","215":"/dev/tutorials/intermediate/1_NeuralODE#Create-and-Initialize-the-Neural-ODE-Layer","216":"/dev/tutorials/intermediate/1_NeuralODE#Define-Utility-Functions","217":"/dev/tutorials/intermediate/1_NeuralODE#training","218":"/dev/tutorials/intermediate/1_NeuralODE#Alternate-Implementation-using-Stateful-Layer","219":"/dev/tutorials/intermediate/1_NeuralODE#Train-the-new-Stateful-Neural-ODE","220":"/dev/tutorials/intermediate/1_NeuralODE#Type-Stability","221":"/dev/tutorials/intermediate/1_NeuralODE#appendix","222":"/dev/tutorials/intermediate/3_HyperNet#Training-a-HyperNetwork-on-MNIST-and-FashionMNIST","223":"/dev/tutorials/intermediate/3_HyperNet#Package-Imports","224":"/dev/tutorials/intermediate/3_HyperNet#Loading-Datasets","225":"/dev/tutorials/intermediate/3_HyperNet#Implement-a-HyperNet-Layer","226":"/dev/tutorials/intermediate/3_HyperNet#Create-and-Initialize-the-HyperNet","227":"/dev/tutorials/intermediate/3_HyperNet#Define-Utility-Functions","228":"/dev/tutorials/intermediate/3_HyperNet#training","229":"/dev/tutorials/intermediate/3_HyperNet#appendix","230":"/dev/tutorials/beginner/2_PolynomialFitting#Fitting-a-Polynomial-using-MLP","231":"/dev/tutorials/beginner/2_PolynomialFitting#Package-Imports","232":"/dev/tutorials/beginner/2_PolynomialFitting#dataset","233":"/dev/tutorials/beginner/2_PolynomialFitting#Neural-Network","234":"/dev/tutorials/beginner/2_PolynomialFitting#optimizer","235":"/dev/tutorials/beginner/2_PolynomialFitting#Loss-Function","236":"/dev/tutorials/beginner/2_PolynomialFitting#training","237":"/dev/tutorials/beginner/2_PolynomialFitting#appendix","238":"/dev/tutorials/intermediate/4_PINN2DPDE#Training-a-PINN-on-2D-PDE","239":"/dev/tutorials/intermediate/4_PINN2DPDE#Package-Imports","240":"/dev/tutorials/intermediate/4_PINN2DPDE#Problem-Definition","241":"/dev/tutorials/intermediate/4_PINN2DPDE#Define-the-Neural-Networks","242":"/dev/tutorials/intermediate/4_PINN2DPDE#Define-the-Loss-Functions","243":"/dev/tutorials/intermediate/4_PINN2DPDE#Generate-the-Data","244":"/dev/tutorials/intermediate/4_PINN2DPDE#training","245":"/dev/tutorials/intermediate/4_PINN2DPDE#Visualizing-the-Results","246":"/dev/tutorials/intermediate/4_PINN2DPDE#appendix","247":"/dev/tutorials/intermediate/2_BayesianNN#Bayesian-Neural-Network","248":"/dev/tutorials/intermediate/2_BayesianNN#Generating-data","249":"/dev/tutorials/intermediate/2_BayesianNN#Building-the-Neural-Network","250":"/dev/tutorials/intermediate/2_BayesianNN#Prediction-Visualization","251":"/dev/tutorials/intermediate/2_BayesianNN#appendix","252":"/dev/tutorials/advanced/1_GravitationalWaveForm#Training-a-Neural-ODE-to-Model-Gravitational-Waveforms","253":"/dev/tutorials/advanced/1_GravitationalWaveForm#Package-Imports","254":"/dev/tutorials/advanced/1_GravitationalWaveForm#Define-some-Utility-Functions","255":"/dev/tutorials/advanced/1_GravitationalWaveForm#Simulating-the-True-Model","256":"/dev/tutorials/advanced/1_GravitationalWaveForm#Defiing-a-Neural-Network-Model","257":"/dev/tutorials/advanced/1_GravitationalWaveForm#Setting-Up-for-Training-the-Neural-Network","258":"/dev/tutorials/advanced/1_GravitationalWaveForm#Training-the-Neural-Network","259":"/dev/tutorials/advanced/1_GravitationalWaveForm#Visualizing-the-Results","260":"/dev/tutorials/advanced/1_GravitationalWaveForm#appendix"},"fieldIds":{"title":0,"titles":1,"text":2},"fieldLength":{"0":[1,1,14],"1":[1,1,46],"2":[2,1,156],"3":[1,1,194],"4":[3,1,93],"5":[1,1,114],"6":[1,1,38],"7":[2,1,160],"8":[1,1,214],"9":[1,1,19],"10":[1,1,62],"11":[2,1,77],"12":[1,1,5],"13":[2,1,100],"14":[2,1,44],"15":[2,1,68],"16":[2,1,128],"17":[1,1,160],"18":[3,1,132],"19":[1,1,232],"20":[2,1,91],"21":[1,1,16],"22":[3,1,27],"23":[2,1,1],"24":[2,3,342],"25":[3,3,92],"26":[3,1,1],"27":[4,3,72],"28":[2,3,86],"29":[4,3,13],"30":[2,1,54],"31":[2,2,131],"32":[3,2,124],"33":[2,2,140],"34":[2,2,118],"35":[6,1,1],"36":[4,6,1],"37":[4,10,172],"38":[6,6,1],"39":[5,11,188],"40":[2,1,11],"41":[1,2,29],"42":[1,2,81],"43":[2,2,18],"44":[2,2,58],"45":[3,2,31],"46":[3,2,35],"47":[1,1,48],"48":[4,1,91],"49":[2,1,127],"50":[3,1,33],"51":[1,1,1],"52":[2,1,228],"53":[2,1,449],"54":[2,1,134],"55":[2,1,164],"56":[4,1,45],"57":[3,1,108],"58":[2,1,112],"59":[2,1,358],"60":[1,1,54],"61":[3,1,1],"62":[1,3,292],"63":[2,3,257],"64":[2,3,115],"65":[2,3,168],"66":[2,3,361],"67":[2,3,191],"68":[3,3,243],"69":[2,3,294],"70":[1,3,165],"71":[6,1,47],"72":[4,1,34],"73":[4,1,29],"74":[1,1,67],"75":[2,1,1],"76":[1,2,51],"77":[1,2,402],"78":[3,2,181],"79":[2,2,36],"80":[2,2,17],"81":[5,1,63],"82":[2,5,58],"83":[6,5,218],"84":[4,1,48],"85":[2,1,35],"86":[1,2,34],"87":[1,2,50],"88":[2,2,65],"89":[1,2,67],"90":[4,1,41],"91":[2,4,1],"92":[2,6,27],"93":[3,6,22],"94":[2,4,1],"95":[2,6,119],"96":[3,6,19],"97":[2,4,30],"98":[2,4,48],"99":[2,6,22],"100":[3,6,30],"101":[2,4,1],"102":[5,5,117],"103":[5,5,54],"104":[5,5,68],"105":[2,5,63],"106":[6,1,493],"107":[4,6,156],"108":[3,1,76],"109":[6,3,156],"110":[3,3,190],"111":[1,3,51],"112":[4,1,27],"113":[7,4,122],"114":[5,4,42],"115":[2,9,28],"116":[2,9,79],"117":[2,4,52],"118":[5,1,1],"119":[7,5,36],"120":[2,5,40],"121":[4,7,72],"122":[3,7,73],"123":[5,7,47],"124":[8,7,46],"125":[3,1,27],"126":[6,3,123],"127":[4,3,53],"128":[5,3,43],"129":[5,3,49],"130":[2,1,71],"131":[6,2,89],"132":[3,2,44],"133":[5,1,67],"134":[3,5,164],"135":[4,5,1],"136":[3,9,49],"137":[7,5,34],"138":[2,1,80],"139":[2,2,1],"140":[2,3,239],"141":[2,3,158],"142":[2,2,151],"143":[2,2,47],"144":[3,1,169],"145":[5,3,265],"146":[6,8,155],"147":[5,3,219],"148":[7,3,216],"149":[3,3,88],"150":[6,6,46],"151":[6,6,31],"152":[5,6,113],"153":[5,1,286],"154":[7,1,18],"155":[3,7,149],"156":[5,7,32],"157":[2,7,39],"158":[2,7,43],"159":[4,7,39],"160":[5,7,75],"161":[4,1,49],"162":[3,4,21],"163":[4,4,40],"164":[3,4,33],"165":[3,4,23],"166":[2,4,61],"167":[4,4,58],"168":[2,1,133],"169":[2,2,64],"170":[6,1,77],"171":[1,6,286],"172":[2,7,86],"173":[3,6,110],"174":[2,6,130],"175":[2,6,96],"176":[1,8,72],"177":[3,8,106],"178":[3,8,30],"179":[2,6,253],"180":[1,6,150],"181":[4,1,38],"182":[2,4,12],"183":[1,4,95],"184":[3,4,160],"185":[4,4,74],"186":[5,4,56],"187":[3,4,532],"188":[3,4,48],"189":[1,4,150],"190":[4,1,33],"191":[2,4,15],"192":[2,4,60],"193":[3,4,64],"194":[2,4,38],"195":[4,4,73],"196":[4,4,127],"197":[1,4,110],"198":[1,1,1],"199":[2,1,1],"200":[2,1,1],"201":[2,1,1],"202":[2,1,41],"203":[4,1,65],"204":[6,1,78],"205":[2,6,29],"206":[4,6,58],"207":[3,6,70],"208":[3,6,239],"209":[3,6,47],"210":[1,6,150],"211":[5,1,24],"212":[2,5,23],"213":[2,5,64],"214":[5,5,139],"215":[7,5,60],"216":[3,5,37],"217":[1,5,241],"218":[5,5,60],"219":[6,5,70],"220":[2,5,180],"221":[1,5,150],"222":[7,1,1],"223":[2,7,18],"224":[2,7,41],"225":[4,7,76],"226":[5,7,42],"227":[3,7,39],"228":[1,7,256],"229":[1,7,150],"230":[5,1,16],"231":[2,5,12],"232":[1,5,274],"233":[2,5,37],"234":[1,5,18],"235":[2,5,112],"236":[1,5,150],"237":[1,5,150],"238":[6,1,45],"239":[2,6,31],"240":[2,6,26],"241":[4,6,55],"242":[4,6,71],"243":[3,6,86],"244":[1,6,1041],"245":[3,6,73],"246":[1,6,150],"247":[3,1,72],"248":[2,3,105],"249":[4,3,548],"250":[2,3,174],"251":[1,3,103],"252":[8,1,21],"253":[2,8,13],"254":[4,8,202],"255":[4,8,124],"256":[5,8,1354],"257":[7,8,73],"258":[4,8,1249],"259":[3,8,83],"260":[1,8,103]},"averageFieldLength":[2.954022988505747,3.6091954022988504,107.4367816091954],"storedFields":{"0":{"title":"MLDataDevices","titles":[]},"1":{"title":"Preferences","titles":["MLDataDevices"]},"2":{"title":"Data Transfer","titles":["MLDataDevices"]},"3":{"title":"Miscellaneous","titles":["MLDataDevices"]},"4":{"title":"Multi-GPU Support","titles":["MLDataDevices"]},"5":{"title":"Iteration","titles":["MLDataDevices"]},"6":{"title":"LuxCore","titles":[]},"7":{"title":"Abstract Types","titles":["LuxCore"]},"8":{"title":"General","titles":["LuxCore"]},"9":{"title":"Parameters","titles":["LuxCore"]},"10":{"title":"States","titles":["LuxCore"]},"11":{"title":"Layer size","titles":["LuxCore"]},"12":{"title":"LuxLib","titles":[]},"13":{"title":"Apply Activation","titles":["LuxLib"]},"14":{"title":"Batched Operations","titles":["LuxLib"]},"15":{"title":"Bias Activation","titles":["LuxLib"]},"16":{"title":"Convolutional Layers","titles":["LuxLib"]},"17":{"title":"Dropout","titles":["LuxLib"]},"18":{"title":"Fully Connected Layers","titles":["LuxLib"]},"19":{"title":"Normalization","titles":["LuxLib"]},"20":{"title":"Helper Functions","titles":["LuxLib"]},"21":{"title":"WeightInitializers","titles":[]},"22":{"title":"Supported RNG Types","titles":["WeightInitializers"]},"23":{"title":"API Reference","titles":["WeightInitializers"]},"24":{"title":"Main Functions","titles":["WeightInitializers","API Reference"]},"25":{"title":"Other Convenience Functions","titles":["WeightInitializers","API Reference"]},"26":{"title":"Automatic Differentiation Helpers","titles":[]},"27":{"title":"JVP & VJP Wrappers","titles":["Automatic Differentiation Helpers"]},"28":{"title":"Batched AD","titles":["Automatic Differentiation Helpers"]},"29":{"title":"Nested 2nd Order AD","titles":["Automatic Differentiation Helpers"]},"30":{"title":"Experimental Features","titles":[]},"31":{"title":"Parameter Freezing","titles":["Experimental Features"]},"32":{"title":"Map over Layer","titles":["Experimental Features"]},"33":{"title":"Debugging Functionality","titles":["Experimental Features"]},"34":{"title":"Tied Parameters","titles":["Experimental Features"]},"35":{"title":"Interoperability between Lux and other packages","titles":[]},"36":{"title":"Switching from older frameworks","titles":["Interoperability between Lux and other packages"]},"37":{"title":"Flux Models to Lux Models","titles":["Interoperability between Lux and other packages","Switching from older frameworks"]},"38":{"title":"Using a different backend for Lux","titles":["Interoperability between Lux and other packages"]},"39":{"title":"Lux Models to Simple Chains","titles":["Interoperability between Lux and other packages","Using a different backend for Lux"]},"40":{"title":"Distributed Utils","titles":[]},"41":{"title":"Backends","titles":["Distributed Utils"]},"42":{"title":"Initialization","titles":["Distributed Utils"]},"43":{"title":"Helper Functions","titles":["Distributed Utils"]},"44":{"title":"Communication Primitives","titles":["Distributed Utils"]},"45":{"title":"Optimizers.jl Integration","titles":["Distributed Utils"]},"46":{"title":"MLUtils.jl Integration","titles":["Distributed Utils"]},"47":{"title":"LuxTestUtils","titles":[]},"48":{"title":"Testing using JET.jl","titles":["LuxTestUtils"]},"49":{"title":"Gradient Correctness","titles":["LuxTestUtils"]},"50":{"title":"Extensions to @test","titles":["LuxTestUtils"]},"51":{"title":"Utilities","titles":[]},"52":{"title":"Training API","titles":["Utilities"]},"53":{"title":"Loss Functions","titles":["Utilities"]},"54":{"title":"LuxOps Module","titles":["Utilities"]},"55":{"title":"Recursive Operations","titles":["Utilities"]},"56":{"title":"Updating Floating Point Precision","titles":["Utilities"]},"57":{"title":"Element Type Matching","titles":["Utilities"]},"58":{"title":"Stateful Layer","titles":["Utilities"]},"59":{"title":"Compact Layer","titles":["Utilities"]},"60":{"title":"Miscellaneous","titles":["Utilities"]},"61":{"title":"Built-In Layers","titles":[]},"62":{"title":"Containers","titles":["Built-In Layers"]},"63":{"title":"Convolutional Layers","titles":["Built-In Layers"]},"64":{"title":"Dropout Layers","titles":["Built-In Layers"]},"65":{"title":"Pooling Layers","titles":["Built-In Layers"]},"66":{"title":"Recurrent Layers","titles":["Built-In Layers"]},"67":{"title":"Linear Layers","titles":["Built-In Layers"]},"68":{"title":"Misc. Helper Layers","titles":["Built-In Layers"]},"69":{"title":"Normalization Layers","titles":["Built-In Layers"]},"70":{"title":"Upsampling","titles":["Built-In Layers"]},"71":{"title":"How to Install Lux.jl?","titles":[]},"72":{"title":"Want GPU Support?","titles":[]},"73":{"title":"Want XLA Support?","titles":[]},"74":{"title":"Citation","titles":[]},"75":{"title":"Getting Started","titles":[]},"76":{"title":"Installation","titles":["Getting Started"]},"77":{"title":"Quickstart","titles":["Getting Started"]},"78":{"title":"Defining Custom Layers","titles":["Getting Started"]},"79":{"title":"Additional Packages","titles":["Getting Started"]},"80":{"title":"GPU Support","titles":["Getting Started"]},"81":{"title":"Why we wrote Lux?","titles":[]},"82":{"title":"Design Principles","titles":["Why we wrote Lux?"]},"83":{"title":"Why use Lux over Flux?","titles":["Why we wrote Lux?"]},"84":{"title":"Resources to Get Started","titles":[]},"85":{"title":"Automatic Differentiation","titles":[]},"86":{"title":"Overview","titles":["Automatic Differentiation"]},"87":{"title":"Recommendations","titles":["Automatic Differentiation"]},"88":{"title":"Support Class","titles":["Automatic Differentiation"]},"89":{"title":"Footnotes","titles":["Automatic Differentiation"]},"90":{"title":"Updating to Lux v1","titles":[]},"91":{"title":"LuxLib.jl","titles":["Updating to Lux v1"]},"92":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxLib.jl"]},"93":{"title":"New Major Features","titles":["Updating to Lux v1","LuxLib.jl"]},"94":{"title":"LuxCore.jl","titles":["Updating to Lux v1"]},"95":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxCore.jl"]},"96":{"title":"New Major Features","titles":["Updating to Lux v1","LuxCore.jl"]},"97":{"title":"WeightInitializers.jl","titles":["Updating to Lux v1"]},"98":{"title":"MLDataDevices.jl","titles":["Updating to Lux v1"]},"99":{"title":"Breaking Changes","titles":["Updating to Lux v1","MLDataDevices.jl"]},"100":{"title":"New Major Features","titles":["Updating to Lux v1","MLDataDevices.jl"]},"101":{"title":"Lux.jl","titles":["Updating to Lux v1"]},"102":{"title":"Breaking Changes (Removed Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"103":{"title":"Breaking Changes (Moved Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"104":{"title":"Breaking Changes (Changes in Defaults)","titles":["Updating to Lux v1","Lux.jl"]},"105":{"title":"New Features","titles":["Updating to Lux v1","Lux.jl"]},"106":{"title":"Compiling Lux Models using Reactant.jl","titles":[]},"107":{"title":"Using the TrainState API","titles":["Compiling Lux Models using Reactant.jl"]},"108":{"title":"Debugging Lux Models","titles":[]},"109":{"title":"Incorrect Model Specification: Dimension Mismatch Problems","titles":["Debugging Lux Models"]},"110":{"title":"Tracking down NaNs","titles":["Debugging Lux Models"]},"111":{"title":"Conclusion","titles":["Debugging Lux Models"]},"112":{"title":"Distributed Data Parallel Training","titles":[]},"113":{"title":"Guide to Integrating DistributedUtils into your code","titles":["Distributed Data Parallel Training"]},"114":{"title":"Migration Guide from FluxMPI.jl","titles":["Distributed Data Parallel Training"]},"115":{"title":"Removed Functionality","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"116":{"title":"Key Differences","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"117":{"title":"Known Shortcomings","titles":["Distributed Data Parallel Training"]},"118":{"title":"Dispatching on Custom Input Types","titles":[]},"119":{"title":"Which function should participate in dispatch?","titles":["Dispatching on Custom Input Types"]},"120":{"title":"Concrete Example","titles":["Dispatching on Custom Input Types"]},"121":{"title":"Time-Dependent Chain Implementation","titles":["Dispatching on Custom Input Types","Concrete Example"]},"122":{"title":"Running the TDChain","titles":["Dispatching on Custom Input Types","Concrete Example"]},"123":{"title":"Writing the Correct Dispatch Rules","titles":["Dispatching on Custom Input Types","Concrete Example"]},"124":{"title":"Using the Same Input for Non-TD Models","titles":["Dispatching on Custom Input Types","Concrete Example"]},"125":{"title":"Freezing Model Parameters","titles":[]},"126":{"title":"Freezing Layers of a Particular Kind","titles":["Freezing Model Parameters"]},"127":{"title":"Freezing by Layer Name","titles":["Freezing Model Parameters"]},"128":{"title":"Freezing Part of the Parameters","titles":["Freezing Model Parameters"]},"129":{"title":"Freezing Part of a Chain","titles":["Freezing Model Parameters"]},"130":{"title":"GPU Management","titles":[]},"131":{"title":"Automatic Backend Management (Recommended Approach)","titles":["GPU Management"]},"132":{"title":"Manual Backend Management","titles":["GPU Management"]},"133":{"title":"Migrating from Flux to Lux","titles":[]},"134":{"title":"Implementing Custom Layers","titles":["Migrating from Flux to Lux"]},"135":{"title":"Certain Important Implementation Details","titles":["Migrating from Flux to Lux"]},"136":{"title":"Training/Inference Mode","titles":["Migrating from Flux to Lux","Certain Important Implementation Details"]},"137":{"title":"Can we still use Flux Layers?","titles":["Migrating from Flux to Lux"]},"138":{"title":"Lux Interface","titles":[]},"139":{"title":"Layer Interface","titles":["Lux Interface"]},"140":{"title":"Singular Layer","titles":["Lux Interface","Layer Interface"]},"141":{"title":"Container Layer","titles":["Lux Interface","Layer Interface"]},"142":{"title":"Parameter Interface","titles":["Lux Interface"]},"143":{"title":"State Interface","titles":["Lux Interface"]},"144":{"title":"Nested Automatic Differentiation","titles":[]},"145":{"title":"Loss Function containing Jacobian Computation","titles":["Nested Automatic Differentiation"]},"146":{"title":"Using Batched Jacobian for Multiple Inputs","titles":["Nested Automatic Differentiation","Loss Function containing Jacobian Computation"]},"147":{"title":"Loss Function contains Gradient Computation","titles":["Nested Automatic Differentiation"]},"148":{"title":"Loss Function computing the Jacobian of the Parameters","titles":["Nested Automatic Differentiation"]},"149":{"title":"Hutchinson Trace Estimation","titles":["Nested Automatic Differentiation"]},"150":{"title":"Computing using the Vector-Jacobian Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"151":{"title":"Computing using the Jacobian-Vector Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"152":{"title":"Computing using the Full Jacobian","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"153":{"title":"Neural Networks Inside GPU Kernels","titles":[]},"154":{"title":"Performance Pitfalls & How to Catch Them","titles":[]},"155":{"title":"Spurious Type-Promotion","titles":["Performance Pitfalls & How to Catch Them"]},"156":{"title":"Scalar Indexing on GPU Arrays","titles":["Performance Pitfalls & How to Catch Them"]},"157":{"title":"Type Instabilities","titles":["Performance Pitfalls & How to Catch Them"]},"158":{"title":"Faster Primitives","titles":["Performance Pitfalls & How to Catch Them"]},"159":{"title":"Optional Dependencies for Performance","titles":["Performance Pitfalls & How to Catch Them"]},"160":{"title":"Data Loading and Device Transfer","titles":["Performance Pitfalls & How to Catch Them"]},"161":{"title":"Preferences for Lux.jl","titles":[]},"162":{"title":"Nested Automatic Differentiation","titles":["Preferences for Lux.jl"]},"163":{"title":"GPU-Aware MPI Support","titles":["Preferences for Lux.jl"]},"164":{"title":"GPU Backend Selection","titles":["Preferences for Lux.jl"]},"165":{"title":"Automatic Eltype Conversion","titles":["Preferences for Lux.jl"]},"166":{"title":"Dispatch Doctor","titles":["Preferences for Lux.jl"]},"167":{"title":"Disabling Loop Vectorization / Octavian","titles":["Preferences for Lux.jl"]},"168":{"title":"Initializing Weights","titles":[]},"169":{"title":"Quick examples","titles":["Initializing Weights"]},"170":{"title":"Julia & Lux for the Uninitiated","titles":[]},"171":{"title":"Arrays","titles":["Julia & Lux for the Uninitiated"]},"172":{"title":"CUDA Arrays","titles":["Julia & Lux for the Uninitiated","Arrays"]},"173":{"title":"(Im)mutability","titles":["Julia & Lux for the Uninitiated"]},"174":{"title":"Managing Randomness","titles":["Julia & Lux for the Uninitiated"]},"175":{"title":"Automatic Differentiation","titles":["Julia & Lux for the Uninitiated"]},"176":{"title":"Gradients","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"177":{"title":"Jacobian-Vector Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"178":{"title":"Vector-Jacobian Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"179":{"title":"Linear Regression","titles":["Julia & Lux for the Uninitiated"]},"180":{"title":"Appendix","titles":["Julia & Lux for the Uninitiated"]},"181":{"title":"Training a Simple LSTM","titles":[]},"182":{"title":"Package Imports","titles":["Training a Simple LSTM"]},"183":{"title":"Dataset","titles":["Training a Simple LSTM"]},"184":{"title":"Creating a Classifier","titles":["Training a Simple LSTM"]},"185":{"title":"Using the @compact API","titles":["Training a Simple LSTM"]},"186":{"title":"Defining Accuracy, Loss and Optimiser","titles":["Training a Simple LSTM"]},"187":{"title":"Training the Model","titles":["Training a Simple LSTM"]},"188":{"title":"Saving the Model","titles":["Training a Simple LSTM"]},"189":{"title":"Appendix","titles":["Training a Simple LSTM"]},"190":{"title":"MNIST Classification with SimpleChains","titles":[]},"191":{"title":"Package Imports","titles":["MNIST Classification with SimpleChains"]},"192":{"title":"Loading MNIST","titles":["MNIST Classification with SimpleChains"]},"193":{"title":"Define the Model","titles":["MNIST Classification with SimpleChains"]},"194":{"title":"Helper Functions","titles":["MNIST Classification with SimpleChains"]},"195":{"title":"Define the Training Loop","titles":["MNIST Classification with SimpleChains"]},"196":{"title":"Finally Training the Model","titles":["MNIST Classification with SimpleChains"]},"197":{"title":"Appendix","titles":["MNIST Classification with SimpleChains"]},"198":{"title":"Tutorials","titles":[]},"199":{"title":"Beginner Tutorials","titles":["Tutorials"]},"200":{"title":"Intermediate Tutorials","titles":["Tutorials"]},"201":{"title":"Advanced Tutorials","titles":["Tutorials"]},"202":{"title":"Larger Models","titles":["Tutorials"]},"203":{"title":"Selected 3rd Party Tutorials","titles":["Tutorials"]},"204":{"title":"Training Lux Models using Optimization.jl","titles":[]},"205":{"title":"Imports packages","titles":["Training Lux Models using Optimization.jl"]},"206":{"title":"Generate some training data","titles":["Training Lux Models using Optimization.jl"]},"207":{"title":"Define the DataLoader","titles":["Training Lux Models using Optimization.jl"]},"208":{"title":"Training the model","titles":["Training Lux Models using Optimization.jl"]},"209":{"title":"Plotting the results","titles":["Training Lux Models using Optimization.jl"]},"210":{"title":"Appendix","titles":["Training Lux Models using Optimization.jl"]},"211":{"title":"MNIST Classification using Neural ODEs","titles":[]},"212":{"title":"Package Imports","titles":["MNIST Classification using Neural ODEs"]},"213":{"title":"Loading MNIST","titles":["MNIST Classification using Neural ODEs"]},"214":{"title":"Define the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"215":{"title":"Create and Initialize the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"216":{"title":"Define Utility Functions","titles":["MNIST Classification using Neural ODEs"]},"217":{"title":"Training","titles":["MNIST Classification using Neural ODEs"]},"218":{"title":"Alternate Implementation using Stateful Layer","titles":["MNIST Classification using Neural ODEs"]},"219":{"title":"Train the new Stateful Neural ODE","titles":["MNIST Classification using Neural ODEs"]},"220":{"title":"Type Stability","titles":["MNIST Classification using Neural ODEs"]},"221":{"title":"Appendix","titles":["MNIST Classification using Neural ODEs"]},"222":{"title":"Training a HyperNetwork on MNIST and FashionMNIST","titles":[]},"223":{"title":"Package Imports","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"224":{"title":"Loading Datasets","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"225":{"title":"Implement a HyperNet Layer","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"226":{"title":"Create and Initialize the HyperNet","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"227":{"title":"Define Utility Functions","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"228":{"title":"Training","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"229":{"title":"Appendix","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"230":{"title":"Fitting a Polynomial using MLP","titles":[]},"231":{"title":"Package Imports","titles":["Fitting a Polynomial using MLP"]},"232":{"title":"Dataset","titles":["Fitting a Polynomial using MLP"]},"233":{"title":"Neural Network","titles":["Fitting a Polynomial using MLP"]},"234":{"title":"Optimizer","titles":["Fitting a Polynomial using MLP"]},"235":{"title":"Loss Function","titles":["Fitting a Polynomial using MLP"]},"236":{"title":"Training","titles":["Fitting a Polynomial using MLP"]},"237":{"title":"Appendix","titles":["Fitting a Polynomial using MLP"]},"238":{"title":"Training a PINN on 2D PDE","titles":[]},"239":{"title":"Package Imports","titles":["Training a PINN on 2D PDE"]},"240":{"title":"Problem Definition","titles":["Training a PINN on 2D PDE"]},"241":{"title":"Define the Neural Networks","titles":["Training a PINN on 2D PDE"]},"242":{"title":"Define the Loss Functions","titles":["Training a PINN on 2D PDE"]},"243":{"title":"Generate the Data","titles":["Training a PINN on 2D PDE"]},"244":{"title":"Training","titles":["Training a PINN on 2D PDE"]},"245":{"title":"Visualizing the Results","titles":["Training a PINN on 2D PDE"]},"246":{"title":"Appendix","titles":["Training a PINN on 2D PDE"]},"247":{"title":"Bayesian Neural Network","titles":[]},"248":{"title":"Generating data","titles":["Bayesian Neural Network"]},"249":{"title":"Building the Neural Network","titles":["Bayesian Neural Network"]},"250":{"title":"Prediction Visualization","titles":["Bayesian Neural Network"]},"251":{"title":"Appendix","titles":["Bayesian Neural Network"]},"252":{"title":"Training a Neural ODE to Model Gravitational Waveforms","titles":[]},"253":{"title":"Package Imports","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"254":{"title":"Define some Utility Functions","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"255":{"title":"Simulating the True Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"256":{"title":"Defiing a Neural Network Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"257":{"title":"Setting Up for Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"258":{"title":"Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"259":{"title":"Visualizing the Results","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"260":{"title":"Appendix","titles":["Training a Neural ODE to Model Gravitational Waveforms"]}},"dirtCount":0,"index":[["π",{"2":{"254":2,"255":1}}],["√",{"2":{"254":2}}],["√t",{"2":{"24":2}}],["↦",{"2":{"254":1}}],["ϕ̇",{"2":{"255":2,"256":2}}],["ϕ",{"2":{"254":5,"255":1,"256":1}}],["χ̇",{"2":{"255":2,"256":2}}],["χ",{"2":{"254":4,"255":4,"256":2}}],["₋₋₋kwargs₋₋₋",{"2":{"220":4}}],["₋₋₋no",{"2":{"220":3}}],["│",{"2":{"220":3,"244":4}}],["─",{"2":{"220":3}}],["θ|x",{"2":{"250":2}}],["θ",{"2":{"208":2,"249":2,"250":9,"257":3}}],["÷",{"2":{"183":4}}],["∥22we",{"2":{"179":1}}],["⟶∑i=1k12∥yi−fw",{"2":{"179":1}}],["∇f",{"2":{"176":3}}],["∇offending",{"2":{"110":2}}],["$",{"2":{"245":1}}],["$i",{"2":{"174":2,"250":1}}],["$name",{"2":{"32":1}}],["⋮",{"2":{"153":4}}],["∞",{"2":{"145":4,"146":4,"147":4,"148":4,"152":8}}],["∘",{"2":{"144":4,"147":1,"242":3}}],["~",{"2":{"122":3,"145":2,"249":2}}],["└──",{"2":{"220":3}}],["└",{"2":{"109":1,"145":2,"244":4}}],["┌",{"2":{"109":1,"145":2,"244":4}}],["∂w",{"2":{"242":2}}],["∂v",{"2":{"242":2}}],["∂y",{"2":{"242":4}}],["∂u",{"2":{"242":10}}],["∂t",{"2":{"147":7,"242":2}}],["∂xyt",{"2":{"242":4}}],["∂x",{"2":{"145":7,"146":7,"148":7,"152":11,"242":4}}],["∂ps",{"2":{"106":4,"145":8,"146":8,"147":8,"148":8,"152":11}}],["∂f∂x",{"2":{"27":2}}],["↩︎",{"2":{"89":8}}],["❌",{"2":{"86":5}}],["❓",{"2":{"86":6}}],["\\tfashionmnist\\ttraining",{"2":{"228":1}}],["\\tfashionmnist\\ttime",{"2":{"228":50}}],["\\ttest",{"2":{"217":45,"219":9,"228":102}}],["\\ttime",{"2":{"217":45,"219":9}}],["\\tloss",{"2":{"107":11}}],["\\t",{"2":{"78":11,"196":60,"228":51,"236":6,"244":400}}],["quadrupole",{"2":{"254":10}}],["quadratic",{"2":{"232":1,"236":1}}],["quantiles",{"2":{"249":1}}],["questions",{"2":{"84":3,"144":1,"175":1}}],["quick",{"0":{"169":1},"2":{"170":1}}],["quickstart",{"0":{"77":1},"2":{"84":1}}],["quite",{"2":{"81":1,"173":1,"175":1}}],["quoting",{"2":{"106":1}}],["quot",{"2":{"1":2,"8":2,"17":4,"18":1,"19":8,"24":12,"53":8,"55":2,"57":16,"59":2,"60":6,"62":4,"68":2,"69":2,"77":4,"116":2,"134":4,"155":2,"164":8}}],["⋅n+z⋅hprevarguments",{"2":{"66":1}}],["↗",{"2":{"62":2}}],["↘",{"2":{"62":2}}],["→",{"2":{"62":5}}],["8325109235756322e",{"2":{"258":1}}],["8321446080941926e",{"2":{"258":1}}],["8326939597203966e",{"2":{"258":1}}],["8327304f",{"2":{"256":1}}],["838308356725419e",{"2":{"258":1}}],["838748973369222e",{"2":{"258":1}}],["838798f",{"2":{"256":1}}],["838451f",{"2":{"256":1}}],["835819f",{"2":{"256":1}}],["8372836237685824e",{"2":{"258":1}}],["8376",{"2":{"249":1}}],["8374606",{"2":{"235":1}}],["837991",{"2":{"148":1}}],["831961f",{"2":{"256":1}}],["83115s\\ttraining",{"2":{"228":1}}],["8317274f",{"2":{"179":1}}],["834524936944457e",{"2":{"258":1}}],["834150919661371e",{"2":{"258":1}}],["834695f",{"2":{"256":1}}],["8346457",{"2":{"232":2}}],["8348s\\ttraining",{"2":{"217":1}}],["8340728",{"2":{"148":1}}],["83s",{"2":{"196":2}}],["83",{"2":{"196":1,"217":6,"219":1}}],["836007981531355e",{"2":{"258":1}}],["836328975620413e",{"2":{"258":1}}],["836329f",{"2":{"256":1}}],["836",{"2":{"193":2}}],["8393",{"2":{"171":1}}],["8393034",{"2":{"171":1}}],["839303",{"2":{"171":2}}],["8339622f",{"2":{"256":1}}],["833518",{"2":{"172":1}}],["8337032",{"2":{"126":1}}],["833333333333333",{"2":{"53":1}}],["8943532f",{"2":{"256":1}}],["894694305967406e",{"2":{"258":1}}],["8946",{"2":{"249":1}}],["895673f",{"2":{"256":1}}],["8904",{"2":{"249":1}}],["8985687635276715e",{"2":{"258":1}}],["89858943",{"2":{"147":1}}],["898879f",{"2":{"256":1}}],["8984",{"2":{"249":1}}],["897745f",{"2":{"256":1}}],["8972864",{"2":{"232":1}}],["8976378",{"2":{"232":2}}],["89s",{"2":{"196":1}}],["89",{"2":{"196":1,"228":1,"249":2}}],["8965529343647317e",{"2":{"258":1}}],["896181281312519e",{"2":{"258":1}}],["896729159416418e",{"2":{"258":1}}],["8963052f",{"2":{"256":1}}],["896282f",{"2":{"256":1}}],["8966172f",{"2":{"256":1}}],["8969011",{"2":{"232":1}}],["896",{"2":{"193":2}}],["899676974598491e",{"2":{"258":1}}],["8990",{"2":{"249":1}}],["899766",{"2":{"171":1}}],["8999977",{"2":{"148":1}}],["812233062043259e",{"2":{"258":1}}],["8125742f",{"2":{"256":1}}],["8102334908473548e",{"2":{"258":1}}],["810257f",{"2":{"256":1}}],["810499283526083e",{"2":{"258":1}}],["810851614175347e",{"2":{"258":1}}],["8108143f",{"2":{"256":1}}],["810199216258969e",{"2":{"258":1}}],["8100",{"2":{"249":1}}],["8158896f",{"2":{"256":1}}],["815336",{"2":{"148":1}}],["816281",{"2":{"232":1}}],["8110236",{"2":{"232":2}}],["811985e",{"2":{"208":1}}],["8199s\\ttraining",{"2":{"217":1}}],["81",{"2":{"196":4,"217":7,"219":1,"228":10,"249":1}}],["8173783982243132",{"2":{"153":2}}],["8146702473339157e",{"2":{"258":1}}],["8146973e",{"2":{"147":2}}],["814318108788655e",{"2":{"258":1}}],["814466090999923e",{"2":{"258":1}}],["81446123",{"2":{"235":1}}],["8144423f",{"2":{"256":1}}],["814566f",{"2":{"256":1}}],["814054f",{"2":{"256":1}}],["81481",{"2":{"217":2,"219":1}}],["8147778",{"2":{"106":1}}],["887120020134511e",{"2":{"258":1}}],["884102203626996e",{"2":{"258":1}}],["884017080330087e",{"2":{"258":1}}],["884206f",{"2":{"256":1}}],["883167635009892e",{"2":{"258":1}}],["883700555925537e",{"2":{"258":1}}],["8837296f",{"2":{"256":1}}],["8839592f",{"2":{"256":1}}],["883664f0",{"2":{"145":1}}],["8888703674137614e",{"2":{"258":1}}],["88889",{"2":{"217":1}}],["8889195f",{"2":{"256":1}}],["881487674465144e",{"2":{"258":1}}],["8815167f",{"2":{"256":1}}],["881955f",{"2":{"256":1}}],["8812",{"2":{"249":1}}],["882123043805916e",{"2":{"258":1}}],["8821635f",{"2":{"256":1}}],["8821303",{"2":{"232":1}}],["88214725",{"2":{"148":1}}],["88951594",{"2":{"232":1}}],["889",{"2":{"221":1}}],["88s",{"2":{"196":1}}],["88",{"2":{"196":2,"228":14}}],["8852221",{"2":{"148":1}}],["863978838779784e",{"2":{"258":1}}],["863811f",{"2":{"256":1}}],["8638202f",{"2":{"256":1}}],["8638",{"2":{"249":1}}],["8614092227059548e",{"2":{"258":1}}],["861306f",{"2":{"256":1}}],["864013388039402e",{"2":{"258":1}}],["865269743548675e",{"2":{"258":1}}],["865217f",{"2":{"256":1}}],["865882373656356e",{"2":{"258":1}}],["8656864f",{"2":{"256":1}}],["868717612555944e",{"2":{"258":1}}],["8689",{"2":{"249":1}}],["8688824",{"2":{"232":1}}],["8680s\\ttraining",{"2":{"217":1}}],["86614174",{"2":{"232":2}}],["86",{"2":{"196":3,"228":3}}],["8601828553599953",{"2":{"174":1}}],["8601870861800127",{"2":{"153":2}}],["867598f",{"2":{"256":1}}],["8672",{"2":{"249":1}}],["867202",{"2":{"145":1}}],["8674763752817414",{"2":{"153":1}}],["8674763752817416",{"2":{"153":1}}],["869632",{"2":{"145":1}}],["862480861009647",{"2":{"153":2}}],["86290836",{"2":{"126":1}}],["8626451f",{"2":{"106":5}}],["8542519807642565e",{"2":{"258":1}}],["8541596",{"2":{"145":1}}],["85824093762756e",{"2":{"258":1}}],["8583716f",{"2":{"256":1}}],["8565315f",{"2":{"256":1}}],["851572968089565e",{"2":{"258":1}}],["851716f",{"2":{"256":1}}],["85185",{"2":{"217":2}}],["8534",{"2":{"249":1}}],["85344f",{"2":{"77":1}}],["85s",{"2":{"196":1}}],["85",{"2":{"196":4,"217":4,"228":7}}],["85004298285759e",{"2":{"258":1}}],["8501",{"2":{"244":1}}],["85050875",{"2":{"232":1}}],["850",{"2":{"193":2}}],["8576512",{"2":{"232":1}}],["857330e",{"2":{"208":1}}],["8573820474674845",{"2":{"171":1}}],["857257",{"2":{"172":1}}],["857126",{"2":{"172":1}}],["859405",{"2":{"147":1,"148":1}}],["8c79",{"2":{"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["803646552512888e",{"2":{"258":1}}],["8031547",{"2":{"232":1}}],["8031496",{"2":{"232":2}}],["808033600365818e",{"2":{"258":1}}],["8089343f",{"2":{"256":1}}],["807031939722761e",{"2":{"258":1}}],["8067217f",{"2":{"256":1}}],["8067698",{"2":{"147":1}}],["8091948056004725e",{"2":{"258":1}}],["809239f",{"2":{"256":1}}],["8094",{"2":{"249":1}}],["809456",{"2":{"171":1}}],["8050",{"2":{"249":1}}],["8013359",{"2":{"232":1}}],["80",{"2":{"180":1,"189":1,"196":1,"210":1,"217":10,"219":2,"221":1,"228":1,"229":1,"237":1,"246":1,"248":1}}],["8048109",{"2":{"235":1}}],["80465555",{"2":{"179":1}}],["80423f",{"2":{"77":1}}],["8001",{"2":{"179":1,"244":1}}],["800792614736468",{"2":{"153":2}}],["8003627",{"2":{"147":1}}],["800",{"2":{"107":1}}],["8f",{"2":{"107":1,"179":1}}],["873629812742245e",{"2":{"258":1}}],["873218f",{"2":{"256":1}}],["876329692578255e",{"2":{"258":1}}],["876209163761416e",{"2":{"258":1}}],["8765184f",{"2":{"256":1}}],["872517700731323e",{"2":{"258":1}}],["872238090043803e",{"2":{"258":1}}],["872073037560773e",{"2":{"258":1}}],["872737f",{"2":{"256":1}}],["8724313f",{"2":{"256":1}}],["8759412f",{"2":{"256":1}}],["875082",{"2":{"232":1}}],["8787",{"2":{"249":1}}],["8740157",{"2":{"232":2}}],["87431f",{"2":{"77":1}}],["87s",{"2":{"196":1}}],["8771",{"2":{"249":1}}],["8775",{"2":{"249":1}}],["877497",{"2":{"177":1,"178":1}}],["87702435",{"2":{"106":1}}],["87",{"2":{"109":1,"228":7}}],["87143785f0",{"2":{"77":1}}],["8237744104818825e",{"2":{"258":1}}],["8236438f",{"2":{"256":1}}],["821833807569876e",{"2":{"258":1}}],["8217807f",{"2":{"256":1}}],["824179453448208e",{"2":{"258":1}}],["825303904700446e",{"2":{"258":1}}],["8257547f",{"2":{"256":1}}],["8255f",{"2":{"256":1}}],["820811861453015e",{"2":{"258":1}}],["820816984579966e",{"2":{"258":1}}],["8205284014101566e",{"2":{"258":1}}],["820955f",{"2":{"256":1}}],["8203609f",{"2":{"256":1}}],["82030004",{"2":{"232":1}}],["829824808588582e",{"2":{"258":1}}],["829484f",{"2":{"256":1}}],["82945824",{"2":{"77":1}}],["829732f",{"2":{"256":1}}],["8277",{"2":{"249":1}}],["8271",{"2":{"249":1}}],["8271f",{"2":{"77":1}}],["82293516",{"2":{"232":1}}],["8224s\\ttraining",{"2":{"217":1}}],["82",{"2":{"196":2,"217":6,"219":2,"228":2}}],["82805836",{"2":{"126":1}}],["8",{"2":{"53":1,"62":2,"68":1,"77":27,"106":28,"145":1,"147":2,"148":3,"171":2,"173":4,"183":1,"187":15,"196":2,"208":2,"217":5,"219":1,"228":2,"232":1,"234":1,"236":1,"249":8,"251":1,"256":75,"258":76,"260":1}}],["847043711492695e",{"2":{"258":1}}],["847020991630263e",{"2":{"258":1}}],["847106",{"2":{"148":1}}],["843420913052272e",{"2":{"258":1}}],["8433718f",{"2":{"256":1}}],["840409790788863e",{"2":{"258":1}}],["84019f",{"2":{"256":1}}],["840588",{"2":{"145":1}}],["846614024266207e",{"2":{"258":1}}],["8469856f",{"2":{"256":1}}],["846937f",{"2":{"256":1}}],["846346f",{"2":{"256":1}}],["8464",{"2":{"249":1}}],["846457",{"2":{"148":1}}],["841901592307562e",{"2":{"258":1}}],["8419723f",{"2":{"256":1}}],["841231",{"2":{"148":1}}],["84972f",{"2":{"256":1}}],["8499767",{"2":{"232":1}}],["8490221",{"2":{"142":1}}],["842115256698744e",{"2":{"258":1}}],["8421153f",{"2":{"256":1}}],["8424188",{"2":{"232":1}}],["8425196",{"2":{"232":2}}],["84s",{"2":{"196":5}}],["8454118",{"2":{"148":1}}],["84576",{"2":{"77":1}}],["84",{"2":{"39":2,"193":6,"196":2,"217":1,"219":1,"228":16}}],["δ",{"2":{"53":1,"110":3,"206":2}}],["ϵ",{"2":{"53":1}}],["ŷ",{"2":{"53":6}}],["∈",{"2":{"53":2,"254":1}}],["∗y+α∗size",{"2":{"53":1}}],["∗y+α∗0",{"2":{"53":1}}],["∗y^−logσ",{"2":{"53":1}}],["∗log⁡",{"2":{"53":1}}],["−log⁡",{"2":{"53":1}}],["−∑y~log⁡",{"2":{"53":1}}],["−",{"2":{"53":1}}],["−y~∗log⁡",{"2":{"53":1}}],["−x",{"2":{"24":1}}],["≈∑θ∼p",{"2":{"250":1}}],["≈",{"2":{"53":25,"59":1,"131":1,"132":1,"174":2}}],["9f",{"2":{"244":8}}],["9371384702104172e",{"2":{"258":1}}],["9370079",{"2":{"232":2}}],["93611139613029e",{"2":{"258":1}}],["9369683f",{"2":{"256":1}}],["936521f",{"2":{"256":1}}],["930050864006232e",{"2":{"258":1}}],["930151f",{"2":{"256":1}}],["930603f",{"2":{"256":1}}],["930726397310587e",{"2":{"258":1}}],["9307",{"2":{"249":1}}],["935723899935839e",{"2":{"258":1}}],["9357703f",{"2":{"256":1}}],["935456f",{"2":{"256":1}}],["935390e",{"2":{"208":1}}],["9334187523694303e",{"2":{"258":1}}],["933069130891232e",{"2":{"258":1}}],["933055f",{"2":{"256":1}}],["9338737",{"2":{"148":1}}],["9317165401249315e",{"2":{"258":1}}],["931786756463552e",{"2":{"258":1}}],["9315197107257145e",{"2":{"258":1}}],["931564f",{"2":{"256":1}}],["931845f",{"2":{"256":1}}],["9310",{"2":{"249":1}}],["93",{"2":{"196":1,"228":2}}],["9340093",{"2":{"147":1}}],["914899489757273e",{"2":{"258":1}}],["917681496832297e",{"2":{"258":1}}],["9172697f",{"2":{"256":1}}],["915869657076024e",{"2":{"258":1}}],["915458f",{"2":{"256":1}}],["9123146f",{"2":{"256":1}}],["912781",{"2":{"152":1}}],["9127817",{"2":{"152":1}}],["912782",{"2":{"152":1}}],["9166",{"2":{"249":1}}],["9161865",{"2":{"232":1}}],["9164896",{"2":{"148":1}}],["910376202589805e",{"2":{"258":1}}],["910377726651017e",{"2":{"258":1}}],["9101567f",{"2":{"256":1}}],["9102543f",{"2":{"256":1}}],["910251",{"2":{"147":1}}],["9109",{"2":{"249":1}}],["9108207",{"2":{"232":1}}],["91",{"2":{"228":3}}],["918918",{"2":{"168":1}}],["913782391622606e",{"2":{"258":1}}],["913399868910976e",{"2":{"258":1}}],["9133418f",{"2":{"256":1}}],["913563f",{"2":{"256":1}}],["9139614",{"2":{"148":1}}],["9130077",{"2":{"106":2}}],["955827273611378e",{"2":{"258":1}}],["955555f",{"2":{"256":1}}],["9567438142124586e",{"2":{"258":1}}],["956562773853718e",{"2":{"258":1}}],["9569262f",{"2":{"256":1}}],["9594993252322064e",{"2":{"258":1}}],["959974962853483e",{"2":{"258":1}}],["959182",{"2":{"148":1}}],["953040253597263e",{"2":{"258":1}}],["9538403f",{"2":{"256":1}}],["9529168f",{"2":{"256":1}}],["95840897031777e",{"2":{"258":1}}],["9585396f",{"2":{"256":1}}],["9588915f",{"2":{"256":1}}],["9515",{"2":{"249":1}}],["9517",{"2":{"249":1}}],["9511775",{"2":{"147":1}}],["9542975643902326e",{"2":{"258":1}}],["95480246680904e",{"2":{"258":1}}],["9546387f",{"2":{"256":1}}],["9547882f",{"2":{"256":1}}],["9547",{"2":{"249":1}}],["95401394",{"2":{"232":1}}],["95",{"2":{"228":6}}],["950003255691461e",{"2":{"258":1}}],["9500",{"2":{"249":1}}],["9500444",{"2":{"147":1}}],["9501",{"2":{"244":1}}],["9501405",{"2":{"148":1}}],["957010263478429e",{"2":{"258":1}}],["957016f",{"2":{"256":1}}],["957585242805049e",{"2":{"258":1}}],["957288+0",{"2":{"168":1}}],["9572f",{"2":{"77":1}}],["9577397",{"2":{"147":1}}],["9789223658090315e",{"2":{"258":1}}],["9787989f",{"2":{"256":1}}],["970390451599896e",{"2":{"258":1}}],["970267f",{"2":{"256":1}}],["9708",{"2":{"249":1}}],["971806f",{"2":{"256":1}}],["971899",{"2":{"109":1}}],["974485531067046e",{"2":{"258":1}}],["9747",{"2":{"249":1}}],["9742292",{"2":{"232":1}}],["9734461f",{"2":{"256":1}}],["9734574",{"2":{"145":1}}],["97378695",{"2":{"232":1}}],["9767632",{"2":{"232":1}}],["9766699015845924",{"2":{"174":4}}],["97667",{"2":{"171":1}}],["97",{"2":{"228":8,"249":1}}],["97s",{"2":{"196":1}}],["975649",{"2":{"172":1}}],["9725193359652275e",{"2":{"258":1}}],["972270233451054e",{"2":{"258":1}}],["97265f",{"2":{"256":1}}],["972074f",{"2":{"256":1}}],["97285",{"2":{"168":1}}],["9721554734769875",{"2":{"153":2}}],["9727281",{"2":{"147":1}}],["9087357604430672e",{"2":{"258":1}}],["9010019277228988e",{"2":{"258":1}}],["9017338637430295e",{"2":{"258":1}}],["901343f",{"2":{"256":1}}],["9016104f",{"2":{"256":1}}],["9019142280758281",{"2":{"153":2}}],["904619f",{"2":{"256":1}}],["904574581106778e",{"2":{"258":1}}],["9045",{"2":{"249":1}}],["9043228",{"2":{"145":1}}],["905497602952288e",{"2":{"258":1}}],["9050243347228516e",{"2":{"258":1}}],["905054f",{"2":{"256":1}}],["90508",{"2":{"126":1,"129":1}}],["9055119",{"2":{"232":2}}],["90",{"2":{"228":3}}],["902982902377702",{"2":{"153":2}}],["90959525086064e",{"2":{"258":1}}],["909403f",{"2":{"256":1}}],["9094505",{"2":{"145":1}}],["90910274",{"2":{"148":1}}],["9071",{"2":{"249":1}}],["9076505",{"2":{"235":1}}],["9076533",{"2":{"145":1}}],["9079035",{"2":{"232":1}}],["90779305",{"2":{"232":1}}],["90772897",{"2":{"126":1}}],["9001",{"2":{"179":1,"244":1}}],["900",{"2":{"107":1}}],["968377116456888e",{"2":{"258":1}}],["9685473f",{"2":{"256":1}}],["968504",{"2":{"232":2}}],["966937688414516e",{"2":{"258":1}}],["964216674049922e",{"2":{"258":1}}],["964033f",{"2":{"256":1}}],["9640",{"2":{"249":1}}],["963454901807268e",{"2":{"258":1}}],["963430330896957e",{"2":{"258":1}}],["9634635f",{"2":{"256":1}}],["963778f",{"2":{"256":1}}],["9636583f",{"2":{"256":1}}],["961164227252867e",{"2":{"258":1}}],["961107f",{"2":{"256":1}}],["961078232888846e",{"2":{"258":1}}],["9613344f",{"2":{"256":1}}],["969496943298209e",{"2":{"258":1}}],["9694",{"2":{"249":1}}],["9693s\\ttraining",{"2":{"217":1}}],["9693878",{"2":{"148":1}}],["965290648994648e",{"2":{"258":1}}],["965601f",{"2":{"256":1}}],["965008",{"2":{"235":1}}],["965117",{"2":{"126":1}}],["967105f",{"2":{"256":1}}],["9679338",{"2":{"232":1}}],["967811",{"2":{"172":1}}],["960741041448202e",{"2":{"258":1}}],["960298f",{"2":{"256":1}}],["96062994",{"2":{"232":2}}],["9604645f",{"2":{"106":4}}],["96",{"2":{"228":10}}],["96296",{"2":{"219":1}}],["9626998",{"2":{"147":1}}],["9894279474126975e",{"2":{"258":1}}],["989958f",{"2":{"256":1}}],["9891374f",{"2":{"256":1}}],["983786f",{"2":{"256":1}}],["983979931949539e",{"2":{"258":1}}],["983922f",{"2":{"256":1}}],["98396176",{"2":{"110":1}}],["982632845594918e",{"2":{"258":1}}],["982801f",{"2":{"256":1}}],["982162476",{"2":{"244":2}}],["984058251908748e",{"2":{"258":1}}],["984889f",{"2":{"256":1}}],["9847003468029552e",{"2":{"258":1}}],["9847",{"2":{"249":1}}],["9849229",{"2":{"235":1}}],["9849193f",{"2":{"106":1}}],["9853",{"2":{"249":1}}],["9856883",{"2":{"232":1}}],["98",{"2":{"228":4}}],["981339",{"2":{"171":1}}],["9803549",{"2":{"232":1}}],["980",{"2":{"159":1}}],["9802322f",{"2":{"106":2}}],["98023f",{"2":{"106":4}}],["987689",{"2":{"145":1}}],["943910967971433e",{"2":{"258":1}}],["9436797",{"2":{"179":1}}],["947026898636771e",{"2":{"258":1}}],["9470847f",{"2":{"256":1}}],["9470896",{"2":{"129":1}}],["944006323101522e",{"2":{"258":1}}],["9441587f",{"2":{"256":1}}],["944118f",{"2":{"256":1}}],["9499822396721737e",{"2":{"258":1}}],["9499244f",{"2":{"256":1}}],["949662f",{"2":{"256":1}}],["948026876144895e",{"2":{"258":1}}],["948203299006832e",{"2":{"258":1}}],["9482746f",{"2":{"256":1}}],["9484713f",{"2":{"256":1}}],["9489941",{"2":{"232":1}}],["9460",{"2":{"249":1}}],["9414",{"2":{"249":1}}],["9413",{"2":{"249":1}}],["9418648",{"2":{"232":1}}],["94259596",{"2":{"235":1}}],["94",{"2":{"228":3}}],["9452923791531558",{"2":{"153":2}}],["94557f",{"2":{"77":1}}],["9402",{"2":{"249":1}}],["9405848223512736",{"2":{"174":4}}],["940585",{"2":{"171":1}}],["940697f",{"2":{"106":1}}],["9407f",{"2":{"106":1}}],["9g",{"2":{"78":1}}],["927437876076855e",{"2":{"258":1}}],["9274546f",{"2":{"256":1}}],["9273887f",{"2":{"256":1}}],["9209102f",{"2":{"256":1}}],["9206433",{"2":{"145":1}}],["926076f",{"2":{"256":1}}],["923272028122853e",{"2":{"258":1}}],["9230",{"2":{"249":1}}],["923982e",{"2":{"208":1}}],["923950547913545",{"2":{"153":2}}],["9248211",{"2":{"232":1}}],["925991620617663e",{"2":{"258":1}}],["92593",{"2":{"217":1,"219":1}}],["9253495",{"2":{"232":1}}],["9293546871173684e",{"2":{"258":1}}],["92932f",{"2":{"256":1}}],["929731422461133e",{"2":{"258":1}}],["9290824f",{"2":{"256":1}}],["92913383",{"2":{"232":2}}],["929f",{"2":{"77":1}}],["92",{"2":{"196":1,"228":2}}],["928373314213804e",{"2":{"258":1}}],["92832f",{"2":{"77":1}}],["928667f",{"2":{"256":1}}],["928609",{"2":{"145":1}}],["928497f",{"2":{"256":1}}],["9285",{"2":{"249":1}}],["9210803307071616e",{"2":{"258":1}}],["92155594",{"2":{"176":4}}],["9214196",{"2":{"148":1}}],["921",{"2":{"59":1}}],["9957840891336297e",{"2":{"258":1}}],["995489911453944e",{"2":{"258":1}}],["9956736f",{"2":{"256":1}}],["99085097987449e",{"2":{"258":1}}],["990891f",{"2":{"256":1}}],["99070690914685e",{"2":{"258":1}}],["9946505662337705e",{"2":{"258":1}}],["9945927f",{"2":{"256":1}}],["9961959f",{"2":{"256":1}}],["991099f",{"2":{"256":1}}],["991714306465219e",{"2":{"258":1}}],["9917674f",{"2":{"256":1}}],["991784e",{"2":{"208":1}}],["9913",{"2":{"249":1}}],["9916",{"2":{"249":1}}],["997566847050341e",{"2":{"258":1}}],["997596f",{"2":{"256":1}}],["9971247",{"2":{"235":1}}],["997003",{"2":{"77":6}}],["9934639",{"2":{"232":1}}],["9930804",{"2":{"232":1}}],["993525e",{"2":{"208":1}}],["992126",{"2":{"232":2}}],["992662",{"2":{"145":1}}],["99",{"2":{"148":1,"228":10}}],["9999881",{"2":{"110":2}}],["999986",{"2":{"109":1}}],["99998605",{"2":{"109":1}}],["999",{"2":{"77":7,"234":1,"236":1}}],["99900760833609",{"2":{"53":1}}],["9",{"2":{"53":4,"62":2,"77":15,"102":1,"106":25,"126":2,"129":2,"145":1,"147":1,"148":2,"152":1,"171":1,"179":1,"187":14,"192":1,"195":2,"196":2,"213":1,"217":52,"219":10,"224":2,"228":2,"234":1,"236":1,"249":3,"256":72,"258":95}}],["✖",{"2":{"49":4}}],["✔️",{"2":{"86":13}}],["✔",{"2":{"49":8}}],["`θ`",{"2":{"250":1}}],["`x`",{"2":{"250":1}}],["`circbuff",{"2":{"244":4}}],["`carry`",{"2":{"184":2}}],["`p",{"2":{"214":1}}],["`ps",{"2":{"141":1}}],["`iterators",{"2":{"184":1}}],["`eachslice`",{"2":{"184":1}}],["`∇`",{"2":{"176":1}}],["`autoforwarddiff",{"2":{"146":1}}],["`autozygote",{"2":{"146":1}}],["`a`",{"2":{"134":3}}],["`val",{"2":{"145":2}}],["`training`",{"2":{"145":2}}],["`zygote",{"2":{"145":1}}],["`denselayerparameters`",{"2":{"142":2}}],["`namedtuple`",{"2":{"142":2}}],["`model",{"2":{"141":1}}],["`st`",{"2":{"141":1,"256":2}}],["`st",{"2":{"141":1}}],["`lag",{"2":{"244":4}}],["`lstm",{"2":{"184":1}}],["`linear",{"2":{"141":2}}],["`l",{"2":{"140":1}}],["`luxcore",{"2":{"145":2}}],["`luxcore`",{"2":{"140":1}}],["`lux",{"2":{"145":2}}],["`lux``",{"2":{"172":1}}],["`lux`",{"2":{"140":1}}],["`b`",{"2":{"134":2}}],["`",{"2":{"48":1,"77":2,"140":1,"141":1,"145":2,"146":2,"176":1,"244":8}}],["`octavian",{"2":{"18":1}}],["×",{"2":{"28":6,"180":1,"183":2,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["×dn−2×1×1`",{"2":{"69":1}}],["×dn−2×1×1",{"2":{"19":1}}],["×dn−2×1×dn",{"2":{"19":1,"69":1}}],["≥",{"2":{"28":2,"54":1,"244":1}}],["znver2",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["zip",{"2":{"107":1,"244":1}}],["zips",{"2":{"32":1}}],["zyg",{"2":{"106":2}}],["zygotevjp",{"2":{"220":13}}],["zygote",{"2":{"27":1,"28":1,"49":1,"52":1,"53":1,"59":2,"77":3,"78":1,"83":1,"86":1,"87":3,"106":3,"110":2,"133":2,"134":2,"144":3,"145":1,"146":1,"147":3,"148":3,"152":3,"170":1,"173":1,"175":3,"176":4,"181":1,"182":1,"191":1,"212":1,"223":1,"231":1,"236":1,"239":1,"242":3}}],["zenodo",{"2":{"74":3}}],["zeroing",{"2":{"32":4,"55":1}}],["zero",{"2":{"24":3,"32":3,"53":2,"54":2,"55":9,"63":1,"64":3,"67":1,"107":1,"146":1,"171":1}}],["zeroed",{"2":{"24":3}}],["zerosc64",{"2":{"25":1}}],["zerosc32",{"2":{"25":1}}],["zerosc16",{"2":{"25":1}}],["zeros64",{"2":{"25":1}}],["zeros32",{"2":{"25":1,"140":2}}],["zeros16",{"2":{"25":1}}],["zeros",{"2":{"24":3,"25":6,"153":2,"171":1,"243":3,"248":1,"249":1}}],["z=σ",{"2":{"66":1}}],["z=",{"2":{"49":1}}],["z",{"2":{"49":2,"67":3,"249":2,"250":8}}],["≤",{"2":{"24":2,"49":2,"67":2}}],["6e",{"2":{"208":1}}],["6d",{"2":{"179":1}}],["661069202042451e",{"2":{"258":1}}],["6610835f",{"2":{"256":1}}],["660069785094722e",{"2":{"258":1}}],["6607282f",{"2":{"256":1}}],["66038f",{"2":{"256":1}}],["668282934418554e",{"2":{"258":1}}],["668987124475024e",{"2":{"258":1}}],["6680107f",{"2":{"256":1}}],["6681195f",{"2":{"256":1}}],["66852736",{"2":{"235":1}}],["6691803f",{"2":{"256":1}}],["6679519563928514e",{"2":{"258":1}}],["667572",{"2":{"232":1}}],["6670s\\ttraining",{"2":{"217":1}}],["66",{"2":{"196":2,"256":1}}],["663036",{"2":{"172":1}}],["6637242",{"2":{"147":1}}],["6642563709240284",{"2":{"153":2}}],["66667",{"2":{"217":15,"219":4}}],["666682",{"2":{"147":1}}],["666965",{"2":{"168":1}}],["6663244",{"2":{"147":1}}],["665241",{"2":{"53":10}}],["6804369970380884e",{"2":{"258":1}}],["680666731479961e",{"2":{"258":1}}],["680748",{"2":{"126":1}}],["6829341070353303e",{"2":{"258":1}}],["6829463f",{"2":{"256":1}}],["6826662f",{"2":{"256":1}}],["681690151930187e",{"2":{"258":1}}],["6812784f",{"2":{"256":1}}],["681728",{"2":{"172":1}}],["683403436631394e",{"2":{"258":1}}],["6831355f",{"2":{"256":1}}],["6830945313415872",{"2":{"174":1}}],["6840108552708576e",{"2":{"258":1}}],["684011f",{"2":{"256":1}}],["6846",{"2":{"249":1}}],["6897",{"2":{"249":1}}],["6861458770761945e",{"2":{"258":1}}],["6862037f",{"2":{"256":1}}],["6869",{"2":{"249":1}}],["6867142",{"2":{"148":1}}],["68",{"2":{"217":2,"219":2,"228":9,"249":1}}],["6852",{"2":{"249":1}}],["6859683",{"2":{"232":1}}],["6850394",{"2":{"232":2}}],["685079",{"2":{"171":1}}],["6853896",{"2":{"148":1}}],["6877",{"2":{"249":1}}],["687357e",{"2":{"208":1}}],["687611",{"2":{"168":1}}],["687264",{"2":{"148":1}}],["687216",{"2":{"131":2}}],["688665",{"2":{"148":1}}],["6886741",{"2":{"126":1}}],["695032732920575e",{"2":{"258":1}}],["699567041563707e",{"2":{"258":1}}],["699378f",{"2":{"256":1}}],["6969002665539284e",{"2":{"258":1}}],["6968424f",{"2":{"256":1}}],["698451242744888e",{"2":{"258":1}}],["698141f",{"2":{"256":1}}],["6983237",{"2":{"232":1}}],["693736542371006e",{"2":{"258":1}}],["693378199953781e",{"2":{"258":1}}],["693837f",{"2":{"256":1}}],["6932663f",{"2":{"256":1}}],["6970",{"2":{"249":1}}],["6945381035152222e",{"2":{"258":1}}],["69471f",{"2":{"256":1}}],["69472f",{"2":{"77":1}}],["6946222f",{"2":{"256":1}}],["6943",{"2":{"249":1}}],["6920981",{"2":{"232":1}}],["6904922",{"2":{"232":1}}],["6909446526419574",{"2":{"153":2}}],["6913705",{"2":{"145":1}}],["69",{"2":{"121":1,"196":1,"217":4}}],["616748794405194e",{"2":{"258":1}}],["6167292f",{"2":{"256":1}}],["616618896214645e",{"2":{"258":1}}],["6169224131586253e",{"2":{"258":1}}],["616521f",{"2":{"256":1}}],["616838f",{"2":{"256":1}}],["6123978535044658e",{"2":{"258":1}}],["6125816f",{"2":{"256":1}}],["6120",{"2":{"24":1}}],["615570908822224e",{"2":{"258":1}}],["615518f",{"2":{"256":1}}],["61512",{"2":{"187":1}}],["6189",{"2":{"249":1}}],["6130755405394208e",{"2":{"258":1}}],["6130465f",{"2":{"256":1}}],["6134",{"2":{"249":1}}],["6133",{"2":{"249":1}}],["6138451",{"2":{"106":1}}],["61384505",{"2":{"106":1}}],["611023436946012e",{"2":{"258":1}}],["6110508477099578e",{"2":{"258":1}}],["611271f",{"2":{"256":1}}],["611109f",{"2":{"256":1}}],["6117",{"2":{"249":1}}],["61155146",{"2":{"232":1}}],["611584190904272",{"2":{"153":2}}],["614593242072826e",{"2":{"258":1}}],["614582f",{"2":{"256":1}}],["614665806544275e",{"2":{"258":1}}],["614401f",{"2":{"256":1}}],["6141",{"2":{"249":1}}],["61417323",{"2":{"232":2}}],["6143286",{"2":{"126":1}}],["61",{"2":{"196":1,"228":2}}],["6105897f",{"2":{"256":1}}],["610",{"2":{"148":1}}],["651491862698763e",{"2":{"258":1}}],["65178",{"2":{"147":1}}],["658994502723074e",{"2":{"258":1}}],["658692361731209e",{"2":{"258":1}}],["658684f",{"2":{"256":1}}],["6586393f",{"2":{"256":1}}],["659298573344575e",{"2":{"258":1}}],["659408425913405e",{"2":{"258":1}}],["6599032319848886e",{"2":{"258":1}}],["659576f",{"2":{"256":1}}],["6561045501900908e",{"2":{"258":1}}],["6566549385943286e",{"2":{"258":1}}],["656613f",{"2":{"106":1}}],["6569027f",{"2":{"256":1}}],["654903f",{"2":{"256":1}}],["6548522f",{"2":{"256":1}}],["6541584",{"2":{"106":1}}],["6541586",{"2":{"106":1}}],["6571946",{"2":{"235":1}}],["657670184695808",{"2":{"153":2}}],["6535434",{"2":{"232":2}}],["65",{"2":{"228":3}}],["655884342282818e",{"2":{"258":1}}],["655881",{"2":{"171":1}}],["6558766f",{"2":{"256":1}}],["6556360448565952",{"2":{"174":1}}],["650458406943489e",{"2":{"258":1}}],["650439f",{"2":{"256":1}}],["650211f",{"2":{"256":1}}],["6501",{"2":{"244":1}}],["650",{"2":{"69":3}}],["6315567522647815e",{"2":{"258":1}}],["6312156f",{"2":{"256":1}}],["6335537694747692e",{"2":{"258":1}}],["6334108f",{"2":{"256":1}}],["633735f",{"2":{"256":1}}],["6395677464177725e",{"2":{"258":1}}],["639050508187e",{"2":{"258":1}}],["6390796f",{"2":{"256":1}}],["6392575f",{"2":{"256":1}}],["63272390271709e",{"2":{"258":1}}],["632059266812866e",{"2":{"258":1}}],["6320302f",{"2":{"256":1}}],["632401f",{"2":{"256":1}}],["6323408",{"2":{"126":1}}],["63644228849465e",{"2":{"258":1}}],["636214902053133e",{"2":{"258":1}}],["6362536f",{"2":{"256":1}}],["636019f",{"2":{"256":1}}],["636308",{"2":{"106":1}}],["6384361411197174e",{"2":{"258":1}}],["638915582927304e",{"2":{"258":1}}],["63891147562606e",{"2":{"258":1}}],["6389992",{"2":{"232":1}}],["6388315f",{"2":{"256":1}}],["638867f",{"2":{"256":1}}],["6386199f",{"2":{"256":1}}],["63838756",{"2":{"147":1}}],["63",{"2":{"228":1}}],["634147099579145e",{"2":{"258":1}}],["6349",{"2":{"249":1}}],["6340362477836592",{"2":{"174":1}}],["634549",{"2":{"168":1}}],["637937355688216e",{"2":{"258":1}}],["63795453",{"2":{"106":1}}],["6378533f",{"2":{"256":1}}],["637114",{"2":{"168":1}}],["6370531107315014",{"2":{"153":2}}],["6372185",{"2":{"148":1}}],["620084831552526e",{"2":{"258":1}}],["6202136",{"2":{"232":1}}],["6290555f",{"2":{"256":1}}],["62963",{"2":{"217":2}}],["628783298934402e",{"2":{"258":1}}],["628777733493452e",{"2":{"258":1}}],["628754f",{"2":{"256":1}}],["628854664852085e",{"2":{"258":1}}],["6285055f",{"2":{"256":1}}],["6214",{"2":{"249":1}}],["621958",{"2":{"171":1}}],["622644858782885e",{"2":{"258":1}}],["622542f",{"2":{"256":1}}],["6220472",{"2":{"232":2}}],["622865",{"2":{"172":1}}],["62",{"2":{"196":1,"228":6}}],["627",{"2":{"180":1}}],["62625877100596",{"2":{"171":1}}],["6240393531484048e",{"2":{"258":1}}],["6243237f",{"2":{"256":1}}],["6243073f",{"2":{"256":1}}],["624383955429775",{"2":{"174":1}}],["624384",{"2":{"171":1}}],["62450767",{"2":{"106":1}}],["625461704009955e",{"2":{"258":1}}],["625632f",{"2":{"256":1}}],["625",{"2":{"53":1}}],["605020984488702e",{"2":{"258":1}}],["605599f",{"2":{"256":1}}],["608690372961008e",{"2":{"258":1}}],["6083617f",{"2":{"256":1}}],["60876",{"2":{"187":1}}],["6072455727442996e",{"2":{"258":1}}],["607851f",{"2":{"256":1}}],["6071615f",{"2":{"256":1}}],["6076053f0",{"2":{"53":2}}],["606933562909097e",{"2":{"258":1}}],["6064957f",{"2":{"256":1}}],["6064619",{"2":{"106":1}}],["606522f",{"2":{"256":1}}],["604789078667043e",{"2":{"258":1}}],["604745f",{"2":{"256":1}}],["6049256",{"2":{"148":1}}],["609160060584738e",{"2":{"258":1}}],["609592460335126e",{"2":{"258":1}}],["609716f",{"2":{"256":1}}],["609",{"2":{"237":1}}],["60",{"2":{"196":2,"249":1}}],["60242",{"2":{"187":1}}],["601441im",{"2":{"168":1}}],["600964373387555e",{"2":{"258":1}}],["600911f",{"2":{"256":1}}],["600401354057221e",{"2":{"258":1}}],["6004457f",{"2":{"256":1}}],["600076f",{"2":{"256":1}}],["6003383",{"2":{"232":1}}],["600134f",{"2":{"256":1}}],["6001",{"2":{"179":1,"244":1}}],["600",{"2":{"107":1}}],["674142362096196e",{"2":{"258":1}}],["6744",{"2":{"249":1}}],["671196955107326e",{"2":{"258":1}}],["6719434458280483e",{"2":{"258":1}}],["671255f",{"2":{"256":1}}],["6712",{"2":{"249":1}}],["673625338984129e",{"2":{"258":1}}],["6738976f",{"2":{"256":1}}],["6739464f",{"2":{"256":1}}],["672318877693364e",{"2":{"258":1}}],["672547f",{"2":{"256":1}}],["6726513",{"2":{"145":1}}],["676786f",{"2":{"256":1}}],["6750511624909356e",{"2":{"258":1}}],["6752",{"2":{"249":1}}],["67595f",{"2":{"77":1}}],["67",{"2":{"217":1,"228":3}}],["677033842129356e",{"2":{"258":1}}],["6770",{"2":{"249":1}}],["6771653",{"2":{"232":2}}],["677913e",{"2":{"208":1}}],["6776220912718907",{"2":{"174":1}}],["67dffc4a8ae",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["678356728850944e",{"2":{"258":1}}],["678993f",{"2":{"256":1}}],["6781636f",{"2":{"256":1}}],["678869500413314e",{"2":{"258":1}}],["6788",{"2":{"249":1}}],["67861",{"2":{"171":1}}],["6784163",{"2":{"77":1}}],["679548484324747e",{"2":{"258":1}}],["6796s\\ttraining",{"2":{"217":1}}],["67909",{"2":{"171":1}}],["67908657",{"2":{"171":1}}],["679087",{"2":{"171":2}}],["6798847",{"2":{"147":1}}],["67094",{"2":{"168":1}}],["6702257",{"2":{"145":1}}],["6",{"2":{"24":1,"39":2,"62":3,"77":7,"106":1,"145":1,"147":5,"148":7,"152":1,"171":2,"173":3,"179":1,"180":5,"187":14,"189":5,"193":8,"196":2,"197":1,"210":5,"217":5,"219":1,"221":5,"228":2,"229":5,"232":6,"237":5,"244":4,"246":5,"249":11,"250":6,"251":1,"255":2,"256":173,"258":168,"260":1}}],["647740218253537e",{"2":{"258":1}}],["6477456",{"2":{"145":1}}],["649340816148732e",{"2":{"258":1}}],["649534f",{"2":{"256":1}}],["648081f",{"2":{"256":1}}],["641572520445694e",{"2":{"258":1}}],["641593f",{"2":{"256":1}}],["6418690381312165e",{"2":{"258":1}}],["641792f",{"2":{"256":1}}],["6417983f",{"2":{"256":1}}],["645165243838028e",{"2":{"258":1}}],["645258701715071e",{"2":{"258":1}}],["6452096f",{"2":{"256":1}}],["6450346f",{"2":{"256":1}}],["6456693",{"2":{"232":2}}],["6456416023530093",{"2":{"174":1}}],["6431484",{"2":{"235":1}}],["64495987",{"2":{"232":1}}],["6448233",{"2":{"232":1}}],["6423511984038721",{"2":{"153":2}}],["6424197",{"2":{"147":1}}],["646131141713119e",{"2":{"258":1}}],["646117f",{"2":{"256":1}}],["64644",{"2":{"232":1}}],["6469914",{"2":{"126":1}}],["64605f",{"2":{"77":1}}],["640589172541938e",{"2":{"258":1}}],["64058f",{"2":{"77":1}}],["6409345",{"2":{"147":1}}],["6409691",{"2":{"106":1}}],["64",{"2":{"18":1,"69":18,"78":1,"180":2,"189":2,"197":2,"210":2,"221":2,"226":2,"229":2,"237":2,"246":2,"251":2,"256":1,"260":2}}],["β",{"2":{"19":4,"69":1,"206":2}}],["γ=0",{"2":{"53":2}}],["γ",{"2":{"19":4,"53":1,"69":1,"206":2}}],["762953174736238e",{"2":{"258":1}}],["7697396587690263e",{"2":{"258":1}}],["7638582f",{"2":{"256":1}}],["765898143658767e",{"2":{"258":1}}],["765726547941438e",{"2":{"258":1}}],["765956f",{"2":{"256":1}}],["7656558f",{"2":{"256":1}}],["768760131342562e",{"2":{"258":1}}],["768746f",{"2":{"256":1}}],["7689272356710357",{"2":{"153":2}}],["76158035",{"2":{"232":1}}],["764905532483414e",{"2":{"258":1}}],["7649314",{"2":{"148":1}}],["7647132f",{"2":{"256":1}}],["76425457",{"2":{"232":1}}],["7663455",{"2":{"232":1}}],["76",{"2":{"196":1,"208":2,"217":6,"219":2,"228":1}}],["767544233524609e",{"2":{"258":1}}],["7671651f",{"2":{"256":1}}],["7672965f",{"2":{"256":1}}],["7672513",{"2":{"106":1}}],["7673453",{"2":{"126":1}}],["730504753415649e",{"2":{"258":1}}],["7309833914647513e",{"2":{"258":1}}],["7309303f",{"2":{"256":1}}],["730941",{"2":{"148":1}}],["739714635673128e",{"2":{"258":1}}],["739644f",{"2":{"256":1}}],["732456804452487e",{"2":{"258":1}}],["7352",{"2":{"249":1}}],["7356s\\ttraining",{"2":{"217":1}}],["7370",{"2":{"249":1}}],["73341",{"2":{"232":1}}],["734828700995126e",{"2":{"258":1}}],["734941f",{"2":{"256":1}}],["7346s\\ttraining",{"2":{"217":1}}],["7347s\\ttraining",{"2":{"217":1}}],["73",{"2":{"196":2,"217":1,"219":1,"228":1}}],["7384",{"2":{"249":1}}],["7384486",{"2":{"232":1}}],["73806",{"2":{"172":1}}],["738356",{"2":{"171":1}}],["731059",{"2":{"53":2}}],["7574398877464584e",{"2":{"258":1}}],["757169253238071e",{"2":{"258":1}}],["757172f",{"2":{"256":1}}],["7520733823738473e",{"2":{"258":1}}],["758535186007862e",{"2":{"258":1}}],["758783f",{"2":{"256":1}}],["7580993408473766",{"2":{"17":1}}],["75910119547398e",{"2":{"258":1}}],["7591155f",{"2":{"256":1}}],["759783f",{"2":{"256":1}}],["759412",{"2":{"148":1}}],["7519702f",{"2":{"256":1}}],["751377",{"2":{"147":1}}],["7533",{"2":{"249":1}}],["7560",{"2":{"249":1}}],["7560s\\ttraining",{"2":{"217":1}}],["7559453",{"2":{"232":1}}],["75",{"2":{"196":1,"217":2,"219":1,"228":55,"249":1,"255":1,"256":2,"259":4}}],["7502",{"2":{"251":1,"260":1}}],["7504",{"2":{"249":1}}],["7501",{"2":{"244":1}}],["750",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["7541456",{"2":{"232":1}}],["754747",{"2":{"148":1}}],["754463",{"2":{"148":1}}],["77785613539327e",{"2":{"258":1}}],["777959f",{"2":{"256":1}}],["77778",{"2":{"217":2}}],["7789",{"2":{"249":1}}],["77165353",{"2":{"232":2}}],["779806041141176e",{"2":{"258":1}}],["77989f",{"2":{"256":1}}],["7795275",{"2":{"232":2}}],["7794657",{"2":{"126":1}}],["77",{"2":{"217":1,"228":1}}],["7746656838366666",{"2":{"174":1}}],["7733535276924052",{"2":{"174":4}}],["773354",{"2":{"171":1}}],["7766466f",{"2":{"256":1}}],["7766",{"2":{"171":1}}],["776598",{"2":{"171":2}}],["77207",{"2":{"148":1}}],["770921364263631e",{"2":{"258":1}}],["770791f",{"2":{"256":1}}],["7703",{"2":{"249":1}}],["77037",{"2":{"147":1}}],["7701346738750905",{"2":{"153":2}}],["7700396",{"2":{"147":1}}],["710255596708555e",{"2":{"258":1}}],["7150199164418895e",{"2":{"258":1}}],["715078f",{"2":{"256":1}}],["715533199795584e",{"2":{"258":1}}],["715484f",{"2":{"256":1}}],["7178926211450773e",{"2":{"258":1}}],["717863343114228",{"2":{"153":2}}],["7179507f",{"2":{"256":1}}],["7190214f",{"2":{"256":1}}],["7199",{"2":{"249":1}}],["7195462169922173",{"2":{"153":1}}],["7195462169922175",{"2":{"153":1}}],["7149",{"2":{"249":1}}],["7126900699024794e",{"2":{"258":1}}],["7121",{"2":{"249":1}}],["7127043f",{"2":{"256":1}}],["7127",{"2":{"249":1}}],["7123551",{"2":{"232":1}}],["712979078066394",{"2":{"153":2}}],["7183194",{"2":{"232":1}}],["7181164",{"2":{"145":1}}],["71",{"2":{"217":2,"228":15}}],["711329",{"2":{"168":1}}],["716015707276895e",{"2":{"258":1}}],["716057767416446e",{"2":{"258":1}}],["716244f",{"2":{"256":1}}],["716201f",{"2":{"256":1}}],["7165354",{"2":{"232":2}}],["7165512278088038",{"2":{"153":2}}],["716134755899883",{"2":{"153":2}}],["713",{"2":{"236":1}}],["713567516238075",{"2":{"153":2}}],["713316",{"2":{"106":1}}],["745046378068568e",{"2":{"258":1}}],["745216f",{"2":{"256":1}}],["7479956155117537e",{"2":{"258":1}}],["749117317459649e",{"2":{"258":1}}],["749130826457307e",{"2":{"258":1}}],["7491754f",{"2":{"256":1}}],["7498905599156158e",{"2":{"258":1}}],["749019f",{"2":{"256":1}}],["7499613f",{"2":{"256":1}}],["74450853961715e",{"2":{"258":1}}],["744973425565604e",{"2":{"258":1}}],["7448626f",{"2":{"256":1}}],["7448054f",{"2":{"256":1}}],["744489",{"2":{"145":1}}],["7482393f",{"2":{"256":1}}],["748238f",{"2":{"256":1}}],["748096575064684e",{"2":{"258":1}}],["74804461795482e",{"2":{"258":1}}],["7480448f",{"2":{"256":1}}],["7480315",{"2":{"232":2}}],["7426",{"2":{"249":1}}],["7429947",{"2":{"129":1}}],["741384652316938e",{"2":{"258":1}}],["741348",{"2":{"232":1}}],["7416314319765904e",{"2":{"258":1}}],["7417964f",{"2":{"256":1}}],["741774f",{"2":{"256":1}}],["7411s\\ttraining",{"2":{"217":1}}],["740898191300713e",{"2":{"258":1}}],["7407676f",{"2":{"256":1}}],["74074",{"2":{"217":3}}],["7401575",{"2":{"232":2}}],["7402",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["74",{"2":{"196":1,"217":5,"228":2}}],["74s",{"2":{"196":1}}],["7468008914093891",{"2":{"174":4}}],["746801",{"2":{"171":1}}],["74694291453392",{"2":{"174":4}}],["746943",{"2":{"171":1}}],["704584962489625e",{"2":{"258":1}}],["7047462f",{"2":{"256":1}}],["704629f",{"2":{"256":1}}],["705018466291802e",{"2":{"258":1}}],["7056854037051348e",{"2":{"258":1}}],["7059134f",{"2":{"256":1}}],["7054554",{"2":{"147":1}}],["7014154634250893e",{"2":{"258":1}}],["7011092f",{"2":{"256":1}}],["7010881f",{"2":{"256":1}}],["7018592",{"2":{"145":1}}],["702716131950309e",{"2":{"258":1}}],["7027s\\ttraining",{"2":{"217":1}}],["7026062338780957e",{"2":{"258":1}}],["702859f",{"2":{"256":1}}],["7024828f",{"2":{"256":1}}],["7024",{"2":{"249":1}}],["70866144",{"2":{"232":2}}],["7089496198123055",{"2":{"153":2}}],["70",{"2":{"196":1,"217":1,"228":3}}],["707534",{"2":{"171":1}}],["703595145715941e",{"2":{"258":1}}],["703495f",{"2":{"256":1}}],["70370",{"2":{"217":1}}],["70390546",{"2":{"126":1}}],["70330536",{"2":{"77":1}}],["7008614742707937e",{"2":{"258":1}}],["700910267623787e",{"2":{"258":1}}],["7009206441595775e",{"2":{"258":1}}],["70074f",{"2":{"256":1}}],["7001468",{"2":{"232":1}}],["7001",{"2":{"179":1,"244":1}}],["700",{"2":{"107":1}}],["722484775317188e",{"2":{"258":1}}],["7221s\\ttraining",{"2":{"219":1}}],["720868200189139e",{"2":{"258":1}}],["7209446508843805e",{"2":{"258":1}}],["720094f",{"2":{"256":1}}],["7200",{"2":{"249":1}}],["721116f",{"2":{"256":1}}],["7213",{"2":{"249":1}}],["7286350831903895e",{"2":{"258":1}}],["7289675f",{"2":{"256":1}}],["7288033f",{"2":{"256":1}}],["7284s\\ttraining",{"2":{"217":1}}],["723527466294975e",{"2":{"258":1}}],["723424f",{"2":{"256":1}}],["7237308297251812",{"2":{"174":1}}],["72373",{"2":{"171":1}}],["723731",{"2":{"171":2}}],["7273",{"2":{"249":1}}],["72",{"2":{"217":3}}],["726782771156918e",{"2":{"258":1}}],["726797f",{"2":{"256":1}}],["726323611742261e",{"2":{"258":1}}],["7262204f",{"2":{"256":1}}],["7262075",{"2":{"147":1}}],["7266s\\ttraining",{"2":{"217":1}}],["724099",{"2":{"168":1}}],["72471225",{"2":{"147":1}}],["72428167",{"2":{"126":1}}],["7252903f",{"2":{"106":8}}],["72948844519828e",{"2":{"258":1}}],["729025622513638e",{"2":{"258":1}}],["729014f",{"2":{"256":1}}],["72908926",{"2":{"148":1}}],["7292133f",{"2":{"256":1}}],["729295f",{"2":{"256":1}}],["7292344",{"2":{"145":1}}],["7296474",{"2":{"147":1}}],["729",{"2":{"77":6}}],["795352858422536e",{"2":{"258":1}}],["7958943f",{"2":{"256":1}}],["7907312049270067e",{"2":{"258":1}}],["7906605f",{"2":{"256":1}}],["798505f",{"2":{"256":1}}],["798027",{"2":{"147":1}}],["7971921131432287e",{"2":{"258":1}}],["79725f",{"2":{"256":1}}],["7977",{"2":{"249":1}}],["7990667",{"2":{"235":1}}],["7941928404855354e",{"2":{"258":1}}],["7942",{"2":{"249":1}}],["7946s\\ttraining",{"2":{"219":1}}],["794908",{"2":{"147":1}}],["79",{"2":{"196":4,"217":4,"228":3,"249":1}}],["791378034823515e",{"2":{"258":1}}],["791942f",{"2":{"256":1}}],["791806",{"2":{"168":1}}],["7911112",{"2":{"147":1}}],["79250443",{"2":{"147":1}}],["7937254051470556e",{"2":{"258":1}}],["7937835f",{"2":{"256":1}}],["7934256088207734e",{"2":{"258":1}}],["793463f",{"2":{"77":1}}],["7932576f",{"2":{"256":1}}],["793159",{"2":{"171":1}}],["7939677f",{"2":{"106":1}}],["79656f",{"2":{"77":1}}],["782044560431486e",{"2":{"258":1}}],["78226817",{"2":{"140":1}}],["787921113496737e",{"2":{"258":1}}],["78758f",{"2":{"256":1}}],["787159",{"2":{"145":1}}],["7863486216872975e",{"2":{"258":1}}],["78637f",{"2":{"256":1}}],["786470195418644e",{"2":{"258":1}}],["786186627506336e",{"2":{"258":1}}],["7862057f",{"2":{"256":1}}],["786613f",{"2":{"256":1}}],["788579306043833e",{"2":{"258":1}}],["788526f",{"2":{"256":1}}],["78818554",{"2":{"232":1}}],["78814f",{"2":{"106":1}}],["7834",{"2":{"249":1}}],["7834251",{"2":{"147":1}}],["7891",{"2":{"249":1}}],["7897024",{"2":{"232":1}}],["78",{"2":{"196":2,"217":4,"219":1,"228":11}}],["781756775257433e",{"2":{"258":1}}],["781786f",{"2":{"256":1}}],["7817315740767116",{"2":{"174":1}}],["7811481",{"2":{"145":1}}],["7800006399111622e",{"2":{"258":1}}],["78000563",{"2":{"141":1}}],["7805",{"2":{"249":1}}],["7808904",{"2":{"74":2}}],["7847404739992356e",{"2":{"258":1}}],["784768",{"2":{"147":1,"148":1}}],["784521f",{"2":{"256":1}}],["784816e",{"2":{"208":1}}],["784",{"2":{"69":6,"215":1,"220":6,"226":1}}],["7",{"2":{"19":9,"53":5,"62":4,"77":17,"106":10,"145":5,"147":2,"148":5,"152":1,"173":3,"179":1,"180":4,"187":14,"189":4,"196":2,"197":1,"210":4,"217":5,"219":1,"221":4,"228":2,"229":4,"232":5,"237":4,"246":4,"249":6,"251":1,"256":91,"258":88,"260":1}}],["5g",{"2":{"236":1}}],["5gb",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["5fs",{"2":{"228":1}}],["5f",{"2":{"187":3,"217":2}}],["5f0",{"2":{"126":1,"129":1,"152":1,"186":1,"248":12}}],["5443363442410884e",{"2":{"258":1}}],["54452",{"2":{"172":1}}],["549098278507849e",{"2":{"258":1}}],["54856367828419e",{"2":{"258":1}}],["5487312f",{"2":{"256":1}}],["5434154f",{"2":{"256":1}}],["5417264207046406e",{"2":{"258":1}}],["541728e",{"2":{"208":1}}],["541993160248347e",{"2":{"258":1}}],["5414985f",{"2":{"256":1}}],["542077f",{"2":{"256":1}}],["5424696",{"2":{"232":1}}],["540484179145182e",{"2":{"258":1}}],["540807f",{"2":{"256":1}}],["540",{"2":{"220":8}}],["54",{"2":{"217":1,"228":1}}],["5476424498276177",{"2":{"174":4}}],["547642",{"2":{"171":1}}],["5×3",{"2":{"171":8,"172":1}}],["5×5",{"2":{"24":1,"171":3}}],["588916760643641e",{"2":{"258":1}}],["588445e",{"2":{"208":1}}],["584856775156783e",{"2":{"258":1}}],["5840",{"2":{"249":1}}],["589330129167569e",{"2":{"258":1}}],["5895496f",{"2":{"256":1}}],["58904f",{"2":{"256":1}}],["58947",{"2":{"187":1}}],["5817545f",{"2":{"256":1}}],["581316f",{"2":{"256":1}}],["581266581936409e",{"2":{"258":1}}],["5812",{"2":{"249":1}}],["5838733200622807e",{"2":{"258":1}}],["5838605f",{"2":{"256":1}}],["583667350718321e",{"2":{"258":1}}],["5830077f",{"2":{"256":1}}],["5839225f",{"2":{"256":1}}],["5834",{"2":{"249":1}}],["585187361669241e",{"2":{"258":1}}],["5854513084383856e",{"2":{"258":1}}],["5850444f",{"2":{"256":1}}],["5853932f",{"2":{"256":1}}],["5853876f",{"2":{"256":1}}],["5852",{"2":{"249":1}}],["5821663295026476e",{"2":{"258":1}}],["5821",{"2":{"249":1}}],["58263564",{"2":{"235":1}}],["5826772",{"2":{"232":2}}],["5806",{"2":{"249":1}}],["5809",{"2":{"249":1}}],["58",{"2":{"196":1,"217":3,"219":1,"228":1}}],["5865265807660498",{"2":{"153":2}}],["58790814860857e",{"2":{"258":1}}],["587933f",{"2":{"256":1}}],["5879354f",{"2":{"106":1}}],["58776483214611e",{"2":{"258":1}}],["58740882046328e",{"2":{"258":1}}],["587859f",{"2":{"256":1}}],["5878153",{"2":{"147":1}}],["58721",{"2":{"171":1}}],["58720636",{"2":{"171":1}}],["58720636f0",{"2":{"171":1}}],["587206",{"2":{"171":2}}],["528550362491412e",{"2":{"258":1}}],["5288183f",{"2":{"256":1}}],["5281",{"2":{"74":2}}],["5218488912733589e",{"2":{"258":1}}],["5215222551617784e",{"2":{"258":1}}],["5217254f",{"2":{"256":1}}],["5213385f",{"2":{"256":1}}],["5210742834996898",{"2":{"153":2}}],["5254225536087655e",{"2":{"258":1}}],["5256801776556165e",{"2":{"258":1}}],["5257921f",{"2":{"256":1}}],["52538",{"2":{"187":1}}],["52628386",{"2":{"232":1}}],["5267549745189349",{"2":{"153":2}}],["52024",{"2":{"232":1}}],["5276172f",{"2":{"256":1}}],["527559",{"2":{"232":2}}],["5271s\\ttraining",{"2":{"217":1}}],["52717",{"2":{"147":1}}],["5221656",{"2":{"179":1}}],["522139",{"2":{"148":1}}],["524250172689278e",{"2":{"258":1}}],["5247253265878265e",{"2":{"258":1}}],["52478315f",{"2":{"256":1}}],["524781",{"2":{"148":1}}],["524128548331127e",{"2":{"258":1}}],["5241285f",{"2":{"256":1}}],["52442f",{"2":{"256":1}}],["524008",{"2":{"171":1}}],["5734997895927483e",{"2":{"258":1}}],["57381f",{"2":{"256":1}}],["573845",{"2":{"168":1}}],["572376782694359e",{"2":{"258":1}}],["572151999524389e",{"2":{"258":1}}],["5720214f",{"2":{"256":1}}],["575341791688568e",{"2":{"258":1}}],["575293f",{"2":{"256":1}}],["5715612f",{"2":{"256":1}}],["5718",{"2":{"254":2}}],["5705431229722175e",{"2":{"258":1}}],["5705988f",{"2":{"256":1}}],["5704312556200466e",{"2":{"258":1}}],["570732f",{"2":{"256":1}}],["576556649567345e",{"2":{"258":1}}],["5766952320339846e",{"2":{"258":1}}],["5766662f",{"2":{"256":1}}],["5767006920644584e",{"2":{"258":1}}],["5767007f",{"2":{"256":1}}],["5767846f",{"2":{"256":1}}],["57671577",{"2":{"142":1}}],["579991856199895e",{"2":{"258":1}}],["579943f",{"2":{"256":1}}],["5794",{"2":{"249":1}}],["578826564146093e",{"2":{"258":1}}],["57882756",{"2":{"232":1}}],["5787146f",{"2":{"256":1}}],["5787",{"2":{"249":1}}],["578374863",{"2":{"244":2}}],["57",{"2":{"217":5,"228":2}}],["57s",{"2":{"196":1}}],["577181",{"2":{"171":1}}],["5779067839062536",{"2":{"153":2}}],["5776367e",{"2":{"148":1}}],["5776052f0",{"2":{"53":1}}],["564508322429582e",{"2":{"258":1}}],["5645374f",{"2":{"256":1}}],["5652669683084067e",{"2":{"258":1}}],["5652423280655655e",{"2":{"258":1}}],["565232283015596e",{"2":{"258":1}}],["5652089f",{"2":{"256":1}}],["5653004f",{"2":{"256":1}}],["5622622981749016e",{"2":{"258":1}}],["562793776851796e",{"2":{"258":1}}],["5627227",{"2":{"145":1}}],["5624903f",{"2":{"256":1}}],["56267f",{"2":{"256":1}}],["5696972f",{"2":{"256":1}}],["5691s\\ttraining",{"2":{"217":1}}],["568244597437593e",{"2":{"258":1}}],["56821f",{"2":{"256":1}}],["568433f",{"2":{"256":1}}],["5680085614623662",{"2":{"153":2}}],["5613943344910157e",{"2":{"258":1}}],["5612637f",{"2":{"256":1}}],["561137f",{"2":{"106":1}}],["5663",{"2":{"249":1}}],["56685",{"2":{"187":1}}],["563711e",{"2":{"208":1}}],["56",{"2":{"196":1,"217":1}}],["567631657302037e",{"2":{"258":1}}],["5673",{"2":{"249":1}}],["5675557670686644",{"2":{"174":1}}],["56779",{"2":{"171":1}}],["567794",{"2":{"171":2}}],["560",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["56055",{"2":{"171":1}}],["56069607",{"2":{"106":1}}],["596973792252105e",{"2":{"258":1}}],["596633f",{"2":{"256":1}}],["594664969491395e",{"2":{"258":1}}],["5941574545216334e",{"2":{"258":1}}],["5941284f",{"2":{"256":1}}],["5942532f",{"2":{"256":1}}],["594261",{"2":{"148":1}}],["5983",{"2":{"249":1}}],["598505080",{"2":{"244":2}}],["5956",{"2":{"249":1}}],["5957",{"2":{"249":1}}],["59543324",{"2":{"235":1}}],["5905511",{"2":{"232":2}}],["59259",{"2":{"219":1}}],["59",{"2":{"148":1,"228":3}}],["5979974",{"2":{"145":1}}],["5975106",{"2":{"77":1}}],["59980381156852e",{"2":{"258":1}}],["599940757515103e",{"2":{"258":1}}],["5999714",{"2":{"126":1}}],["5991",{"2":{"249":1}}],["5994072",{"2":{"106":1}}],["5994074",{"2":{"106":1}}],["5370234029661434e",{"2":{"258":1}}],["5378918",{"2":{"77":1}}],["53331836429825e",{"2":{"258":1}}],["533486f",{"2":{"256":1}}],["5392246768680794e",{"2":{"258":1}}],["5391956f",{"2":{"256":1}}],["5350297f",{"2":{"256":1}}],["5356s\\ttraining",{"2":{"217":1}}],["538971818941064e",{"2":{"258":1}}],["5387439f",{"2":{"256":1}}],["5386",{"2":{"249":1}}],["5381101016248151",{"2":{"153":2}}],["5348017174812176e",{"2":{"258":1}}],["53467",{"2":{"187":1}}],["5345716",{"2":{"77":1}}],["5310851",{"2":{"179":1}}],["53144f",{"2":{"77":1}}],["536628116455649e",{"2":{"258":1}}],["536287f",{"2":{"256":1}}],["536804f",{"2":{"256":1}}],["5368917f",{"2":{"256":1}}],["536791302036357e",{"2":{"258":1}}],["536743e",{"2":{"152":1}}],["5367613",{"2":{"147":1}}],["5365157",{"2":{"148":1}}],["53",{"2":{"110":1,"228":1,"249":1}}],["5d",{"2":{"70":1,"208":1,"244":2,"257":1}}],["5140502142324223e",{"2":{"258":1}}],["5141534f",{"2":{"256":1}}],["519529771869915e",{"2":{"258":1}}],["51968503",{"2":{"232":2}}],["5115101f",{"2":{"256":1}}],["51165134",{"2":{"235":1}}],["5171857969606e",{"2":{"258":1}}],["517210275390521e",{"2":{"258":1}}],["517043f",{"2":{"256":1}}],["5174026f",{"2":{"256":1}}],["5178",{"2":{"249":1}}],["51785",{"2":{"187":1}}],["518613097626927e",{"2":{"258":1}}],["518599f",{"2":{"256":1}}],["51852",{"2":{"217":1}}],["518999f",{"2":{"256":1}}],["5107s\\ttraining",{"2":{"217":1}}],["5167208",{"2":{"235":1}}],["5168s\\ttraining",{"2":{"217":1}}],["5164032",{"2":{"179":1}}],["515824991407245e",{"2":{"258":1}}],["5158im",{"2":{"168":1}}],["5156055f",{"2":{"256":1}}],["51524514",{"2":{"179":1}}],["51363",{"2":{"168":1}}],["51",{"2":{"69":3,"109":1,"208":2,"228":2,"236":1}}],["512",{"2":{"59":3,"243":1}}],["507069855223133e",{"2":{"258":1}}],["50798f",{"2":{"77":1}}],["509010482523759e",{"2":{"258":1}}],["5097923f",{"2":{"256":1}}],["505270944949432e",{"2":{"258":1}}],["505146553864519e",{"2":{"258":1}}],["505499f",{"2":{"256":1}}],["5053",{"2":{"249":1}}],["504735f",{"2":{"256":1}}],["50640313796474e",{"2":{"258":1}}],["506485",{"2":{"172":1}}],["506546f",{"2":{"256":1}}],["508194899831204e",{"2":{"258":1}}],["508443f",{"2":{"256":1}}],["5089574f",{"2":{"256":1}}],["508674e",{"2":{"208":1}}],["5012522901353627e",{"2":{"258":1}}],["5010563f",{"2":{"256":1}}],["50151f",{"2":{"256":1}}],["5017",{"2":{"249":1}}],["5017s\\ttraining",{"2":{"219":1}}],["501",{"2":{"244":1}}],["5016017",{"2":{"147":1}}],["5021s\\ttraining",{"2":{"217":1}}],["502312852219817",{"2":{"53":1}}],["5030s\\ttraining",{"2":{"217":1}}],["50398",{"2":{"187":1}}],["503186",{"2":{"172":1}}],["5031421",{"2":{"106":1}}],["5000379381177776e",{"2":{"258":1}}],["5000×30×1",{"2":{"249":1}}],["50000",{"2":{"244":100}}],["5000",{"2":{"244":1,"249":5}}],["5009556",{"2":{"235":1}}],["5001",{"2":{"179":1,"244":1}}],["500",{"2":{"107":1,"183":2,"244":1}}],["50",{"2":{"63":2,"69":3,"196":12,"217":1,"228":105,"236":1,"249":1}}],["5∗δ",{"2":{"53":1}}],["5∗|y−y^|2if",{"2":{"53":1}}],["5507062187873954e",{"2":{"258":1}}],["550528f",{"2":{"256":1}}],["5501",{"2":{"244":1}}],["5517104f",{"2":{"256":1}}],["5511811",{"2":{"232":2}}],["558777712008384e",{"2":{"258":1}}],["5585098f",{"2":{"256":1}}],["55803",{"2":{"187":1}}],["55624196620906e",{"2":{"258":1}}],["5567568473656066e",{"2":{"258":1}}],["5567038f",{"2":{"256":1}}],["5564",{"2":{"249":1}}],["55658394",{"2":{"232":1}}],["55694",{"2":{"187":1}}],["55583f",{"2":{"256":1}}],["55556",{"2":{"217":2}}],["5553797706980106",{"2":{"174":1}}],["55s",{"2":{"196":1}}],["5594571",{"2":{"232":1}}],["5590551",{"2":{"232":2}}],["5595843665394066",{"2":{"174":1}}],["55989707",{"2":{"148":1}}],["5532",{"2":{"171":1}}],["553631",{"2":{"147":1}}],["5530689",{"2":{"147":1}}],["55476f",{"2":{"77":1}}],["55",{"2":{"53":1,"196":1,"219":1}}],["5where",{"2":{"53":1}}],["5",{"2":{"19":9,"24":8,"32":2,"39":4,"53":15,"59":11,"63":2,"69":6,"74":1,"77":16,"86":1,"102":1,"106":5,"121":5,"129":1,"130":1,"147":1,"148":2,"168":6,"171":29,"172":1,"173":3,"177":3,"179":6,"180":3,"187":14,"189":3,"193":14,"196":2,"205":1,"206":1,"210":3,"217":5,"218":2,"219":1,"220":40,"221":3,"228":2,"229":3,"232":8,"236":2,"237":3,"239":1,"246":3,"248":4,"249":19,"250":1,"254":4,"255":2,"256":793,"258":755,"259":3}}],["^2",{"2":{"255":2,"256":1}}],["^",{"2":{"19":9,"59":1,"78":1,"106":1,"107":1,"254":2}}],["λβ",{"2":{"17":1}}],["α",{"2":{"17":3,"53":3,"206":2,"250":3}}],["+=",{"2":{"194":2,"216":2,"227":2,"244":1,"249":1}}],["+ϵ∗γ+βwhere",{"2":{"69":1}}],["+ϵ∗γ+βand",{"2":{"19":1}}],["+",{"2":{"16":1,"18":1,"24":3,"32":1,"37":1,"49":2,"52":1,"55":1,"59":4,"62":5,"63":4,"65":6,"67":4,"70":1,"93":1,"140":2,"145":1,"146":1,"158":2,"171":4,"179":3,"183":2,"206":1,"232":1,"242":4,"243":3,"245":1,"248":4,"249":1,"254":13,"255":1,"256":2}}],["ys",{"2":{"245":6}}],["y∈",{"2":{"243":1}}],["ylabel=",{"2":{"232":1,"236":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["y=x2−2x",{"2":{"232":1}}],["y=x−e",{"2":{"19":1,"69":1}}],["yᵢ",{"2":{"107":2}}],["yes",{"2":{"82":1}}],["year",{"2":{"74":2}}],["yet",{"2":{"37":1,"106":1}}],["y3",{"2":{"62":1}}],["y2",{"2":{"62":1,"248":2}}],["y1",{"2":{"62":1,"248":2}}],["yann",{"2":{"53":1}}],["y+ϵ",{"2":{"53":1}}],["yi∈rm",{"2":{"179":1}}],["yi",{"2":{"53":2,"179":1}}],["y~=",{"2":{"53":2}}],["y~",{"2":{"53":4}}],["y^2+y∗max",{"2":{"53":1}}],["y^−y∗log⁡",{"2":{"53":1}}],["y^−y",{"2":{"53":1}}],["y^",{"2":{"53":7}}],["y^+ϵ",{"2":{"53":3}}],["ŷ",{"2":{"53":11,"145":2,"146":2,"147":2,"186":2,"187":3}}],["yoshua",{"2":{"24":2}}],["yourself",{"2":{"203":1}}],["your",{"0":{"113":1},"2":{"37":2,"47":1,"59":2,"69":3,"90":1,"108":2,"110":1,"117":1,"137":2,"138":1,"140":2,"141":2,"142":3,"144":1,"145":2,"157":1,"161":1,"163":2,"188":1}}],["you",{"2":{"2":1,"6":2,"33":2,"37":1,"47":1,"48":1,"52":3,"59":5,"71":3,"74":2,"76":3,"77":1,"78":1,"79":1,"89":1,"102":1,"106":2,"107":1,"108":3,"109":2,"110":2,"111":1,"113":2,"117":1,"127":1,"133":2,"137":2,"138":1,"140":8,"141":6,"142":5,"144":5,"145":4,"146":1,"153":1,"156":1,"157":2,"163":2,"168":1,"171":1,"172":3,"173":1,"175":1,"181":1,"185":1,"188":1,"202":1,"203":1,"217":2,"225":1,"233":1,"243":1,"256":1}}],["yuxin",{"2":{"19":1}}],["y",{"2":{"14":2,"15":2,"19":1,"24":3,"28":5,"49":3,"53":63,"54":3,"55":6,"58":1,"59":10,"62":8,"63":2,"65":9,"67":10,"68":6,"69":4,"77":1,"78":4,"106":11,"107":2,"110":2,"123":4,"140":3,"141":4,"145":7,"146":7,"153":2,"160":5,"179":11,"183":4,"184":5,"185":3,"186":12,"187":5,"192":6,"194":2,"195":4,"206":4,"213":6,"216":2,"217":2,"224":4,"225":2,"227":2,"228":2,"232":5,"236":5,"243":7,"245":1,"248":1,"254":6,"256":3}}],["σ=identity",{"2":{"19":1}}],["σ",{"2":{"13":10,"15":5,"16":3,"18":3,"19":7,"158":3}}],["know",{"2":{"88":1,"109":1,"144":1,"256":1}}],["known",{"0":{"117":1},"2":{"24":2,"52":1,"88":1}}],["knet",{"2":{"81":1}}],["k",{"2":{"63":6,"67":4,"179":1,"214":2,"218":2}}],["kwarg",{"2":{"102":2,"103":1}}],["kwargs",{"2":{"25":24,"33":1,"42":2,"49":4,"59":3,"62":9,"68":2,"103":1,"168":2,"169":2,"195":1,"214":7,"217":2,"218":4,"220":4}}],["kw",{"2":{"59":2}}],["kullback",{"2":{"53":1}}],["kl",{"2":{"53":1}}],["kldivergenceloss",{"2":{"53":5}}],["klambauer",{"2":{"17":1}}],["ki−1",{"2":{"63":1,"65":3}}],["kind",{"0":{"126":1},"2":{"34":1,"39":1,"59":2,"62":1,"109":1,"126":1}}],["kinds",{"2":{"8":1}}],["kiros",{"2":{"19":1}}],["kaiming",{"2":{"19":1,"24":4,"63":2,"67":1,"168":6}}],["keith",{"2":{"252":1}}],["keep",{"2":{"136":1}}],["kept",{"2":{"133":1}}],["kernelabstractions",{"2":{"20":1,"153":7}}],["kernels",{"0":{"153":1},"2":{"16":1,"153":3}}],["kernel",{"2":{"16":1,"20":1,"24":1,"63":5,"153":6}}],["keys",{"2":{"102":1,"121":1,"143":2}}],["keypath=keypath",{"2":{"33":1}}],["keypath",{"2":{"32":7,"102":1,"103":1,"109":4,"110":10,"127":2,"128":4}}],["key",{"0":{"116":1},"2":{"10":2,"112":1}}],["keywords",{"2":{"59":1,"70":1,"169":1}}],["keyword",{"2":{"2":1,"10":1,"33":2,"37":1,"48":1,"49":2,"62":5,"63":2,"64":4,"65":3,"66":2,"67":4,"68":2,"69":4,"70":2,"92":1,"104":1,"105":1,"169":1}}],["hₓ",{"2":{"254":4}}],["h₊",{"2":{"254":4}}],["h12",{"2":{"254":12}}],["h11",{"2":{"254":12}}],["h22",{"2":{"254":12}}],["hmc",{"2":{"249":2}}],["hcat",{"2":{"243":1,"249":1}}],["hn",{"2":{"225":2}}],["hnew",{"2":{"66":6}}],["hnew=activation",{"2":{"66":1}}],["hnew=",{"2":{"66":1}}],["hypernet",{"0":{"225":1,"226":1},"2":{"225":4,"226":1}}],["hypernetwork",{"0":{"222":1},"1":{"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1}}],["hh",{"2":{"66":6,"105":1}}],["h",{"2":{"65":6,"66":3,"70":4,"192":1,"213":1,"254":14}}],["home",{"2":{"142":1}}],["hosts",{"2":{"79":1}}],["hold",{"2":{"77":1,"82":1}}],["hot",{"2":{"55":1}}],["how",{"0":{"71":1,"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"39":1,"53":2,"55":1,"62":2,"66":3,"69":8,"76":1,"77":1,"83":1,"106":1,"107":1,"108":1,"110":2,"125":1,"134":1,"140":1,"142":1,"144":1,"148":1,"153":1,"157":1,"161":2,"171":2,"177":1,"190":1,"204":1,"214":1,"225":1,"247":1,"249":1,"250":1}}],["however",{"2":{"8":1,"11":1,"55":1,"59":1,"62":1,"77":1,"81":1,"89":1,"104":1,"111":1,"116":1,"148":1,"167":1,"173":2,"175":1,"202":1,"204":3,"214":1,"238":1,"254":1,"256":2}}],["hutchinson",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":2,"150":3,"151":2,"152":8}}],["huber",{"2":{"53":1}}],["huberloss",{"2":{"53":3}}],["human",{"2":{"24":2}}],["https",{"2":{"24":1,"66":1,"68":1,"71":1,"74":1,"177":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"254":2,"260":1}}],["heatmap",{"2":{"245":1}}],["head",{"2":{"83":1,"184":1}}],["heavily",{"2":{"58":1}}],["heavy",{"2":{"6":1}}],["height",{"2":{"63":1}}],["hence",{"2":{"37":1,"47":1,"134":1,"136":1,"150":1,"174":1,"175":1,"196":1,"208":1,"214":1,"256":1}}],["helps",{"2":{"214":1}}],["help",{"2":{"30":1,"31":1,"32":1,"33":2,"53":2,"57":1,"59":1,"62":6,"63":2,"65":3,"66":2,"68":1,"69":4,"70":1,"108":1,"157":1}}],["helpers",{"0":{"26":1},"1":{"27":1,"28":1,"29":1},"2":{"54":1}}],["helper",{"0":{"20":1,"43":1,"68":1,"194":1},"2":{"52":1,"57":1,"250":1}}],["here",{"2":{"30":1,"33":1,"52":2,"54":1,"59":3,"77":1,"81":1,"88":1,"113":1,"120":1,"141":1,"145":3,"146":1,"147":1,"160":1,"161":1,"171":3,"177":2,"208":1,"214":1,"241":1,"248":1,"249":2}}],["hessian",{"2":{"24":1}}],["he",{"2":{"19":1,"24":2}}],["historical",{"2":{"214":1}}],["historically",{"2":{"30":1}}],["hidden",{"2":{"66":43,"184":6,"185":3,"241":11,"244":2,"249":1}}],["hierarchically",{"2":{"59":1}}],["hinge",{"2":{"53":2}}],["hingeloss",{"2":{"53":2}}],["hinton",{"2":{"19":1}}],["highlight",{"2":{"116":1}}],["highly",{"2":{"53":1}}],["higher",{"2":{"70":1,"147":1,"155":1}}],["highest",{"2":{"16":1,"88":1,"155":1,"250":1}}],["high",{"2":{"62":1,"153":1,"171":1,"177":1}}],["hi",{"2":{"24":2}}],["hit",{"2":{"8":1}}],["hamiltonian",{"2":{"249":2}}],["hat",{"2":{"209":2}}],["had",{"2":{"175":1}}],["hadsell",{"2":{"53":1}}],["hand",{"2":{"171":1}}],["handles",{"2":{"83":1,"185":1}}],["handle",{"2":{"66":1,"130":1,"153":1,"214":1,"249":1}}],["handling",{"2":{"54":1,"57":1,"138":1,"155":1,"165":1}}],["happened",{"2":{"155":1}}],["happening",{"2":{"146":1,"160":1}}],["happens",{"2":{"109":1,"144":1,"160":1,"165":1}}],["haven",{"2":{"37":1}}],["have",{"2":{"24":1,"28":1,"30":2,"63":1,"66":1,"68":1,"83":2,"84":1,"88":3,"89":1,"95":1,"99":2,"102":3,"103":1,"104":2,"105":1,"109":1,"110":2,"111":2,"117":1,"133":2,"137":1,"140":1,"141":1,"142":1,"144":2,"145":1,"146":1,"152":1,"155":1,"172":1,"173":1,"174":1,"175":3,"184":3,"208":1,"226":1,"249":1,"250":2,"254":1}}],["having",{"2":{"6":1,"34":1,"59":1}}],["harder",{"2":{"83":1,"108":1}}],["hardware",{"2":{"82":1}}],["hard",{"2":{"11":1,"53":1,"110":1,"140":1,"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["has",{"2":{"8":3,"31":1,"37":2,"59":1,"63":1,"65":3,"70":1,"81":1,"83":1,"92":2,"95":6,"99":1,"102":7,"103":4,"104":1,"105":1,"108":1,"109":4,"112":1,"115":1,"130":1,"141":1,"148":1,"170":1,"171":2,"175":1,"249":1,"252":1,"256":1}}],["2f",{"2":{"195":2,"228":4}}],["2fs",{"2":{"195":1}}],["2`",{"2":{"141":1}}],["278660068048847e",{"2":{"258":1}}],["2780457f",{"2":{"256":1}}],["2785366f",{"2":{"256":1}}],["272485404699174e",{"2":{"258":1}}],["2727911835596774e",{"2":{"258":1}}],["2722575f",{"2":{"256":1}}],["2728916f",{"2":{"256":1}}],["27287",{"2":{"187":1}}],["274246929811238e",{"2":{"258":1}}],["2743545065074921e",{"2":{"258":1}}],["2743402f",{"2":{"256":1}}],["2746266f",{"2":{"256":1}}],["27471745",{"2":{"235":1}}],["2711861021368445e",{"2":{"258":1}}],["2710024f",{"2":{"256":1}}],["2714",{"2":{"249":1}}],["2717",{"2":{"249":1}}],["27596473545436e",{"2":{"258":1}}],["275076620182626e",{"2":{"258":1}}],["27501",{"2":{"244":1}}],["275776f",{"2":{"256":1}}],["2751344f",{"2":{"256":1}}],["2755905",{"2":{"232":2}}],["27545732",{"2":{"106":1}}],["27",{"2":{"228":3,"249":1}}],["276",{"2":{"208":1}}],["273258880055931e",{"2":{"258":1}}],["273924f",{"2":{"256":1}}],["27335f",{"2":{"256":1}}],["273091f",{"2":{"256":1}}],["2731825069079087e",{"2":{"258":1}}],["27318",{"2":{"187":1}}],["27373654",{"2":{"126":1}}],["270600915259254e",{"2":{"258":1}}],["270333f",{"2":{"256":1}}],["27035674",{"2":{"179":1}}],["2709",{"2":{"249":1}}],["27001",{"2":{"244":1}}],["2701178",{"2":{"232":1}}],["27051234",{"2":{"179":1}}],["2795349908595105e",{"2":{"258":1}}],["27993",{"2":{"172":1}}],["279012",{"2":{"168":1}}],["2797048270773437",{"2":{"153":2}}],["2791573282471997",{"2":{"153":2}}],["2773043322517866e",{"2":{"258":1}}],["27737",{"2":{"187":1}}],["2771924f",{"2":{"256":1}}],["2771789",{"2":{"179":1}}],["2771862617010155",{"2":{"153":2}}],["2774315",{"2":{"145":1}}],["27th",{"2":{"24":1}}],["2×4",{"2":{"173":1}}],["2×5",{"2":{"168":6}}],["2×32",{"2":{"106":2}}],["2×2",{"2":{"53":2,"171":1}}],["291384196131248e",{"2":{"258":1}}],["2913357",{"2":{"106":1}}],["2915765f",{"2":{"256":1}}],["295964210175835e",{"2":{"258":1}}],["2957967f",{"2":{"256":1}}],["2950",{"2":{"249":1}}],["29501",{"2":{"244":1}}],["29561827",{"2":{"179":1}}],["29315922",{"2":{"235":1}}],["2932029",{"2":{"235":1}}],["293211",{"2":{"106":1}}],["2938747",{"2":{"235":1}}],["2922773459631004e",{"2":{"258":1}}],["2922556",{"2":{"232":1}}],["29251926039576e",{"2":{"258":1}}],["2925251f",{"2":{"256":1}}],["2929784734312002e",{"2":{"258":1}}],["2923765f",{"2":{"256":1}}],["2928104f",{"2":{"256":1}}],["29206026",{"2":{"235":1}}],["292002",{"2":{"168":1}}],["29",{"2":{"217":1,"228":2,"249":1}}],["2903607f",{"2":{"256":1}}],["29001",{"2":{"244":1}}],["29056",{"2":{"187":1}}],["2904677",{"2":{"129":1}}],["2991",{"2":{"249":1}}],["2992126",{"2":{"232":2}}],["2990853",{"2":{"179":1}}],["29944375",{"2":{"179":1}}],["29955",{"2":{"168":1}}],["296805415528246e",{"2":{"258":1}}],["2968054f",{"2":{"256":1}}],["29630",{"2":{"217":2,"219":1}}],["296372",{"2":{"168":1}}],["296496",{"2":{"171":1}}],["2986593934707995e",{"2":{"258":1}}],["29892860151009e",{"2":{"258":1}}],["298081f",{"2":{"256":1}}],["2988795f",{"2":{"256":1}}],["29828635",{"2":{"179":1}}],["298409",{"2":{"168":1}}],["298787",{"2":{"148":1}}],["297959481822617",{"2":{"153":2}}],["2695",{"2":{"249":1}}],["2647",{"2":{"249":1}}],["26477236",{"2":{"106":2}}],["26486295702061e",{"2":{"258":1}}],["2648138f",{"2":{"256":1}}],["2648",{"2":{"249":1}}],["2607443612697635e",{"2":{"258":1}}],["2607066f",{"2":{"256":1}}],["2602118058605339e",{"2":{"258":1}}],["2602372f",{"2":{"256":1}}],["2601100669914204e",{"2":{"258":1}}],["260992f",{"2":{"256":1}}],["2604041f",{"2":{"256":1}}],["2600393f",{"2":{"256":1}}],["26001",{"2":{"244":1}}],["260598",{"2":{"235":1}}],["2606924",{"2":{"148":1}}],["26",{"2":{"196":1,"208":2,"217":1,"228":2,"249":1}}],["268739495137848e",{"2":{"258":1}}],["26874",{"2":{"187":1}}],["2686864f",{"2":{"256":1}}],["2681656",{"2":{"232":1}}],["2689521",{"2":{"106":2}}],["268941",{"2":{"53":2}}],["2610295515187747e",{"2":{"258":1}}],["2615936",{"2":{"147":1}}],["261273",{"2":{"131":2}}],["2672",{"2":{"249":1}}],["26793814",{"2":{"235":1}}],["26771653",{"2":{"232":2}}],["267644",{"2":{"147":1,"148":1}}],["26700416",{"2":{"106":1}}],["26700422",{"2":{"106":1}}],["26311737",{"2":{"106":1}}],["26657984",{"2":{"106":1}}],["26501",{"2":{"244":1}}],["265788",{"2":{"168":1}}],["265372",{"2":{"168":1}}],["265",{"2":{"78":1}}],["221811348996438e",{"2":{"258":1}}],["22104",{"2":{"187":1}}],["22996305276747e",{"2":{"258":1}}],["229147941019859e",{"2":{"258":1}}],["2293367f",{"2":{"256":1}}],["2290224145974982",{"2":{"153":2}}],["2271",{"2":{"249":1}}],["227513",{"2":{"168":1}}],["2209135920094944e",{"2":{"258":1}}],["2208558f",{"2":{"256":1}}],["22001",{"2":{"244":1}}],["220",{"2":{"220":6}}],["2261",{"2":{"249":1}}],["226",{"2":{"208":1}}],["22635892",{"2":{"179":1}}],["226381",{"2":{"77":1}}],["228334790081066e",{"2":{"258":1}}],["228447f",{"2":{"256":1}}],["22846f",{"2":{"77":1}}],["2287",{"2":{"249":1}}],["22807482",{"2":{"232":1}}],["22898",{"2":{"187":1}}],["2252003989273822e",{"2":{"258":1}}],["22501",{"2":{"244":1}}],["225",{"2":{"184":1,"207":2,"214":1,"220":18,"241":1}}],["22543387",{"2":{"77":1}}],["2220308f",{"2":{"256":1}}],["22222",{"2":{"217":2,"219":1}}],["2227837233928576",{"2":{"153":2}}],["222528",{"2":{"148":1}}],["2221465",{"2":{"148":1}}],["2228831",{"2":{"106":1}}],["22",{"2":{"145":1,"187":14,"196":3,"217":1,"228":2,"254":14}}],["223777130231299e",{"2":{"258":1}}],["2237958",{"2":{"77":1}}],["2239306697871398e",{"2":{"258":1}}],["223889f",{"2":{"256":1}}],["22363997",{"2":{"106":1}}],["2236399",{"2":{"106":1}}],["2243",{"2":{"249":1}}],["22406094",{"2":{"106":1}}],["22459084",{"2":{"77":1}}],["254199192663128e",{"2":{"258":1}}],["2545935",{"2":{"77":1}}],["259896126273798e",{"2":{"258":1}}],["25926",{"2":{"217":3,"219":1}}],["2559276f",{"2":{"256":1}}],["2557388628770116e",{"2":{"258":1}}],["2557",{"2":{"249":1}}],["2555",{"2":{"249":1}}],["25501",{"2":{"244":1}}],["25587f",{"2":{"77":1}}],["2504314216214893e",{"2":{"258":1}}],["250048676956555e",{"2":{"258":1}}],["25001",{"2":{"244":1}}],["2505346f",{"2":{"256":1}}],["25055695",{"2":{"177":1,"178":1}}],["2501",{"2":{"244":1}}],["250",{"2":{"236":2,"250":1,"255":1}}],["2581186",{"2":{"232":1}}],["2586653",{"2":{"106":1}}],["25160843350352e",{"2":{"258":1}}],["251267f",{"2":{"256":1}}],["251",{"2":{"208":1}}],["25",{"2":{"171":1,"187":15,"208":2,"217":1,"228":11,"249":1}}],["2528160954460315e",{"2":{"258":1}}],["25278285",{"2":{"179":1}}],["2525357728685884",{"2":{"153":2}}],["252985",{"2":{"148":1}}],["252925",{"2":{"148":1}}],["2523673",{"2":{"148":1}}],["253106300035819e",{"2":{"258":1}}],["2539",{"2":{"249":1}}],["25378025",{"2":{"106":2}}],["25384",{"2":{"187":1}}],["25389904",{"2":{"106":1}}],["25385493",{"2":{"106":1}}],["257776021481562e",{"2":{"258":1}}],["25775",{"2":{"187":1}}],["257429f",{"2":{"256":1}}],["2579597f",{"2":{"256":1}}],["257",{"2":{"77":1}}],["25683780594787e",{"2":{"258":1}}],["256922f",{"2":{"256":1}}],["25662464",{"2":{"126":1}}],["256",{"2":{"39":1,"59":1,"77":6,"193":3,"226":2}}],["2427448783123144e",{"2":{"258":1}}],["242915f",{"2":{"256":1}}],["243526291244579e",{"2":{"258":1}}],["2435263f",{"2":{"256":1}}],["2498555f",{"2":{"256":1}}],["24913",{"2":{"187":1}}],["244984534627751e",{"2":{"258":1}}],["2449845f",{"2":{"256":1}}],["2440945",{"2":{"232":2}}],["244728",{"2":{"53":10}}],["247446e",{"2":{"208":1}}],["2453570706699625e",{"2":{"258":1}}],["2451684f",{"2":{"256":1}}],["24501",{"2":{"244":1}}],["24549",{"2":{"187":1}}],["24520",{"2":{"187":1}}],["24844",{"2":{"187":1}}],["24",{"2":{"121":2,"180":3,"187":14,"189":3,"197":2,"210":3,"221":3,"228":2,"229":3,"237":3,"246":3,"249":2}}],["241937",{"2":{"232":1}}],["2415948",{"2":{"148":1}}],["24150778",{"2":{"107":1}}],["24169",{"2":{"106":1}}],["2464001",{"2":{"106":1}}],["24001",{"2":{"244":1}}],["24099025",{"2":{"77":1}}],["240",{"2":{"69":3}}],["238807910196516e",{"2":{"258":1}}],["2381458977603318e",{"2":{"258":1}}],["23824",{"2":{"187":1}}],["2346542742130683e",{"2":{"258":1}}],["234977861049552e",{"2":{"258":1}}],["23412165f",{"2":{"256":1}}],["2348104f",{"2":{"256":1}}],["2372690728922117e",{"2":{"258":1}}],["2379622f",{"2":{"256":1}}],["2373182f",{"2":{"256":1}}],["23710957",{"2":{"142":1}}],["23501",{"2":{"244":1}}],["2354317",{"2":{"232":1}}],["2351742f",{"2":{"106":3}}],["2338493976435376e",{"2":{"258":1}}],["233503937777888e",{"2":{"258":1}}],["233557f",{"2":{"256":1}}],["23303",{"2":{"187":1}}],["23325463",{"2":{"126":1}}],["23234403",{"2":{"235":1}}],["23238",{"2":{"187":1}}],["23242",{"2":{"147":1}}],["2395651228799042e",{"2":{"258":1}}],["2390274f",{"2":{"256":1}}],["2390s\\ttraining",{"2":{"217":1}}],["2393728f",{"2":{"256":1}}],["23989",{"2":{"187":1}}],["23992883",{"2":{"179":1}}],["239756",{"2":{"172":1}}],["23921265",{"2":{"147":1}}],["230126833709747e",{"2":{"258":1}}],["230468f",{"2":{"256":1}}],["23001",{"2":{"244":1}}],["23095",{"2":{"171":1}}],["230954",{"2":{"171":2}}],["23021114",{"2":{"106":1}}],["231723",{"2":{"168":1}}],["23162955",{"2":{"106":1}}],["2366",{"2":{"249":1}}],["23622048",{"2":{"232":2}}],["23626f",{"2":{"77":1}}],["2361472",{"2":{"126":1}}],["23",{"2":{"62":2,"180":1,"187":14,"189":1,"197":1,"210":1,"217":1,"221":1,"228":2,"229":1,"237":1,"246":1,"249":2,"251":1,"260":1}}],["209229224170496e",{"2":{"258":1}}],["209021229092797e",{"2":{"258":1}}],["2090098f",{"2":{"256":1}}],["2093648167863977e",{"2":{"258":1}}],["209826193506313e",{"2":{"258":1}}],["209781764071136e",{"2":{"258":1}}],["2097818f",{"2":{"256":1}}],["2091914f",{"2":{"256":1}}],["2094652f",{"2":{"256":1}}],["207739367079272e",{"2":{"258":1}}],["20778",{"2":{"187":1}}],["207923f",{"2":{"256":1}}],["203989697923951e",{"2":{"258":1}}],["203806f",{"2":{"256":1}}],["203036",{"2":{"106":1}}],["208222530434039e",{"2":{"258":1}}],["208326f",{"2":{"256":1}}],["20881107",{"2":{"126":1}}],["205675840138618e",{"2":{"258":1}}],["205676f",{"2":{"256":1}}],["205091731042327e",{"2":{"258":1}}],["2050",{"2":{"249":1}}],["20501",{"2":{"244":1}}],["2058456",{"2":{"235":1}}],["20579764",{"2":{"235":1}}],["20548",{"2":{"168":1}}],["20472442",{"2":{"232":2}}],["2042089",{"2":{"106":1}}],["202256116754194e",{"2":{"258":1}}],["2020674f",{"2":{"256":1}}],["2021",{"2":{"252":1}}],["20299",{"2":{"187":1}}],["2024",{"2":{"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["20277858",{"2":{"179":1}}],["2023",{"2":{"74":2}}],["2067",{"2":{"249":1}}],["206476",{"2":{"168":1}}],["20668754",{"2":{"147":1}}],["200522833174072e",{"2":{"258":1}}],["2005086f",{"2":{"256":1}}],["20053944",{"2":{"106":1}}],["20001",{"2":{"244":1}}],["20001543",{"2":{"179":1}}],["2000",{"2":{"192":1}}],["2001",{"2":{"179":1,"244":1}}],["200",{"2":{"107":1,"220":18}}],["2006",{"2":{"53":2}}],["20",{"2":{"62":1,"69":3,"77":1,"122":1,"147":1,"171":2,"179":3,"187":14,"215":5,"217":1,"220":24,"228":2,"249":9}}],["201",{"2":{"208":1,"220":18,"236":1}}],["20114",{"2":{"171":1}}],["201145",{"2":{"171":2}}],["2010",{"2":{"24":3}}],["2016",{"2":{"19":2,"53":2,"69":1}}],["2018618f",{"2":{"256":1}}],["20188306",{"2":{"106":1}}],["2018",{"2":{"19":1}}],["2015",{"2":{"19":1,"24":2}}],["2014",{"2":{"17":1,"24":1,"168":1}}],["2017",{"2":{"17":1,"53":2}}],["2121187f",{"2":{"256":1}}],["2125984",{"2":{"232":2}}],["21252f",{"2":{"77":1}}],["2140146852432286e",{"2":{"258":1}}],["214827603331037e",{"2":{"258":1}}],["214813f",{"2":{"256":1}}],["21470",{"2":{"187":1}}],["2182312430513194e",{"2":{"258":1}}],["218061f",{"2":{"256":1}}],["2180483f",{"2":{"256":1}}],["21856f",{"2":{"77":1}}],["2172112434066336e",{"2":{"258":1}}],["217936380518698e",{"2":{"258":1}}],["2173145f",{"2":{"256":1}}],["21737",{"2":{"187":1}}],["2170",{"2":{"249":1}}],["21501",{"2":{"244":1}}],["211330116623792e",{"2":{"258":1}}],["2119952301216232e",{"2":{"258":1}}],["21164f",{"2":{"256":1}}],["211",{"2":{"220":6}}],["2111273",{"2":{"148":1}}],["2108931596208436e",{"2":{"258":1}}],["2109375f",{"2":{"256":1}}],["2100185f",{"2":{"256":1}}],["21001",{"2":{"244":1}}],["210",{"2":{"220":18}}],["213847f",{"2":{"256":1}}],["2130",{"2":{"249":1}}],["2136s\\ttraining",{"2":{"217":1}}],["21332",{"2":{"187":1}}],["21s",{"2":{"196":1}}],["216586965387029e",{"2":{"258":1}}],["2166448f",{"2":{"256":1}}],["21690917",{"2":{"179":1}}],["21672015",{"2":{"126":1}}],["219640436510303e",{"2":{"258":1}}],["2199084f",{"2":{"256":1}}],["2191",{"2":{"249":1}}],["21940619",{"2":{"232":1}}],["2197981041108443",{"2":{"171":1}}],["2192001",{"2":{"148":1}}],["21950458",{"2":{"106":1}}],["21",{"2":{"48":1,"69":1,"187":14,"217":1,"228":2,"249":4}}],["2833",{"2":{"249":1}}],["283579809077081e",{"2":{"258":1}}],["2835",{"2":{"249":1}}],["28355134",{"2":{"106":1}}],["2878225f",{"2":{"256":1}}],["287945f",{"2":{"256":1}}],["287719254267265e",{"2":{"258":1}}],["2877",{"2":{"249":1}}],["28725442",{"2":{"106":1}}],["2852",{"2":{"249":1}}],["28501",{"2":{"244":1}}],["28560",{"2":{"187":1}}],["280597112235319e",{"2":{"258":1}}],["28001",{"2":{"244":1}}],["28042415",{"2":{"106":1}}],["2899156",{"2":{"235":1}}],["28946856",{"2":{"235":1}}],["2896426",{"2":{"232":1}}],["2896142",{"2":{"147":1}}],["281252",{"2":{"232":1}}],["28119",{"2":{"187":1}}],["281053",{"2":{"168":1}}],["2822224980909573e",{"2":{"258":1}}],["2822374f",{"2":{"256":1}}],["282055f",{"2":{"256":1}}],["28291",{"2":{"187":1}}],["28289196",{"2":{"77":1}}],["28825172841227e",{"2":{"258":1}}],["288929910830549e",{"2":{"258":1}}],["28894",{"2":{"187":1}}],["28844719267135e",{"2":{"258":1}}],["2884916f",{"2":{"256":1}}],["2880594f",{"2":{"256":1}}],["2888855f",{"2":{"256":1}}],["28813831506549e",{"2":{"258":1}}],["28813",{"2":{"187":1}}],["2869913410456831",{"2":{"153":2}}],["2864522",{"2":{"148":1}}],["28682f",{"2":{"77":1}}],["284701142105474e",{"2":{"258":1}}],["284601f",{"2":{"256":1}}],["2849667",{"2":{"148":1}}],["2842",{"2":{"147":1}}],["28",{"2":{"39":4,"180":1,"189":1,"193":2,"195":2,"196":1,"197":1,"210":1,"217":1,"220":2,"221":1,"224":4,"228":2,"229":1,"237":1,"246":1,"249":1,"251":1,"260":1}}],["2=dense",{"2":{"32":1}}],["2nd",{"0":{"29":1},"2":{"86":1,"121":1,"240":2,"242":1}}],["2d",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"24":2,"63":3,"64":2,"65":3,"195":2,"238":1}}],["2",{"2":{"5":5,"24":5,"28":2,"31":5,"32":4,"34":8,"37":4,"39":4,"48":1,"49":1,"53":22,"59":13,"62":30,"63":9,"65":15,"66":11,"67":2,"68":15,"69":6,"70":3,"77":49,"78":5,"86":7,"106":32,"107":4,"109":21,"110":41,"121":1,"122":1,"123":2,"126":3,"127":2,"128":2,"129":4,"131":2,"132":1,"133":6,"134":6,"140":3,"141":12,"142":2,"145":7,"146":4,"147":13,"148":13,"152":1,"153":23,"155":5,"168":8,"171":20,"172":1,"173":4,"174":3,"176":1,"177":1,"180":4,"183":5,"184":1,"185":1,"187":15,"188":1,"189":4,"192":1,"193":18,"196":2,"197":2,"206":6,"208":14,"209":5,"210":4,"213":1,"214":1,"215":1,"217":5,"219":1,"220":67,"221":4,"226":1,"228":4,"229":5,"232":14,"233":1,"235":6,"236":3,"237":4,"242":4,"243":10,"245":4,"246":4,"248":2,"249":34,"254":50,"255":8,"256":97,"258":94}}],["v=v",{"2":{"242":1}}],["vtav",{"2":{"149":1}}],["vvt",{"2":{"149":1}}],["v∈rd",{"2":{"149":1}}],["vs",{"2":{"138":1,"208":1}}],["v0",{"2":{"74":1,"102":1,"130":1,"218":1}}],["voila",{"2":{"109":1}}],["vocabulary",{"2":{"67":2}}],["volterra",{"2":{"206":2}}],["vol",{"2":{"53":1}}],["volumetric",{"2":{"53":1}}],["vcat",{"2":{"66":1,"183":1,"254":1}}],["v",{"2":{"27":2,"53":1,"54":3,"69":1,"95":1,"150":5,"151":5,"152":10,"176":4,"177":2,"178":1,"241":4,"242":10,"254":21}}],["vjp",{"0":{"27":1},"2":{"27":1,"150":7,"152":16,"175":3,"178":4,"236":3}}],["visualization",{"0":{"250":1}}],["visualizing",{"0":{"245":1,"259":1}}],["visualize",{"2":{"232":1,"250":1,"259":1}}],["vision",{"2":{"19":1,"24":2,"53":4}}],["virtual",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["vi",{"2":{"149":1}}],["vitj",{"2":{"149":1}}],["viewaxis",{"2":{"220":68}}],["views",{"2":{"214":2,"242":1,"254":1}}],["view",{"2":{"68":4,"173":1,"249":1,"250":1}}],["victor",{"2":{"19":1,"69":1}}],["via",{"2":{"3":1,"5":2,"6":1,"7":1,"18":1,"24":1,"40":1,"52":2,"57":2,"79":2,"83":2,"102":1,"103":1,"105":1,"106":1,"113":1,"116":1,"153":1,"174":1,"176":4,"177":1,"249":1}}],["vanilla",{"2":{"106":1}}],["various",{"2":{"79":1,"83":1,"85":1,"204":1}}],["variants",{"2":{"158":1}}],["variance",{"2":{"17":2,"19":8,"69":6,"249":1}}],["variationalhiddendropout",{"2":{"64":5}}],["variable",{"2":{"59":2,"102":1,"249":1}}],["variables",{"2":{"52":2,"59":1,"77":1,"83":2,"241":1,"249":1,"254":1}}],["var",{"2":{"19":5,"59":1,"69":7,"109":2,"110":2,"126":1,"129":1,"184":1,"207":2,"214":1,"220":24,"241":1,"244":4}}],["validate",{"2":{"187":1}}],["validated",{"2":{"1":1}}],["validation",{"2":{"183":1,"187":101}}],["valid",{"2":{"69":1,"140":1,"164":1}}],["val",{"2":{"10":2,"17":4,"19":2,"33":3,"39":1,"53":4,"54":5,"55":4,"62":7,"64":1,"109":1,"110":2,"126":2,"129":2,"183":4,"184":1,"185":1,"187":2,"207":1,"220":13,"243":8,"245":3}}],["valued",{"2":{"176":1}}],["value>",{"2":{"161":3}}],["value",{"2":{"3":1,"8":1,"10":2,"17":1,"19":4,"33":1,"44":2,"50":1,"53":6,"54":1,"55":3,"59":6,"62":1,"66":5,"69":6,"110":1,"146":2,"179":16,"183":1,"241":1,"244":4,"249":1,"250":4}}],["valuestorage",{"2":{"220":3}}],["values",{"2":{"3":2,"42":2,"52":4,"53":1,"55":1,"56":3,"70":1,"235":1}}],["vec",{"2":{"152":2,"184":1,"185":1,"214":4,"225":1,"243":1,"245":1}}],["vectors",{"2":{"66":1,"67":2,"69":3,"149":1,"174":6,"247":1}}],["vectorization",{"0":{"167":1},"2":{"20":1,"159":1,"167":1}}],["vectorize",{"2":{"13":1}}],["vector",{"0":{"150":1,"151":1,"177":1,"178":1},"2":{"15":1,"18":1,"24":2,"27":9,"34":2,"48":1,"53":3,"59":2,"66":16,"67":5,"83":1,"122":2,"144":2,"149":4,"150":1,"151":1,"153":3,"171":4,"174":2,"175":5,"176":1,"177":3,"178":1,"179":1,"188":1,"207":1,"214":3,"217":1,"249":4,"250":1,"254":1}}],["vendor",{"2":{"20":1}}],["vedaldi",{"2":{"19":1,"69":1}}],["verbatim",{"2":{"59":1}}],["verified",{"2":{"152":1}}],["verification",{"2":{"8":1}}],["verify",{"2":{"55":1,"145":1,"146":1,"147":1,"148":1,"152":1}}],["versioninfo",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"229":3,"237":3,"246":3,"251":3,"260":3}}],["versioning",{"2":{"145":2}}],["versions",{"2":{"102":1,"159":1,"167":2}}],["version",{"2":{"31":1,"52":2,"55":1,"59":1,"71":3,"74":1,"76":1,"95":1,"120":1,"140":1,"150":1,"180":1,"189":1,"197":1,"203":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["very",{"2":{"7":1,"67":1,"77":1,"83":1,"106":2,"108":1,"109":1,"110":2,"133":1,"134":1,"170":1,"171":1,"254":1,"258":1}}],["v1",{"0":{"90":1},"1":{"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":1,"103":1,"104":1,"105":1},"2":{"7":1,"63":2,"76":4,"90":2,"93":1,"130":1}}],["||",{"2":{"78":1,"107":1,"179":1,"236":1,"244":1}}],["|>",{"2":{"77":2,"78":3,"106":4,"107":2,"131":2,"132":3,"153":2,"160":2,"179":1,"187":3,"207":1,"208":2,"209":1,"215":2,"217":1,"225":2,"228":3,"235":1,"236":1,"244":3,"256":1}}],["|p|−di×",{"2":{"63":1,"65":3}}],["|y^−y|",{"2":{"53":1}}],["|y−y^|−0",{"2":{"53":1}}],["|y−y^|≤δδ∗",{"2":{"53":1}}],["|",{"2":{"3":4,"100":1,"109":5,"110":13,"245":1}}],["x~",{"2":{"250":1}}],["x~|θ",{"2":{"250":1}}],["x~|x",{"2":{"250":1}}],["x∈",{"2":{"243":1}}],["xyt",{"2":{"242":17,"243":18,"244":10}}],["xi∈rn",{"2":{"179":1}}],["xi",{"2":{"179":2}}],["xᵢ",{"2":{"107":4}}],["xdev",{"2":{"106":5,"107":3}}],["x3",{"2":{"62":1}}],["x3c",{"2":{"3":3,"4":2,"7":2,"16":1,"18":1,"42":3,"63":4,"66":2,"121":2,"123":2,"134":1,"140":1,"141":1,"161":6,"176":1,"184":1,"208":1,"214":2,"218":2,"220":2,"241":1,"244":2,"254":2}}],["x2s",{"2":{"248":8}}],["x2",{"2":{"62":3,"248":2,"250":18}}],["x26",{"2":{"34":6,"74":1,"127":2,"128":4,"180":4,"189":4,"197":4,"208":2,"210":4,"221":4,"229":4,"237":4,"244":4,"246":4,"251":4,"254":4,"260":4}}],["x1s",{"2":{"248":8}}],["x1",{"2":{"62":3,"248":2,"250":18}}],["xlogy",{"2":{"54":1}}],["xlogx",{"2":{"54":1}}],["xlabel=",{"2":{"232":1,"236":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["xladevice",{"2":{"2":2}}],["xla",{"0":{"73":1},"2":{"2":1,"73":3,"106":5,"107":1}}],["x=",{"2":{"49":1}}],["xoshiro",{"2":{"24":3,"34":1,"59":5,"78":1,"109":1,"126":2,"153":1,"155":1,"174":3,"187":1,"228":1}}],["xavier",{"2":{"24":4}}],["xs",{"2":{"20":2,"245":6,"248":1,"249":3}}],["xt0s",{"2":{"248":5}}],["xt1s",{"2":{"248":5}}],["xt",{"2":{"19":2,"123":1,"124":1}}],["x86",{"2":{"18":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["x",{"2":{"3":12,"5":12,"7":3,"8":12,"11":2,"13":10,"14":2,"15":10,"16":3,"17":8,"18":3,"19":26,"20":1,"24":7,"27":6,"28":10,"37":2,"39":2,"49":5,"53":2,"54":18,"55":16,"58":2,"59":31,"62":24,"63":23,"64":9,"65":27,"66":20,"67":15,"68":30,"69":13,"70":14,"77":4,"78":12,"106":14,"107":3,"109":4,"110":10,"114":3,"119":3,"121":9,"122":2,"123":9,"126":3,"129":2,"131":4,"132":9,"133":6,"134":10,"140":6,"141":5,"142":7,"144":6,"145":10,"146":12,"148":8,"150":2,"151":2,"152":9,"153":3,"155":3,"158":3,"160":5,"171":7,"172":2,"173":4,"176":7,"177":5,"178":1,"179":11,"183":6,"184":8,"185":8,"186":2,"187":4,"192":6,"194":2,"195":4,"206":4,"213":6,"214":10,"216":2,"217":2,"218":2,"220":13,"224":4,"225":2,"227":2,"228":2,"232":9,"236":8,"243":7,"245":1,"248":1,"249":5,"250":4,"254":6,"258":2}}],["3+0",{"2":{"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["378816223378737e",{"2":{"258":1}}],["373354659957113e",{"2":{"258":1}}],["3733055f",{"2":{"256":1}}],["3775488128143772e",{"2":{"258":1}}],["37712055415936e",{"2":{"258":1}}],["3773209f",{"2":{"256":1}}],["375791f",{"2":{"256":1}}],["37501",{"2":{"244":1}}],["379666400123996e",{"2":{"258":1}}],["3796269929557296e",{"2":{"258":1}}],["379862709088791e",{"2":{"258":1}}],["379711f",{"2":{"256":1}}],["379966f",{"2":{"256":1}}],["3727",{"2":{"249":1}}],["37275374",{"2":{"232":1}}],["3729234",{"2":{"147":1}}],["370822625345749e",{"2":{"258":1}}],["370722f",{"2":{"256":1}}],["3707023",{"2":{"145":1}}],["370217f",{"2":{"256":1}}],["3705",{"2":{"249":1}}],["370086340629576e",{"2":{"258":1}}],["37001",{"2":{"244":1}}],["3700787",{"2":{"232":2}}],["37037",{"2":{"217":2}}],["37",{"2":{"217":5,"219":1,"228":2}}],["376114024911682e",{"2":{"258":1}}],["376376062284453e",{"2":{"258":1}}],["3763510631684424e",{"2":{"258":1}}],["376383",{"2":{"148":1}}],["3765398f",{"2":{"256":1}}],["376",{"2":{"208":1}}],["374030813774188e",{"2":{"258":1}}],["374089f",{"2":{"256":1}}],["37454",{"2":{"187":1}}],["3742405e",{"2":{"179":1}}],["3719285",{"2":{"148":1}}],["3rd",{"0":{"203":1},"2":{"147":1}}],["3469765802004945e",{"2":{"258":1}}],["3453546f",{"2":{"256":1}}],["34501",{"2":{"244":1}}],["345025",{"2":{"148":1}}],["348691927585022e",{"2":{"258":1}}],["348867571678961e",{"2":{"258":1}}],["3480024232437663e",{"2":{"258":1}}],["34875f",{"2":{"256":1}}],["3487f",{"2":{"256":1}}],["34855f",{"2":{"77":1}}],["3470885f",{"2":{"256":1}}],["3479493f",{"2":{"256":1}}],["34717593",{"2":{"179":1}}],["340563489471322e",{"2":{"258":1}}],["3405635f",{"2":{"256":1}}],["3401",{"2":{"249":1}}],["34001",{"2":{"244":1}}],["34",{"2":{"228":3,"249":1}}],["341577e",{"2":{"208":1}}],["3410565",{"2":{"141":1}}],["34254",{"2":{"171":1}}],["34253937",{"2":{"171":1}}],["342539",{"2":{"171":2}}],["343680732386287e",{"2":{"258":1}}],["3436353225087703",{"2":{"153":2}}],["3435106f",{"2":{"256":1}}],["34351700231383653",{"2":{"174":1}}],["34310",{"2":{"187":1}}],["343211im",{"2":{"168":1}}],["34383082",{"2":{"126":1}}],["349724314275018e",{"2":{"258":1}}],["3491435f",{"2":{"256":1}}],["3490",{"2":{"249":1}}],["349",{"2":{"131":2}}],["3619614507954043e",{"2":{"258":1}}],["36834049720832e",{"2":{"258":1}}],["36821f",{"2":{"256":1}}],["36829436",{"2":{"179":1}}],["3669",{"2":{"249":1}}],["36001",{"2":{"244":1}}],["3602331",{"2":{"147":1}}],["363144265837922e",{"2":{"258":1}}],["3633118f",{"2":{"256":1}}],["3637495",{"2":{"232":1}}],["36399",{"2":{"187":1}}],["36",{"2":{"228":5}}],["36741",{"2":{"187":1}}],["3645461",{"2":{"232":1}}],["36482",{"2":{"187":1}}],["36447",{"2":{"187":1}}],["36428708",{"2":{"145":1}}],["3694",{"2":{"249":1}}],["3690",{"2":{"249":1}}],["3693867",{"2":{"232":1}}],["36938f",{"2":{"77":1}}],["3691778381831775",{"2":{"174":1}}],["369178",{"2":{"171":2}}],["36918",{"2":{"171":1}}],["3653",{"2":{"249":1}}],["36501",{"2":{"244":1}}],["3650193059617517",{"2":{"153":2}}],["36542922",{"2":{"147":1}}],["3623026f",{"2":{"256":1}}],["3623813f",{"2":{"256":1}}],["3628",{"2":{"249":1}}],["36220473",{"2":{"232":2}}],["36221",{"2":{"187":1}}],["36222499022985155",{"2":{"153":2}}],["362593",{"2":{"106":1}}],["316926886863775e",{"2":{"258":1}}],["316659f",{"2":{"256":1}}],["3121163312703365e",{"2":{"258":1}}],["312072f",{"2":{"256":1}}],["314327421523914e",{"2":{"258":1}}],["3141594f",{"2":{"256":1}}],["3147238",{"2":{"77":1}}],["311395112156149e",{"2":{"258":1}}],["311366f",{"2":{"256":1}}],["3118267576678817e",{"2":{"258":1}}],["3114480480411414e",{"2":{"258":1}}],["3119574f",{"2":{"256":1}}],["311501f",{"2":{"256":1}}],["31128f",{"2":{"77":1}}],["3192",{"2":{"249":1}}],["31074571517319e",{"2":{"258":1}}],["3109782f",{"2":{"256":1}}],["3105534f",{"2":{"256":1}}],["31001",{"2":{"244":1}}],["3108876",{"2":{"147":1}}],["3170582",{"2":{"232":1}}],["31",{"2":{"217":1,"228":2,"249":2}}],["313153729003915e",{"2":{"258":1}}],["313154082994632e",{"2":{"258":1}}],["3130655f",{"2":{"256":1}}],["3133377113386027e",{"2":{"258":1}}],["3133375f",{"2":{"256":1}}],["313347f",{"2":{"256":1}}],["313944e",{"2":{"208":1}}],["313226f",{"2":{"106":3}}],["31889191259365e",{"2":{"258":1}}],["318696f",{"2":{"256":1}}],["31861955",{"2":{"145":1}}],["31828",{"2":{"187":1}}],["3155",{"2":{"249":1}}],["31550723",{"2":{"106":1}}],["31501",{"2":{"244":1}}],["31571",{"2":{"187":1}}],["359462955510926e",{"2":{"258":1}}],["3596170846996523e",{"2":{"258":1}}],["3592706f",{"2":{"256":1}}],["357425245849798e",{"2":{"258":1}}],["3575589763591236e",{"2":{"258":1}}],["3575146f",{"2":{"256":1}}],["357693f",{"2":{"256":1}}],["354340099566823e",{"2":{"258":1}}],["3542095f",{"2":{"256":1}}],["354464",{"2":{"106":1}}],["3552827f",{"2":{"256":1}}],["35501",{"2":{"244":1}}],["3558193508137715",{"2":{"153":2}}],["35001",{"2":{"244":1}}],["3501",{"2":{"244":1}}],["35395628",{"2":{"232":1}}],["35608864083336e",{"2":{"258":1}}],["3561467f",{"2":{"256":1}}],["35653025",{"2":{"232":1}}],["35638642",{"2":{"147":1}}],["3521537011914334e",{"2":{"258":1}}],["3524",{"2":{"249":1}}],["3525s\\ttraining",{"2":{"217":1}}],["3523193",{"2":{"179":1}}],["35",{"2":{"180":2,"189":2,"210":2,"217":1,"221":2,"228":3,"229":2,"237":2,"246":2,"249":1}}],["3585333399770132e",{"2":{"258":1}}],["3580245f",{"2":{"256":1}}],["3580s\\ttraining",{"2":{"217":1}}],["3582856f",{"2":{"256":1}}],["35817",{"2":{"171":1}}],["35836592",{"2":{"77":1}}],["351",{"2":{"208":1}}],["35149138733595564",{"2":{"174":4}}],["351491",{"2":{"171":1}}],["351662",{"2":{"168":1}}],["35181466",{"2":{"126":1}}],["35188f",{"2":{"77":1}}],["35152f",{"2":{"77":1}}],["3834254f",{"2":{"256":1}}],["3834447782571341",{"2":{"153":1}}],["3834447782571344",{"2":{"153":1}}],["388100691361851e",{"2":{"258":1}}],["388877069345636e",{"2":{"258":1}}],["38898644391641e",{"2":{"258":1}}],["388293f",{"2":{"256":1}}],["3886",{"2":{"249":1}}],["3861532641912734e",{"2":{"258":1}}],["386831f",{"2":{"256":1}}],["3864255f",{"2":{"256":1}}],["3864",{"2":{"249":1}}],["389925853996703e",{"2":{"258":1}}],["389176814917797e",{"2":{"258":1}}],["389117f",{"2":{"256":1}}],["38911813",{"2":{"106":1}}],["3891182",{"2":{"106":1}}],["389868881752165e",{"2":{"258":1}}],["3898245f",{"2":{"256":1}}],["3897335f",{"2":{"256":1}}],["3890446f",{"2":{"256":1}}],["387014516933704e",{"2":{"258":1}}],["3871873",{"2":{"235":1}}],["38798f",{"2":{"77":1}}],["38",{"2":{"219":1,"228":16}}],["38501",{"2":{"244":1}}],["3850993",{"2":{"126":1}}],["3855s\\ttraining",{"2":{"217":1}}],["38588",{"2":{"187":1}}],["38200346",{"2":{"232":1}}],["382971",{"2":{"168":1}}],["38242024",{"2":{"148":1}}],["380812272082148e",{"2":{"258":1}}],["3809155f",{"2":{"256":1}}],["3801225f",{"2":{"256":1}}],["38001",{"2":{"244":1}}],["38009",{"2":{"187":1}}],["380777f0",{"2":{"146":1}}],["38068f",{"2":{"77":1}}],["38187847",{"2":{"126":1}}],["3841858f",{"2":{"106":2}}],["325155222358853e",{"2":{"258":1}}],["3251843f",{"2":{"256":1}}],["3257828950352319e",{"2":{"258":1}}],["3258273f",{"2":{"256":1}}],["32501",{"2":{"244":1}}],["3294",{"2":{"249":1}}],["3291591",{"2":{"235":1}}],["32984197",{"2":{"77":1}}],["328861237231886e",{"2":{"258":1}}],["3281066f",{"2":{"256":1}}],["3287905f",{"2":{"256":1}}],["328542",{"2":{"232":1}}],["32839986131789933",{"2":{"153":2}}],["321",{"2":{"220":6}}],["321506",{"2":{"168":1}}],["320390905171548e",{"2":{"258":1}}],["3204353f",{"2":{"256":1}}],["32001",{"2":{"244":1}}],["320",{"2":{"220":6}}],["32s",{"2":{"196":1}}],["326",{"2":{"208":1}}],["32659",{"2":{"187":1}}],["32623518",{"2":{"126":1}}],["327784718404539e",{"2":{"258":1}}],["3279143085132605e",{"2":{"258":1}}],["3279041356366077",{"2":{"153":2}}],["327565f",{"2":{"256":1}}],["3270",{"2":{"249":1}}],["32705",{"2":{"187":1}}],["32788",{"2":{"187":1}}],["32322538",{"2":{"126":1}}],["32",{"2":{"37":2,"59":2,"78":13,"106":9,"107":2,"109":1,"110":1,"193":2,"206":1,"208":4,"226":2,"228":4,"233":1,"244":1,"251":1,"256":8,"260":1}}],["3=dense",{"2":{"32":1}}],["3dv",{"2":{"53":1}}],["3d",{"2":{"19":1,"28":1,"53":1,"146":1,"187":1,"228":2,"236":1}}],["308393406822397e",{"2":{"258":1}}],["3037814352846633e",{"2":{"258":1}}],["3039032035799645e",{"2":{"258":1}}],["303562f",{"2":{"256":1}}],["3035",{"2":{"249":1}}],["3043880969255088e",{"2":{"258":1}}],["3044723807714801e",{"2":{"258":1}}],["3049663f",{"2":{"256":1}}],["3046683f",{"2":{"256":1}}],["3040036f",{"2":{"256":1}}],["3040",{"2":{"249":1}}],["3048",{"2":{"249":1}}],["30501",{"2":{"244":1}}],["3050475",{"2":{"235":1}}],["30569053",{"2":{"235":1}}],["305844",{"2":{"147":1,"148":1}}],["3070866",{"2":{"232":2}}],["3071294",{"2":{"148":1}}],["30277",{"2":{"187":1}}],["30215",{"2":{"171":1}}],["302317",{"2":{"168":1}}],["3062737465727374e",{"2":{"258":1}}],["30673835",{"2":{"232":1}}],["30674052",{"2":{"179":1}}],["306641",{"2":{"168":1}}],["306147",{"2":{"168":1}}],["3092",{"2":{"249":1}}],["3096",{"2":{"249":1}}],["3097294",{"2":{"179":1}}],["309",{"2":{"145":2}}],["3008",{"2":{"249":1}}],["30001",{"2":{"244":1}}],["3001",{"2":{"179":1,"244":1}}],["300",{"2":{"107":1}}],["301",{"2":{"77":1,"208":1}}],["30",{"2":{"17":1,"24":1,"196":1,"217":2,"228":4,"249":1}}],["3×2",{"2":{"59":2,"131":2}}],["3×5",{"2":{"53":4,"59":1}}],["3×3×1×1",{"2":{"24":1}}],["3×7",{"2":{"5":2}}],["3×13",{"2":{"5":4}}],["331929988234545e",{"2":{"258":1}}],["332073583762591e",{"2":{"258":1}}],["3320535f",{"2":{"256":1}}],["3320516f",{"2":{"256":1}}],["332175074837196e",{"2":{"258":1}}],["3326544f",{"2":{"256":1}}],["3335",{"2":{"249":1}}],["3333333333333335",{"2":{"249":1}}],["33333",{"2":{"217":15,"219":2}}],["3354409367994097e",{"2":{"258":1}}],["33544478",{"2":{"232":1}}],["3352486f",{"2":{"256":1}}],["3358998f",{"2":{"256":1}}],["3356",{"2":{"249":1}}],["33501",{"2":{"244":1}}],["3385826",{"2":{"232":2}}],["3306438163226055e",{"2":{"258":1}}],["33063",{"2":{"187":1}}],["330396f",{"2":{"256":1}}],["33001",{"2":{"244":1}}],["33070865",{"2":{"232":2}}],["33094",{"2":{"168":1}}],["3360425811123943e",{"2":{"258":1}}],["33662",{"2":{"187":1}}],["3367505",{"2":{"126":1}}],["3343754",{"2":{"232":1}}],["33435",{"2":{"187":1}}],["3344351",{"2":{"148":1}}],["33442986",{"2":{"106":1}}],["339079",{"2":{"148":1}}],["33758628",{"2":{"124":1}}],["33",{"2":{"5":1,"77":2,"78":1,"196":1,"228":2}}],["398162442536506e",{"2":{"258":1}}],["398038",{"2":{"232":1}}],["39800483",{"2":{"148":1}}],["3998375610902485e",{"2":{"258":1}}],["3993283f",{"2":{"256":1}}],["3915336681637446e",{"2":{"258":1}}],["391393253576819e",{"2":{"258":1}}],["391393f",{"2":{"256":1}}],["391122f",{"2":{"256":1}}],["3956",{"2":{"249":1}}],["3952",{"2":{"249":1}}],["39501",{"2":{"244":1}}],["39588368",{"2":{"232":1}}],["395306",{"2":{"145":1}}],["3935",{"2":{"249":1}}],["39300445",{"2":{"235":1}}],["39370078",{"2":{"232":2}}],["3939896",{"2":{"148":1}}],["39264",{"2":{"187":1}}],["3928175",{"2":{"145":1}}],["3960",{"2":{"249":1}}],["396323",{"2":{"168":1}}],["3969839f",{"2":{"106":1}}],["390775700579875e",{"2":{"258":1}}],["390664f",{"2":{"256":1}}],["39001",{"2":{"244":1}}],["39088",{"2":{"187":1}}],["390200114697191",{"2":{"153":2}}],["3903124",{"2":{"106":1}}],["3945689463266466e",{"2":{"258":1}}],["394569f",{"2":{"256":1}}],["3942307655311696",{"2":{"153":2}}],["3940224213371761",{"2":{"153":2}}],["3977319",{"2":{"148":1}}],["39",{"2":{"2":1,"4":2,"8":1,"10":1,"13":2,"16":1,"18":1,"25":1,"32":1,"34":1,"37":1,"47":1,"52":1,"53":1,"54":1,"57":3,"59":7,"62":1,"66":1,"68":1,"69":3,"70":1,"77":2,"83":2,"88":3,"89":1,"95":2,"98":1,"102":1,"106":2,"107":1,"109":1,"110":1,"111":1,"114":2,"116":1,"117":1,"119":1,"120":1,"126":1,"134":2,"136":1,"137":1,"140":5,"141":1,"142":3,"143":1,"144":4,"145":4,"146":1,"147":2,"148":1,"152":3,"153":2,"155":1,"170":5,"171":8,"173":2,"176":1,"177":1,"179":1,"184":3,"186":1,"188":2,"196":2,"204":2,"207":1,"214":1,"217":1,"225":1,"228":3,"232":1,"233":1,"236":1,"248":1,"249":4,"254":1,"255":1,"256":2}}],["3",{"2":{"2":1,"5":3,"24":5,"32":7,"37":2,"39":1,"48":1,"52":1,"53":30,"59":27,"62":17,"63":2,"64":2,"66":4,"68":4,"69":3,"77":10,"78":4,"86":2,"106":16,"107":1,"109":12,"110":7,"121":3,"122":1,"124":1,"126":4,"129":6,"131":1,"132":1,"140":1,"142":1,"145":4,"147":12,"148":7,"153":2,"171":23,"172":1,"173":3,"174":7,"180":6,"184":1,"185":1,"187":14,"189":6,"192":1,"193":9,"195":1,"196":3,"206":1,"208":1,"210":6,"213":1,"215":2,"217":5,"219":1,"220":56,"221":7,"228":7,"229":6,"232":7,"235":4,"237":6,"241":3,"242":2,"243":1,"244":2,"246":6,"249":17,"250":1,"254":12,"255":1,"256":90,"258":94}}],["4f",{"2":{"256":1}}],["4fs",{"2":{"217":1}}],["4t",{"2":{"243":1}}],["4th",{"2":{"127":1}}],["4x",{"2":{"196":1}}],["469541374929302e",{"2":{"258":1}}],["4691588f",{"2":{"256":1}}],["46969",{"2":{"187":1}}],["466916525082814e",{"2":{"258":1}}],["4667796f",{"2":{"256":1}}],["466118f",{"2":{"256":1}}],["4688357680417196e",{"2":{"258":1}}],["468014271430115e",{"2":{"258":1}}],["468396894470992e",{"2":{"258":1}}],["468266f",{"2":{"256":1}}],["4687857",{"2":{"148":1}}],["465138829544799e",{"2":{"258":1}}],["46514034",{"2":{"179":1}}],["46501",{"2":{"244":1}}],["4609526801058776e",{"2":{"258":1}}],["46001",{"2":{"244":1}}],["46061087",{"2":{"232":1}}],["4604845",{"2":{"148":1}}],["464971f",{"2":{"256":1}}],["4644",{"2":{"249":1}}],["464567",{"2":{"232":2}}],["4642s\\ttraining",{"2":{"219":2}}],["46429",{"2":{"187":1}}],["467955117070908e",{"2":{"258":1}}],["4671888f",{"2":{"256":1}}],["4673",{"2":{"249":1}}],["4674s\\ttraining",{"2":{"219":1}}],["46705",{"2":{"187":1}}],["46",{"2":{"196":2,"228":2,"249":1}}],["46293",{"2":{"187":1}}],["461263f",{"2":{"256":1}}],["461908",{"2":{"171":1}}],["4619778701480337",{"2":{"153":2}}],["4614",{"2":{"147":1}}],["46327f",{"2":{"77":1}}],["48501",{"2":{"244":1}}],["4854s\\ttraining",{"2":{"217":1}}],["4851346",{"2":{"142":1}}],["4870646289379445e",{"2":{"258":1}}],["487817f",{"2":{"256":1}}],["4872606f",{"2":{"256":1}}],["4873s\\ttraining",{"2":{"217":1}}],["487762e",{"2":{"208":1}}],["48983693",{"2":{"232":1}}],["4896s\\ttraining",{"2":{"217":1}}],["4899s\\ttraining",{"2":{"217":1}}],["484",{"2":{"210":1,"246":1}}],["484056",{"2":{"168":1}}],["480564260780046e",{"2":{"258":1}}],["48051",{"2":{"187":1}}],["480454500546205e",{"2":{"258":1}}],["4804s\\ttraining",{"2":{"217":1}}],["4803448f",{"2":{"256":1}}],["4803426f",{"2":{"256":1}}],["48001",{"2":{"244":1}}],["48027995",{"2":{"232":1}}],["480",{"2":{"220":3}}],["48089",{"2":{"187":1}}],["48",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"228":4,"229":3,"237":3,"246":3}}],["48263642",{"2":{"179":1}}],["48203f",{"2":{"77":1}}],["48351598",{"2":{"179":1}}],["486214",{"2":{"168":1}}],["486550215883336",{"2":{"153":2}}],["488228711731884e",{"2":{"258":1}}],["488287f",{"2":{"256":1}}],["48800909240845e",{"2":{"258":1}}],["4880",{"2":{"249":1}}],["488028235902512",{"2":{"153":2}}],["48818898",{"2":{"232":2}}],["488105",{"2":{"147":1,"148":1}}],["488623",{"2":{"148":1}}],["481152845444233e",{"2":{"258":1}}],["481463f",{"2":{"256":1}}],["48148",{"2":{"217":6,"219":1}}],["481039e",{"2":{"208":1}}],["4810884",{"2":{"145":1}}],["48193252",{"2":{"145":1}}],["48137343",{"2":{"126":1}}],["4296521011660889e",{"2":{"258":1}}],["429942f",{"2":{"256":1}}],["428551165225825e",{"2":{"258":1}}],["4281108653324745e",{"2":{"258":1}}],["4281251f",{"2":{"256":1}}],["428122e",{"2":{"208":1}}],["428924640335857e",{"2":{"258":1}}],["4289274f",{"2":{"256":1}}],["428917371946206e",{"2":{"258":1}}],["4286144f",{"2":{"256":1}}],["4283674f",{"2":{"256":1}}],["4277078442324252e",{"2":{"258":1}}],["4271764724227435e",{"2":{"258":1}}],["4270458f",{"2":{"256":1}}],["427237f",{"2":{"256":1}}],["422680986988012e",{"2":{"258":1}}],["42242297816603e",{"2":{"258":1}}],["4227842f",{"2":{"256":1}}],["422292f",{"2":{"256":1}}],["4225",{"2":{"249":1}}],["422918",{"2":{"171":1}}],["4258927529299515e",{"2":{"258":1}}],["42501",{"2":{"244":1}}],["42519686",{"2":{"232":2}}],["42512",{"2":{"187":1}}],["4266933634826405e",{"2":{"258":1}}],["42669478",{"2":{"106":1}}],["42621575f",{"2":{"256":1}}],["4267512f",{"2":{"256":1}}],["426",{"2":{"208":1}}],["42391",{"2":{"187":1}}],["42189494",{"2":{"179":1}}],["4216236",{"2":{"126":1}}],["4242767",{"2":{"179":1}}],["42",{"2":{"168":2,"228":2}}],["42001",{"2":{"244":1}}],["420058350197999",{"2":{"153":1}}],["42005835019799886",{"2":{"153":1}}],["420698",{"2":{"147":1,"148":1}}],["40987514531405e",{"2":{"258":1}}],["409552f",{"2":{"256":1}}],["404527913585472e",{"2":{"258":1}}],["40473",{"2":{"171":1}}],["404728",{"2":{"171":2}}],["408731097284129e",{"2":{"258":1}}],["408923263489361e",{"2":{"258":1}}],["408915f",{"2":{"256":1}}],["40898",{"2":{"187":1}}],["40887f",{"2":{"256":1}}],["405886536388658e",{"2":{"258":1}}],["40556f",{"2":{"256":1}}],["4059",{"2":{"249":1}}],["40501",{"2":{"244":1}}],["4051151",{"2":{"176":4}}],["4061",{"2":{"249":1}}],["40623155",{"2":{"235":1}}],["4066515",{"2":{"148":1}}],["407404283008284e",{"2":{"258":1}}],["40741",{"2":{"217":3}}],["4079825f",{"2":{"256":1}}],["40727592",{"2":{"235":1}}],["40789893",{"2":{"179":1}}],["40",{"2":{"217":5,"219":1,"228":3}}],["4016783171330845e",{"2":{"258":1}}],["4018213f",{"2":{"256":1}}],["4018757",{"2":{"232":1}}],["4017",{"2":{"249":1}}],["4015749",{"2":{"232":2}}],["401",{"2":{"208":1}}],["401215",{"2":{"123":1}}],["402007845222216e",{"2":{"258":1}}],["40298",{"2":{"187":1}}],["40229",{"2":{"171":1}}],["40gb",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["40354207",{"2":{"179":1}}],["400898535837695e",{"2":{"258":1}}],["4008604f",{"2":{"256":1}}],["400692347391857e",{"2":{"258":1}}],["4005883f",{"2":{"256":1}}],["40001",{"2":{"244":1}}],["4001",{"2":{"179":1,"244":1}}],["4007292",{"2":{"147":1}}],["4007905",{"2":{"145":1}}],["400",{"2":{"107":1}}],["4563545887372177e",{"2":{"258":1}}],["4564578f",{"2":{"256":1}}],["4566929",{"2":{"232":2}}],["454497624310447e",{"2":{"258":1}}],["454405293004695e",{"2":{"258":1}}],["454949785197743e",{"2":{"258":1}}],["45467f",{"2":{"256":1}}],["454679",{"2":{"168":1}}],["454598f",{"2":{"256":1}}],["454391f",{"2":{"256":1}}],["4543",{"2":{"249":1}}],["455441f",{"2":{"256":1}}],["45501",{"2":{"244":1}}],["45583528",{"2":{"232":1}}],["4552384158732863",{"2":{"174":4}}],["455238",{"2":{"171":1}}],["45",{"2":{"228":3}}],["45393957640346e",{"2":{"258":1}}],["453099f",{"2":{"256":1}}],["453",{"2":{"189":1}}],["45325348",{"2":{"140":1}}],["451919842992198e",{"2":{"258":1}}],["4517763361935025e",{"2":{"258":1}}],["451",{"2":{"208":1}}],["45149",{"2":{"187":1}}],["4510427",{"2":{"126":1}}],["452113f",{"2":{"256":1}}],["4524014",{"2":{"232":1}}],["45247704",{"2":{"179":1}}],["45298263",{"2":{"77":1}}],["4590840329298094e",{"2":{"258":1}}],["459019",{"2":{"171":1}}],["4591332f",{"2":{"256":1}}],["4596",{"2":{"249":1}}],["4593922",{"2":{"176":4}}],["4505709f",{"2":{"256":1}}],["450581f",{"2":{"106":5}}],["45001",{"2":{"244":1}}],["4500963619011972",{"2":{"153":2}}],["4501",{"2":{"244":1}}],["45016125",{"2":{"179":1}}],["4585489",{"2":{"126":1}}],["417890075803277e",{"2":{"258":1}}],["417246882848665e",{"2":{"258":1}}],["417051f",{"2":{"256":1}}],["417079f",{"2":{"256":1}}],["417747f",{"2":{"256":1}}],["4110441278516157e",{"2":{"258":1}}],["411434079108195e",{"2":{"258":1}}],["411186f",{"2":{"256":1}}],["4112944",{"2":{"106":1}}],["4112945",{"2":{"106":1}}],["4195743426863102e",{"2":{"258":1}}],["4193551194260185e",{"2":{"258":1}}],["419326f",{"2":{"256":1}}],["419951724424006e",{"2":{"258":1}}],["4197938f",{"2":{"256":1}}],["419821f",{"2":{"256":1}}],["41919118",{"2":{"179":1}}],["415563703280549e",{"2":{"258":1}}],["415075687402303e",{"2":{"258":1}}],["41501",{"2":{"244":1}}],["415396f",{"2":{"256":1}}],["4158693023143286",{"2":{"153":2}}],["415858",{"2":{"106":1}}],["41001",{"2":{"244":1}}],["4101033",{"2":{"147":1}}],["41",{"2":{"217":1,"228":3}}],["416051086201375e",{"2":{"258":1}}],["416182f",{"2":{"256":1}}],["416",{"2":{"193":2}}],["4141174802176025e",{"2":{"258":1}}],["4145293f",{"2":{"256":1}}],["414769",{"2":{"171":1}}],["4144732",{"2":{"148":1}}],["412951032746447e",{"2":{"258":1}}],["412172157702465e",{"2":{"258":1}}],["4121052656024447e",{"2":{"258":1}}],["412396f",{"2":{"256":1}}],["41238892",{"2":{"147":1}}],["412848f",{"2":{"256":1}}],["41220",{"2":{"187":1}}],["41246277",{"2":{"126":1}}],["434318725546035e",{"2":{"258":1}}],["4343187f",{"2":{"256":1}}],["432608720218951e",{"2":{"258":1}}],["432554883556168e",{"2":{"258":1}}],["4323365f",{"2":{"256":1}}],["4320224f",{"2":{"256":1}}],["4322f",{"2":{"77":1}}],["43501",{"2":{"244":1}}],["4353992",{"2":{"232":1}}],["4303570092641735e",{"2":{"258":1}}],["4303477f",{"2":{"256":1}}],["4304481076488265e",{"2":{"258":1}}],["430130635767008e",{"2":{"258":1}}],["4301196472136584e",{"2":{"258":1}}],["430164f",{"2":{"256":1}}],["430852f",{"2":{"256":1}}],["43001",{"2":{"244":1}}],["4305115e",{"2":{"152":1}}],["4305781",{"2":{"126":1}}],["437515830585584e",{"2":{"258":1}}],["437757977212133e",{"2":{"258":1}}],["4370183f",{"2":{"256":1}}],["4371308",{"2":{"232":1}}],["43718",{"2":{"187":1}}],["4376572711003852",{"2":{"153":2}}],["436825141329051e",{"2":{"258":1}}],["43669",{"2":{"187":1}}],["43676835",{"2":{"147":1}}],["4339540841721874e",{"2":{"258":1}}],["4339262",{"2":{"179":1}}],["43336467641714e",{"2":{"258":1}}],["4333955f",{"2":{"256":1}}],["4335606f",{"2":{"256":1}}],["43350348",{"2":{"147":1}}],["4330709",{"2":{"232":2}}],["433656",{"2":{"168":1}}],["43118258435799e",{"2":{"258":1}}],["431872115106359e",{"2":{"258":1}}],["4318977",{"2":{"126":1}}],["43102022008836e",{"2":{"258":1}}],["4312666f",{"2":{"256":1}}],["431984f",{"2":{"256":1}}],["4319346874291314",{"2":{"171":1}}],["4314566",{"2":{"148":1}}],["4397957",{"2":{"145":1}}],["4391564",{"2":{"106":1}}],["43af",{"2":{"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["43",{"2":{"95":1,"228":3}}],["438026f",{"2":{"256":1}}],["4382721",{"2":{"232":1}}],["43846482",{"2":{"179":1}}],["438734+0",{"2":{"168":1}}],["438519",{"2":{"106":1}}],["43866f",{"2":{"77":1}}],["438919e",{"2":{"208":1}}],["4389",{"2":{"68":1}}],["474695297555959e",{"2":{"258":1}}],["4743723f",{"2":{"256":1}}],["4742s\\ttraining",{"2":{"219":1}}],["471632677356395e",{"2":{"258":1}}],["471935f",{"2":{"256":1}}],["4715092f",{"2":{"256":1}}],["4791",{"2":{"249":1}}],["4795s\\ttraining",{"2":{"217":1}}],["475672003305463e",{"2":{"258":1}}],["475444f",{"2":{"256":1}}],["47501",{"2":{"244":1}}],["47551f",{"2":{"77":1}}],["470162464794341e",{"2":{"258":1}}],["4703054f",{"2":{"256":1}}],["47001",{"2":{"244":1}}],["47082",{"2":{"187":1}}],["4775001941826805e",{"2":{"258":1}}],["477275972726577e",{"2":{"258":1}}],["4773994f",{"2":{"256":1}}],["4773s\\ttraining",{"2":{"217":1}}],["4778283",{"2":{"232":1}}],["4763s\\ttraining",{"2":{"219":1}}],["4763277207638013",{"2":{"153":1}}],["4763277207638008",{"2":{"153":1}}],["4765s\\ttraining",{"2":{"217":1}}],["476",{"2":{"208":1}}],["47",{"2":{"193":2,"196":1,"228":3}}],["4730743722547668",{"2":{"153":2}}],["47872233",{"2":{"145":1}}],["4721628342109925e",{"2":{"258":1}}],["472176f",{"2":{"256":1}}],["472",{"2":{"66":1}}],["4d",{"2":{"70":3,"107":2}}],["492569097013449e",{"2":{"258":1}}],["492753f",{"2":{"256":1}}],["4949526227566494e",{"2":{"258":1}}],["494954386793892e",{"2":{"258":1}}],["4946847f",{"2":{"256":1}}],["497072391919446e",{"2":{"258":1}}],["4975s\\ttraining",{"2":{"217":1}}],["4910950206325143e",{"2":{"258":1}}],["491066f",{"2":{"256":1}}],["4919331313177951e",{"2":{"258":1}}],["49199095f",{"2":{"256":1}}],["4911s\\ttraining",{"2":{"217":1}}],["495977143793123e",{"2":{"258":1}}],["4951638028921765e",{"2":{"258":1}}],["4951495f",{"2":{"256":1}}],["4952162635599276e",{"2":{"258":1}}],["4953282f",{"2":{"256":1}}],["49501",{"2":{"244":1}}],["49001",{"2":{"244":1}}],["49019",{"2":{"187":1}}],["4901161f",{"2":{"106":6}}],["49012f",{"2":{"106":2}}],["4937",{"2":{"249":1}}],["4936414",{"2":{"232":1}}],["493142im",{"2":{"168":1}}],["496006f",{"2":{"256":1}}],["496063",{"2":{"232":2}}],["4962607f",{"2":{"256":1}}],["4968",{"2":{"249":1}}],["4963s\\ttraining",{"2":{"217":1}}],["498955501547213e",{"2":{"258":1}}],["4989173f",{"2":{"256":1}}],["49804282",{"2":{"232":1}}],["49806",{"2":{"187":1}}],["4987s\\ttraining",{"2":{"217":1}}],["4983s\\ttraining",{"2":{"217":1}}],["4981351",{"2":{"106":1}}],["4996765751026667e",{"2":{"258":1}}],["4996185f",{"2":{"256":1}}],["4990479017262914e",{"2":{"258":1}}],["4999375f",{"2":{"256":1}}],["49998975",{"2":{"106":1}}],["49913f",{"2":{"77":1}}],["49",{"2":{"59":1,"228":2,"233":1,"236":1}}],["449177061193522e",{"2":{"258":1}}],["449167f",{"2":{"256":1}}],["449066849547358e",{"2":{"258":1}}],["444589632336342e",{"2":{"258":1}}],["444844242959467e",{"2":{"258":1}}],["444795f",{"2":{"256":1}}],["44439f",{"2":{"77":1}}],["4472",{"2":{"249":1}}],["44746",{"2":{"171":1}}],["4426369299592315e",{"2":{"258":1}}],["442865f",{"2":{"256":1}}],["4420",{"2":{"249":1}}],["44272",{"2":{"187":1}}],["4456",{"2":{"249":1}}],["44501",{"2":{"244":1}}],["44571686",{"2":{"148":1}}],["44062367160201e",{"2":{"258":1}}],["4406375f",{"2":{"256":1}}],["4404700106549434e",{"2":{"258":1}}],["4404535f",{"2":{"256":1}}],["44048917",{"2":{"179":1}}],["44001",{"2":{"244":1}}],["44096947",{"2":{"235":1}}],["44078717",{"2":{"232":1}}],["441859320852504e",{"2":{"258":1}}],["4418593f",{"2":{"256":1}}],["4418456",{"2":{"179":1}}],["44166",{"2":{"187":1}}],["448889474922055e",{"2":{"258":1}}],["448867f",{"2":{"256":1}}],["4488101521328277",{"2":{"174":1}}],["4486215f",{"2":{"256":1}}],["44807646",{"2":{"107":1}}],["4436004f",{"2":{"256":1}}],["44379275944988e",{"2":{"258":1}}],["443749f",{"2":{"256":1}}],["4437156",{"2":{"147":1}}],["44397",{"2":{"148":1}}],["446214",{"2":{"147":1}}],["44",{"2":{"34":1,"196":3,"228":2,"249":1}}],["4",{"2":{"2":1,"24":1,"31":1,"32":1,"34":8,"52":2,"53":2,"59":2,"64":1,"66":4,"68":1,"69":5,"77":3,"78":1,"86":1,"106":2,"107":4,"121":8,"124":6,"126":5,"129":8,"131":1,"133":8,"134":4,"140":2,"141":2,"145":9,"146":2,"147":3,"148":11,"152":7,"153":16,"155":2,"171":15,"173":5,"176":2,"179":1,"180":3,"187":17,"189":3,"193":2,"195":1,"196":2,"210":3,"217":5,"219":1,"220":33,"221":2,"224":1,"228":2,"229":2,"232":6,"235":1,"237":3,"244":9,"246":3,"248":9,"249":20,"254":7,"255":1,"256":96,"258":94}}],["0f4",{"2":{"255":1}}],["0f",{"2":{"195":1,"215":2}}],["0f0",{"2":{"59":2,"78":2,"123":2,"124":1,"152":2,"183":2,"206":4,"214":4,"218":2,"232":2,"243":8,"245":6,"248":4,"255":1}}],["0+560",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["0x1911b814c02405e8",{"2":{"174":1}}],["0x12c522b8034ae186",{"2":{"126":1}}],["0x8c49bc52dc8a77ea",{"2":{"174":1}}],["0x8e0c3a65079041bb",{"2":{"126":1}}],["0x48d73dc42d195740",{"2":{"174":1}}],["0x4fa3403dd074e603",{"2":{"126":1}}],["0xdb2fa90498613fdf",{"2":{"174":1}}],["0x22a21880af5dc689",{"2":{"126":1,"174":1}}],["0x21617f7747d97206",{"2":{"126":1}}],["0e",{"2":{"77":7,"234":1,"236":1}}],["083119573195054e",{"2":{"258":1}}],["083243f",{"2":{"256":1}}],["0836414",{"2":{"168":1}}],["081267089944231e",{"2":{"258":1}}],["081071f",{"2":{"256":1}}],["0813",{"2":{"249":1}}],["0842",{"2":{"249":1}}],["0843",{"2":{"249":1}}],["08456781",{"2":{"77":1}}],["082484596841146e",{"2":{"258":1}}],["08284969230501e",{"2":{"258":1}}],["0825427f",{"2":{"256":1}}],["0826565f",{"2":{"256":1}}],["082086",{"2":{"236":1}}],["08214",{"2":{"187":1}}],["0829282",{"2":{"145":1}}],["08",{"2":{"177":1}}],["08517",{"2":{"171":1}}],["085170805",{"2":{"171":1}}],["0851708",{"2":{"171":2}}],["08526564",{"2":{"77":1}}],["0880441639035963e",{"2":{"258":1}}],["0885",{"2":{"249":1}}],["0887",{"2":{"249":1}}],["08820387",{"2":{"232":1}}],["088205874",{"2":{"106":1}}],["08845",{"2":{"187":1}}],["0884834705765853",{"2":{"153":2}}],["0867",{"2":{"249":1}}],["0865",{"2":{"249":1}}],["0866141",{"2":{"232":2}}],["08667634",{"2":{"106":2}}],["08694684883050086",{"2":{"174":4}}],["0869468",{"2":{"171":1}}],["086429894",{"2":{"147":1}}],["0872456420034774e",{"2":{"258":1}}],["0875661377992795e",{"2":{"258":1}}],["087552f",{"2":{"256":1}}],["0878162f",{"2":{"256":1}}],["0874344f",{"2":{"256":1}}],["0876",{"2":{"249":1}}],["08794",{"2":{"187":1}}],["08791955",{"2":{"106":1}}],["08737",{"2":{"187":1}}],["08713347",{"2":{"142":1}}],["08922641",{"2":{"232":1}}],["089530945",{"2":{"106":1}}],["08953094",{"2":{"106":1}}],["08993f",{"2":{"77":1}}],["0805876705333804e",{"2":{"258":1}}],["080362002041241e",{"2":{"258":1}}],["0803379",{"2":{"232":1}}],["0804447f",{"2":{"256":1}}],["0800082f",{"2":{"256":1}}],["0801",{"2":{"78":1}}],["08085155",{"2":{"78":1}}],["08022",{"2":{"19":1,"69":1}}],["036589722165776e",{"2":{"258":1}}],["03653",{"2":{"187":1}}],["03",{"2":{"234":1,"236":1}}],["03f0",{"2":{"234":1}}],["0342363f",{"2":{"256":1}}],["03429s\\ttraining",{"2":{"228":1}}],["034259234",{"2":{"179":1}}],["03479s\\ttraining",{"2":{"228":1}}],["03438",{"2":{"187":1}}],["03443",{"2":{"187":1}}],["03704",{"2":{"217":3}}],["03715",{"2":{"187":1}}],["03718",{"2":{"187":1}}],["03779",{"2":{"187":1}}],["03729",{"2":{"187":1}}],["037500158",{"2":{"106":1}}],["0352584f",{"2":{"256":1}}],["035586f",{"2":{"256":1}}],["035556547",{"2":{"244":1}}],["0355555",{"2":{"171":1}}],["03531",{"2":{"187":1}}],["03540",{"2":{"187":1}}],["0350433",{"2":{"77":1}}],["033757f",{"2":{"256":1}}],["033320315",{"2":{"244":1}}],["03322105",{"2":{"232":1}}],["03314",{"2":{"187":1}}],["03353",{"2":{"187":1}}],["0338569",{"2":{"147":1}}],["03301",{"2":{"187":1}}],["0330944",{"2":{"77":1}}],["03306928",{"2":{"77":1}}],["0311341709135985e",{"2":{"258":1}}],["0311537f",{"2":{"256":1}}],["0313460071441674e",{"2":{"258":1}}],["031768262",{"2":{"244":1}}],["03175",{"2":{"187":1}}],["03104",{"2":{"187":1}}],["03100",{"2":{"187":1}}],["03180",{"2":{"187":1}}],["03147",{"2":{"187":1}}],["031414255389526885",{"2":{"153":2}}],["031403534",{"2":{"106":1}}],["031403527",{"2":{"106":1}}],["03152",{"2":{"187":1}}],["031503614",{"2":{"106":1}}],["0315817",{"2":{"77":1}}],["0321347406851244e",{"2":{"258":1}}],["032705",{"2":{"232":1}}],["03235",{"2":{"187":1}}],["03260",{"2":{"187":1}}],["03261588",{"2":{"106":1}}],["032615878",{"2":{"106":1}}],["03205",{"2":{"187":1}}],["0320963",{"2":{"106":1}}],["03258145",{"2":{"179":1}}],["032846544",{"2":{"147":1}}],["032405667",{"2":{"106":1}}],["032405674",{"2":{"106":1}}],["0324213",{"2":{"106":1}}],["03242131",{"2":{"106":1}}],["0390674270919503e",{"2":{"258":1}}],["03901",{"2":{"187":1}}],["039198f",{"2":{"256":1}}],["039132368",{"2":{"77":1}}],["0398",{"2":{"249":1}}],["0392s\\ttraining",{"2":{"217":1}}],["03930",{"2":{"187":1}}],["03994",{"2":{"187":1}}],["039647065",{"2":{"106":2}}],["030419719275371e",{"2":{"258":1}}],["03047",{"2":{"187":1}}],["0303675f",{"2":{"256":1}}],["030331207",{"2":{"244":1}}],["03032s\\ttraining",{"2":{"228":1}}],["030095264244136e",{"2":{"258":1}}],["030016",{"2":{"236":1}}],["0300211165293852e",{"2":{"258":1}}],["03002",{"2":{"187":1}}],["03070261",{"2":{"107":1}}],["030926049",{"2":{"106":1}}],["030926052",{"2":{"106":1}}],["03094203",{"2":{"77":1}}],["030881835",{"2":{"106":2}}],["0301095f",{"2":{"256":1}}],["03018",{"2":{"187":1}}],["03012",{"2":{"187":1}}],["0301",{"2":{"78":1}}],["0302464",{"2":{"77":1}}],["0386783729061495e",{"2":{"258":1}}],["0383734970663711e",{"2":{"258":1}}],["03834",{"2":{"187":1}}],["0389506f",{"2":{"256":1}}],["0384627f",{"2":{"256":1}}],["0385415f",{"2":{"256":1}}],["03853",{"2":{"187":1}}],["03820s\\ttraining",{"2":{"228":1}}],["03805s\\ttraining",{"2":{"228":1}}],["03806",{"2":{"187":1}}],["03804328",{"2":{"77":1}}],["03813s\\ttraining",{"2":{"228":1}}],["0381317",{"2":{"148":1}}],["03811",{"2":{"187":1}}],["0381727",{"2":{"147":1}}],["038",{"2":{"69":3}}],["02f0",{"2":{"245":2}}],["02",{"2":{"208":21,"228":1}}],["0212086273065653e",{"2":{"258":1}}],["0212578f",{"2":{"256":1}}],["02121s\\ttraining",{"2":{"228":1}}],["02151s\\ttraining",{"2":{"228":1}}],["02158",{"2":{"187":1}}],["02184s\\ttraining",{"2":{"228":1}}],["02107s\\ttraining",{"2":{"228":1}}],["02104",{"2":{"187":1}}],["021166507",{"2":{"244":1}}],["02111s\\ttraining",{"2":{"228":1}}],["02119s\\ttraining",{"2":{"228":1}}],["02117s\\ttraining",{"2":{"228":1}}],["02177s\\ttraining",{"2":{"228":1}}],["02176",{"2":{"187":1}}],["02198s\\ttraining",{"2":{"228":1}}],["02131",{"2":{"187":1}}],["02132s\\ttraining",{"2":{"228":1}}],["02132",{"2":{"187":1}}],["02162s\\ttraining",{"2":{"228":3}}],["02167s\\ttraining",{"2":{"228":1}}],["02167",{"2":{"187":1}}],["02165",{"2":{"187":1}}],["02163",{"2":{"187":1}}],["02168",{"2":{"187":1}}],["02145s\\ttraining",{"2":{"228":1}}],["02141s\\ttraining",{"2":{"228":1}}],["02141",{"2":{"187":1}}],["02143306",{"2":{"107":1}}],["0256",{"2":{"249":1}}],["02561",{"2":{"187":1}}],["025140665",{"2":{"244":1}}],["025281426",{"2":{"244":1}}],["02524s\\ttraining",{"2":{"228":1}}],["025583776",{"2":{"244":1}}],["025800283338583e",{"2":{"258":1}}],["02583s\\ttraining",{"2":{"228":1}}],["02589s\\ttraining",{"2":{"228":1}}],["02502s\\ttraining",{"2":{"228":1}}],["02508s\\ttraining",{"2":{"228":1}}],["02500",{"2":{"187":1}}],["02578",{"2":{"187":1}}],["02531",{"2":{"187":1}}],["02537",{"2":{"187":1}}],["02532",{"2":{"187":1}}],["02536",{"2":{"187":1}}],["02538",{"2":{"187":1}}],["02535537",{"2":{"179":1}}],["02535769",{"2":{"147":1}}],["0295358070672964e",{"2":{"258":1}}],["02955f",{"2":{"256":1}}],["0298279f",{"2":{"256":1}}],["02983",{"2":{"187":1}}],["029164422",{"2":{"244":1}}],["02918",{"2":{"187":1}}],["02964765308691042",{"2":{"174":4}}],["0296477",{"2":{"171":1}}],["029290024",{"2":{"77":1}}],["0288",{"2":{"249":1}}],["02888",{"2":{"187":1}}],["028305896",{"2":{"244":1}}],["02835",{"2":{"187":1}}],["028128654",{"2":{"244":1}}],["02814",{"2":{"187":1}}],["02874s\\ttraining",{"2":{"228":1}}],["02872",{"2":{"187":1}}],["028738942",{"2":{"147":1}}],["0285962",{"2":{"147":1}}],["028461456",{"2":{"126":1}}],["02827571",{"2":{"106":1}}],["028275706",{"2":{"106":1}}],["028251555",{"2":{"77":1}}],["027273500",{"2":{"244":1}}],["027226416",{"2":{"106":1}}],["027226413",{"2":{"106":1}}],["02788s\\ttraining",{"2":{"228":1}}],["02742s\\ttraining",{"2":{"228":1}}],["02744",{"2":{"187":1}}],["0277897",{"2":{"232":1}}],["02777",{"2":{"187":1}}],["027702922",{"2":{"77":1}}],["027620461",{"2":{"244":1}}],["02762",{"2":{"187":1}}],["02009s\\ttraining",{"2":{"228":1}}],["02005s\\ttraining",{"2":{"228":1}}],["02024s\\ttraining",{"2":{"228":1}}],["02027s\\ttraining",{"2":{"228":1}}],["0209",{"2":{"249":1}}],["02091s\\ttraining",{"2":{"228":1}}],["02097s\\ttraining",{"2":{"228":1}}],["020301901106715e",{"2":{"258":1}}],["02030s\\ttraining",{"2":{"228":1}}],["020389127",{"2":{"244":1}}],["02036s\\ttraining",{"2":{"228":1}}],["020624701",{"2":{"244":1}}],["02068s\\ttraining",{"2":{"228":1}}],["02061s\\ttraining",{"2":{"228":2}}],["02069s\\ttraining",{"2":{"228":1}}],["02063s\\ttraining",{"2":{"228":1}}],["02065s\\ttraining",{"2":{"228":1}}],["02066s\\ttraining",{"2":{"228":1}}],["02060",{"2":{"187":2}}],["0204",{"2":{"249":1}}],["020493284",{"2":{"244":1}}],["02049s\\ttraining",{"2":{"228":2}}],["02048s\\ttraining",{"2":{"228":1}}],["02044s\\ttraining",{"2":{"228":1}}],["02040s\\ttraining",{"2":{"228":1}}],["02047s\\ttraining",{"2":{"228":1}}],["02042s\\ttraining",{"2":{"228":1}}],["02046s\\ttraining",{"2":{"228":2}}],["02085s\\ttraining",{"2":{"228":1}}],["02059s\\ttraining",{"2":{"228":1}}],["02057s\\ttraining",{"2":{"228":2}}],["02058s\\ttraining",{"2":{"228":2}}],["02051s\\ttraining",{"2":{"228":1}}],["02050s\\ttraining",{"2":{"228":1}}],["02052s\\ttraining",{"2":{"228":2}}],["02056s\\ttraining",{"2":{"228":3}}],["02055s\\ttraining",{"2":{"228":3}}],["02054s\\ttraining",{"2":{"228":1}}],["02053s\\ttraining",{"2":{"228":4}}],["02074s\\ttraining",{"2":{"228":1}}],["02073s\\ttraining",{"2":{"228":1}}],["02076s\\ttraining",{"2":{"228":1}}],["02075s\\ttraining",{"2":{"228":1}}],["02078s\\ttraining",{"2":{"228":1}}],["02078",{"2":{"187":1}}],["0207279",{"2":{"140":1}}],["0207926",{"2":{"77":1}}],["020171f",{"2":{"256":1}}],["02013",{"2":{"187":1}}],["02014",{"2":{"187":1}}],["0201",{"2":{"78":1}}],["0237235054718365e",{"2":{"258":1}}],["023768f",{"2":{"256":1}}],["02377",{"2":{"187":1}}],["023622",{"2":{"232":2}}],["02363",{"2":{"187":1}}],["02384s\\ttraining",{"2":{"228":1}}],["02335",{"2":{"187":1}}],["02393",{"2":{"187":1}}],["02390",{"2":{"187":1}}],["02328",{"2":{"187":1}}],["0231775",{"2":{"77":1}}],["023031702",{"2":{"77":1}}],["022698f",{"2":{"256":1}}],["022618050",{"2":{"244":1}}],["02268s\\ttraining",{"2":{"228":1}}],["0227933728156163e",{"2":{"258":1}}],["022739200",{"2":{"244":1}}],["022735020",{"2":{"244":1}}],["0227737",{"2":{"77":1}}],["022293953",{"2":{"244":1}}],["022215",{"2":{"236":1}}],["02228s\\ttraining",{"2":{"228":1}}],["022150228",{"2":{"244":1}}],["02218s\\ttraining",{"2":{"228":1}}],["02210",{"2":{"187":1}}],["02205",{"2":{"187":1}}],["02283",{"2":{"187":1}}],["02280",{"2":{"187":1}}],["0228109822209213",{"2":{"153":2}}],["0229614f",{"2":{"256":1}}],["02299",{"2":{"187":1}}],["02297",{"2":{"187":1}}],["02237s\\ttraining",{"2":{"228":1}}],["02236",{"2":{"187":1}}],["02232",{"2":{"187":1}}],["02259",{"2":{"187":1}}],["022434518",{"2":{"77":1}}],["026113275134092e",{"2":{"258":1}}],["026170366",{"2":{"244":1}}],["02618",{"2":{"187":1}}],["026297f",{"2":{"256":1}}],["02629s\\ttraining",{"2":{"228":1}}],["026282774",{"2":{"244":1}}],["02605s\\ttraining",{"2":{"228":1}}],["02675s\\ttraining",{"2":{"228":1}}],["02676",{"2":{"187":1}}],["02694s\\ttraining",{"2":{"228":1}}],["02699",{"2":{"187":1}}],["02654",{"2":{"187":1}}],["026516732",{"2":{"77":1}}],["02630",{"2":{"187":1}}],["026851978",{"2":{"77":1}}],["024325899761714e",{"2":{"258":1}}],["02438",{"2":{"187":1}}],["0242",{"2":{"249":1}}],["0240",{"2":{"249":1}}],["02482s\\ttraining",{"2":{"228":1}}],["02483",{"2":{"187":1}}],["02471s\\ttraining",{"2":{"228":1}}],["0246",{"2":{"249":1}}],["02462",{"2":{"187":1}}],["02466",{"2":{"187":1}}],["02499",{"2":{"187":1}}],["02441f",{"2":{"256":1}}],["02443",{"2":{"187":1}}],["0244548f",{"2":{"106":1}}],["0244214",{"2":{"77":1}}],["024",{"2":{"77":1}}],["094196443108954e",{"2":{"258":1}}],["0949668200481244e",{"2":{"258":1}}],["09450001936559e",{"2":{"258":1}}],["0945444f",{"2":{"256":1}}],["0946174",{"2":{"168":1}}],["09",{"2":{"228":1,"249":1}}],["0935077",{"2":{"254":2}}],["09351656",{"2":{"177":1,"178":1}}],["09384",{"2":{"187":1}}],["09368",{"2":{"187":1}}],["093925126",{"2":{"179":1}}],["098011617376218e",{"2":{"258":1}}],["098719f",{"2":{"256":1}}],["0987538",{"2":{"168":1}}],["09855",{"2":{"187":1}}],["09852",{"2":{"171":1}}],["0985227",{"2":{"171":2}}],["098861866",{"2":{"77":1}}],["091562801430886e",{"2":{"258":1}}],["0913433f",{"2":{"256":1}}],["0913593358445652",{"2":{"153":1}}],["091359335844565",{"2":{"153":1}}],["0914626",{"2":{"148":1}}],["09255",{"2":{"187":1}}],["09299411",{"2":{"107":1}}],["09289699",{"2":{"106":1}}],["0963497f",{"2":{"256":1}}],["0961197f",{"2":{"256":1}}],["0965776141546363e",{"2":{"258":1}}],["0965",{"2":{"249":1}}],["09685",{"2":{"187":1}}],["09693718",{"2":{"235":1}}],["09699093",{"2":{"147":1}}],["096988946",{"2":{"126":1}}],["09692798",{"2":{"77":1}}],["09643307",{"2":{"106":2}}],["095104054592046e",{"2":{"258":1}}],["095135f",{"2":{"256":1}}],["0958833081965815e",{"2":{"258":1}}],["095372f",{"2":{"256":1}}],["0959952f",{"2":{"256":1}}],["09599f",{"2":{"77":1}}],["095568806",{"2":{"235":1}}],["0954186",{"2":{"106":1}}],["0991955",{"2":{"106":1}}],["09919551",{"2":{"106":1}}],["099906564",{"2":{"106":1}}],["09926938",{"2":{"106":1}}],["0973",{"2":{"249":1}}],["0971",{"2":{"249":1}}],["097182155",{"2":{"106":1}}],["09701932",{"2":{"179":1}}],["097253",{"2":{"126":1}}],["09724942",{"2":{"106":1}}],["09724939",{"2":{"106":1}}],["097701035",{"2":{"106":2}}],["09785451",{"2":{"77":1}}],["09041",{"2":{"187":1}}],["0903271",{"2":{"106":2}}],["0901",{"2":{"78":1}}],["09098f",{"2":{"77":1}}],["0900306",{"2":{"53":10}}],["078306823363043e",{"2":{"258":1}}],["0782",{"2":{"249":1}}],["0787",{"2":{"249":1}}],["07874016",{"2":{"232":2}}],["07892",{"2":{"187":1}}],["076645e",{"2":{"208":1}}],["076404385",{"2":{"106":1}}],["07640441",{"2":{"106":1}}],["073286f",{"2":{"256":1}}],["073344233711706e",{"2":{"258":1}}],["0733",{"2":{"249":1}}],["07375115",{"2":{"235":1}}],["07302",{"2":{"187":1}}],["073442355",{"2":{"77":1}}],["072849633268878e",{"2":{"258":1}}],["072823511703495e",{"2":{"258":1}}],["072864f",{"2":{"256":1}}],["072868f",{"2":{"256":1}}],["0728675615927385",{"2":{"53":1}}],["07276",{"2":{"187":1}}],["07565363",{"2":{"235":1}}],["07520",{"2":{"187":1}}],["075135306",{"2":{"147":1}}],["07708",{"2":{"187":1}}],["07702",{"2":{"171":1}}],["0770206",{"2":{"171":2}}],["07704306",{"2":{"77":1}}],["07938",{"2":{"187":1}}],["079208925",{"2":{"124":1}}],["07964921",{"2":{"107":1}}],["074235209701761e",{"2":{"258":1}}],["0742047",{"2":{"232":1}}],["07425846",{"2":{"106":1}}],["07425845",{"2":{"106":1}}],["07477",{"2":{"187":1}}],["07412187",{"2":{"77":1}}],["070823f",{"2":{"256":1}}],["070843f",{"2":{"256":1}}],["0709",{"2":{"232":1}}],["07027",{"2":{"187":1}}],["07038",{"2":{"187":1}}],["0701",{"2":{"78":1}}],["07058401",{"2":{"77":1}}],["07057328",{"2":{"77":1}}],["0714210104147525e",{"2":{"258":1}}],["071482725",{"2":{"179":1}}],["07133968",{"2":{"109":1}}],["07167f",{"2":{"77":1}}],["07125676",{"2":{"77":1}}],["04",{"2":{"228":2}}],["044373336409806e",{"2":{"258":1}}],["0441475f",{"2":{"256":1}}],["04416",{"2":{"236":1}}],["044077f",{"2":{"256":1}}],["04409",{"2":{"187":1}}],["0442053f",{"2":{"256":1}}],["0444",{"2":{"249":1}}],["04444s\\ttraining",{"2":{"228":1}}],["04s",{"2":{"196":1}}],["041618678380332e",{"2":{"258":1}}],["041207f",{"2":{"256":1}}],["04124",{"2":{"187":1}}],["041541956",{"2":{"244":1}}],["04117",{"2":{"187":1}}],["04199602",{"2":{"179":1}}],["041031003",{"2":{"145":1}}],["04108f",{"2":{"77":1}}],["0492674f",{"2":{"256":1}}],["04972s\\ttraining",{"2":{"228":1}}],["04992",{"2":{"187":1}}],["0499427f",{"2":{"256":1}}],["04994",{"2":{"187":1}}],["04903387",{"2":{"179":1}}],["049417756",{"2":{"106":1}}],["049417753",{"2":{"106":1}}],["0437818246196376e",{"2":{"258":1}}],["043837259072836e",{"2":{"258":1}}],["043883",{"2":{"171":1}}],["043909705477776e",{"2":{"258":1}}],["0434169508857686e",{"2":{"258":1}}],["0434613f",{"2":{"256":1}}],["043639f",{"2":{"256":1}}],["04360",{"2":{"187":1}}],["0435",{"2":{"249":1}}],["0435721",{"2":{"77":1}}],["0469301",{"2":{"148":1}}],["0462954",{"2":{"148":1}}],["040918160",{"2":{"244":1}}],["04058",{"2":{"187":2}}],["04001",{"2":{"187":1}}],["04008",{"2":{"187":1}}],["0402095",{"2":{"147":1}}],["0401",{"2":{"78":1}}],["04019f",{"2":{"77":1}}],["0454979281233715e",{"2":{"258":1}}],["045491105306053e",{"2":{"258":1}}],["0454488f",{"2":{"256":1}}],["045295f",{"2":{"256":1}}],["04517s\\ttraining",{"2":{"228":1}}],["04513",{"2":{"187":1}}],["04558s\\ttraining",{"2":{"228":1}}],["04551310",{"2":{"107":1}}],["04576",{"2":{"187":1}}],["04565",{"2":{"187":1}}],["04566476",{"2":{"106":1}}],["0476384594093793e",{"2":{"258":1}}],["0477504f",{"2":{"256":1}}],["047244094",{"2":{"232":2}}],["047292e",{"2":{"208":1}}],["04751",{"2":{"187":1}}],["047897488",{"2":{"106":2}}],["047356773",{"2":{"77":1}}],["04821",{"2":{"187":1}}],["048109",{"2":{"148":1}}],["048133414",{"2":{"106":2}}],["048522998",{"2":{"129":1}}],["04853325",{"2":{"106":1}}],["048533253",{"2":{"106":1}}],["048811335",{"2":{"106":2}}],["0480501",{"2":{"77":1}}],["04d",{"2":{"78":1}}],["0421841f",{"2":{"256":1}}],["042499166",{"2":{"244":1}}],["0428307",{"2":{"232":1}}],["04291",{"2":{"187":1}}],["0425071290386567e",{"2":{"258":1}}],["04256",{"2":{"187":1}}],["0425432",{"2":{"147":1}}],["04265762",{"2":{"179":1}}],["0420834",{"2":{"77":1}}],["042254835",{"2":{"106":1}}],["04225484",{"2":{"106":1}}],["042256515",{"2":{"106":1}}],["04224006",{"2":{"77":1}}],["04226003",{"2":{"77":1}}],["068789240413099e",{"2":{"258":1}}],["068646f",{"2":{"256":1}}],["0686756",{"2":{"232":1}}],["0612678763170755e",{"2":{"258":1}}],["0618",{"2":{"249":1}}],["0613257f",{"2":{"256":1}}],["0613",{"2":{"249":1}}],["069982546962557e",{"2":{"258":1}}],["0690823",{"2":{"235":1}}],["06942138",{"2":{"77":1}}],["06433817",{"2":{"232":1}}],["06450",{"2":{"19":1}}],["0672",{"2":{"249":1}}],["067601e",{"2":{"208":1}}],["06707",{"2":{"187":1}}],["067872316",{"2":{"77":1}}],["062907",{"2":{"236":1}}],["06286",{"2":{"187":1}}],["062155924699545",{"2":{"153":1}}],["0621559246995447",{"2":{"153":1}}],["062079933",{"2":{"147":1}}],["066539f",{"2":{"256":1}}],["06629",{"2":{"187":1}}],["06669",{"2":{"131":2}}],["066128",{"2":{"148":1}}],["06612328",{"2":{"107":1}}],["06618893",{"2":{"106":1}}],["06675106",{"2":{"106":1}}],["066751055",{"2":{"106":1}}],["063478545881434e",{"2":{"258":1}}],["063480526",{"2":{"77":1}}],["0631555f",{"2":{"256":1}}],["0635576f",{"2":{"256":1}}],["06350s\\ttraining",{"2":{"228":1}}],["0633",{"2":{"249":1}}],["06301335",{"2":{"106":1}}],["06325f",{"2":{"77":1}}],["06083",{"2":{"187":1}}],["06027697",{"2":{"106":1}}],["06026962",{"2":{"77":1}}],["0601",{"2":{"78":1}}],["060364496",{"2":{"77":1}}],["06053336",{"2":{"77":1}}],["065810589678892e",{"2":{"258":1}}],["065926520404789e",{"2":{"258":1}}],["065984f",{"2":{"256":1}}],["0656676f",{"2":{"256":1}}],["06521",{"2":{"187":1}}],["06519687",{"2":{"107":1}}],["06550143",{"2":{"77":1}}],["065317236",{"2":{"77":1}}],["06",{"2":{"53":1,"196":2}}],["0514342765836275e",{"2":{"258":1}}],["05132087680134e",{"2":{"258":1}}],["0516537f",{"2":{"256":1}}],["051517f",{"2":{"256":1}}],["051093874657169e",{"2":{"258":1}}],["0510",{"2":{"249":1}}],["0546",{"2":{"249":1}}],["0540",{"2":{"249":1}}],["0540537",{"2":{"77":1}}],["05f0",{"2":{"244":2,"245":1}}],["0552927789685223e",{"2":{"258":1}}],["05522",{"2":{"187":1}}],["0554815f",{"2":{"256":1}}],["0551181",{"2":{"232":2}}],["05537s\\ttraining",{"2":{"228":1}}],["05230",{"2":{"187":1}}],["05267",{"2":{"187":1}}],["05218964",{"2":{"179":1}}],["0584",{"2":{"249":1}}],["05812",{"2":{"187":1}}],["0585f",{"2":{"77":1}}],["0590s\\ttraining",{"2":{"217":1}}],["05903",{"2":{"187":1}}],["05999",{"2":{"187":1}}],["05914",{"2":{"187":1}}],["059518166",{"2":{"106":2}}],["0578545",{"2":{"235":1}}],["05738",{"2":{"187":1}}],["05727",{"2":{"187":2}}],["05729505",{"2":{"106":1}}],["057005208",{"2":{"177":1,"178":1}}],["05779423",{"2":{"126":1}}],["0534",{"2":{"249":1}}],["0536",{"2":{"249":1}}],["053627778",{"2":{"106":1}}],["05375",{"2":{"187":1}}],["053747915",{"2":{"106":2}}],["0568",{"2":{"249":1}}],["0569",{"2":{"249":1}}],["05626",{"2":{"187":1}}],["05657739601024536",{"2":{"174":1}}],["0564903986057956",{"2":{"153":2}}],["056",{"2":{"78":3,"256":1}}],["050299558748259e",{"2":{"258":1}}],["0508744f",{"2":{"256":1}}],["050110792785467e",{"2":{"258":1}}],["0501",{"2":{"78":1}}],["05058f",{"2":{"77":1}}],["050089",{"2":{"77":1}}],["05",{"2":{"53":1,"249":1}}],["00",{"2":{"196":12,"228":75}}],["008724037",{"2":{"244":1}}],["008772802",{"2":{"244":1}}],["008732248",{"2":{"244":1}}],["008925664",{"2":{"244":1}}],["008984680549249e",{"2":{"258":1}}],["00898",{"2":{"187":1}}],["008556721",{"2":{"244":1}}],["00852",{"2":{"187":3}}],["008153505",{"2":{"244":1}}],["008185850",{"2":{"244":1}}],["00812",{"2":{"187":1}}],["00819",{"2":{"187":1}}],["00833",{"2":{"187":1}}],["00832",{"2":{"187":1}}],["008405063",{"2":{"244":1}}],["008404830",{"2":{"244":1}}],["00840",{"2":{"187":1}}],["008026988",{"2":{"244":1}}],["008074892",{"2":{"244":1}}],["00803",{"2":{"187":1}}],["00809",{"2":{"187":1}}],["008014920",{"2":{"244":1}}],["00801",{"2":{"187":1}}],["008808360",{"2":{"244":1}}],["00880",{"2":{"187":1}}],["00886",{"2":{"187":1}}],["008870317",{"2":{"77":1}}],["00823",{"2":{"187":1}}],["00823197",{"2":{"171":1}}],["00824",{"2":{"187":1}}],["00862",{"2":{"187":2}}],["007075997",{"2":{"244":1}}],["00706411",{"2":{"77":1}}],["007615f",{"2":{"256":1}}],["007676640",{"2":{"244":1}}],["007679918",{"2":{"244":1}}],["00766",{"2":{"187":1}}],["007501942",{"2":{"244":1}}],["007509941",{"2":{"244":1}}],["007593059",{"2":{"244":1}}],["007535893",{"2":{"244":1}}],["00755",{"2":{"187":1}}],["007885863",{"2":{"244":1}}],["00786",{"2":{"187":1}}],["007947534",{"2":{"244":1}}],["007950513",{"2":{"244":1}}],["007905648",{"2":{"244":1}}],["007984845",{"2":{"147":1}}],["007187600",{"2":{"244":1}}],["007129070",{"2":{"244":1}}],["00710",{"2":{"187":1}}],["00713",{"2":{"187":1}}],["007387250702925e",{"2":{"258":1}}],["007325693",{"2":{"244":1}}],["007372384",{"2":{"244":1}}],["007333768",{"2":{"244":1}}],["007363341",{"2":{"244":1}}],["00731",{"2":{"187":1}}],["0073489705",{"2":{"106":2}}],["00778",{"2":{"187":1}}],["00774",{"2":{"187":2}}],["00776",{"2":{"187":1}}],["00775",{"2":{"187":1}}],["007220342",{"2":{"244":1}}],["007273634",{"2":{"244":1}}],["00726",{"2":{"187":1}}],["00720",{"2":{"187":1}}],["007249862",{"2":{"106":1}}],["0072498624",{"2":{"106":1}}],["009895653",{"2":{"244":1}}],["009887908",{"2":{"244":1}}],["009693242",{"2":{"244":1}}],["009083126",{"2":{"244":1}}],["009084041f0",{"2":{"53":1}}],["00909",{"2":{"187":1}}],["00902",{"2":{"187":1}}],["00903",{"2":{"187":1}}],["009766437",{"2":{"232":1}}],["00978",{"2":{"187":1}}],["00977",{"2":{"187":1}}],["00974",{"2":{"187":1}}],["00970",{"2":{"187":1}}],["00973899",{"2":{"131":2}}],["00945",{"2":{"187":1}}],["009313912",{"2":{"244":1}}],["00930",{"2":{"187":1}}],["00933",{"2":{"187":1}}],["009155f",{"2":{"256":1}}],["009196416",{"2":{"244":1}}],["00912",{"2":{"187":1}}],["00918",{"2":{"187":1}}],["00914141",{"2":{"179":1}}],["009987052",{"2":{"244":1}}],["00995",{"2":{"187":2}}],["00994",{"2":{"187":1}}],["009963844931984",{"2":{"153":2}}],["00924",{"2":{"187":1}}],["009256126",{"2":{"244":1}}],["00925",{"2":{"187":1}}],["00923",{"2":{"187":1}}],["00926",{"2":{"187":1}}],["009514628",{"2":{"244":1}}],["009516605",{"2":{"244":1}}],["00957",{"2":{"187":1}}],["009578831",{"2":{"77":1}}],["00950",{"2":{"187":2}}],["00956",{"2":{"187":1}}],["005956927589448e",{"2":{"258":1}}],["005978104",{"2":{"244":1}}],["0057638f",{"2":{"256":1}}],["00571707im",{"2":{"168":1}}],["005315070",{"2":{"244":1}}],["005351806",{"2":{"244":1}}],["0053856913",{"2":{"77":1}}],["005200472",{"2":{"244":1}}],["005807751",{"2":{"244":1}}],["005835793",{"2":{"244":1}}],["005824474",{"2":{"244":1}}],["005463609",{"2":{"244":1}}],["005461216",{"2":{"244":1}}],["005433898",{"2":{"244":1}}],["005112350",{"2":{"244":1}}],["005f0",{"2":{"244":1}}],["005",{"2":{"208":1}}],["005674531032161e",{"2":{"258":1}}],["005628760",{"2":{"244":1}}],["005615299",{"2":{"244":1}}],["00560",{"2":{"187":1}}],["00564f",{"2":{"77":1}}],["005065940",{"2":{"244":1}}],["005044522",{"2":{"244":1}}],["005014097",{"2":{"77":1}}],["005000000000000009",{"2":{"53":1}}],["0060156f",{"2":{"256":1}}],["006616294",{"2":{"244":1}}],["006642802",{"2":{"244":1}}],["0066712513",{"2":{"179":1}}],["006305961",{"2":{"244":1}}],["006221022",{"2":{"244":1}}],["006288857",{"2":{"106":1}}],["0062888456",{"2":{"106":1}}],["006572613",{"2":{"244":1}}],["0065859724",{"2":{"77":1}}],["006942283",{"2":{"244":1}}],["006962581",{"2":{"244":1}}],["006906096",{"2":{"244":1}}],["0069942693",{"2":{"106":1}}],["006432486",{"2":{"77":1}}],["006105572",{"2":{"244":1}}],["0061099795",{"2":{"77":1}}],["006184267",{"2":{"77":1}}],["006736353",{"2":{"77":1}}],["0067729075",{"2":{"77":1}}],["004855067",{"2":{"244":1}}],["004880759",{"2":{"244":1}}],["00488f",{"2":{"77":1}}],["004699f",{"2":{"256":1}}],["004657542",{"2":{"244":1}}],["004607514",{"2":{"244":1}}],["004590358",{"2":{"244":1}}],["004518208",{"2":{"244":1}}],["004542338",{"2":{"244":1}}],["004978007",{"2":{"244":1}}],["004947479",{"2":{"244":1}}],["004958895",{"2":{"244":1}}],["004958282",{"2":{"244":1}}],["004790282",{"2":{"244":1}}],["004397592",{"2":{"244":1}}],["004335414",{"2":{"244":1}}],["004338491",{"2":{"244":1}}],["004365115",{"2":{"244":1}}],["004326937",{"2":{"244":1}}],["004329988",{"2":{"244":1}}],["004287162261593e",{"2":{"258":1}}],["004287243",{"2":{"106":2}}],["004253434",{"2":{"244":1}}],["004434146",{"2":{"244":1}}],["004466736",{"2":{"244":1}}],["004491198",{"2":{"244":1}}],["0044189203",{"2":{"77":1}}],["004043682",{"2":{"244":1}}],["004047669",{"2":{"77":1}}],["004061943",{"2":{"244":1}}],["00407581",{"2":{"179":1}}],["0041787219358996e",{"2":{"258":1}}],["004119421",{"2":{"244":1}}],["004116789",{"2":{"244":1}}],["0041674743",{"2":{"106":1}}],["0041674753",{"2":{"106":1}}],["004169049",{"2":{"77":1}}],["004144425",{"2":{"77":1}}],["001398915",{"2":{"244":1}}],["00139126",{"2":{"77":1}}],["001329915",{"2":{"244":1}}],["001325711",{"2":{"244":1}}],["001325290",{"2":{"244":1}}],["001367094",{"2":{"244":1}}],["001362987",{"2":{"244":1}}],["001362466",{"2":{"244":1}}],["001364744",{"2":{"244":1}}],["001380907",{"2":{"244":1}}],["001386037",{"2":{"244":1}}],["001346401",{"2":{"244":1}}],["001356683",{"2":{"244":1}}],["001336156",{"2":{"244":1}}],["001307026",{"2":{"244":1}}],["001303701",{"2":{"244":1}}],["001302390",{"2":{"244":1}}],["001302087",{"2":{"244":1}}],["001300987",{"2":{"244":1}}],["001309370",{"2":{"244":1}}],["001305370",{"2":{"244":1}}],["001636085",{"2":{"244":1}}],["001639404",{"2":{"244":1}}],["001659834",{"2":{"244":1}}],["001658659",{"2":{"244":1}}],["001672024",{"2":{"244":1}}],["001671217",{"2":{"244":1}}],["001649067",{"2":{"244":1}}],["001608545",{"2":{"244":1}}],["001694928",{"2":{"244":1}}],["001690981",{"2":{"244":1}}],["001615586",{"2":{"244":1}}],["001611185",{"2":{"244":1}}],["0015762f",{"2":{"256":1}}],["001571818",{"2":{"244":1}}],["001592739",{"2":{"244":1}}],["001594031",{"2":{"244":1}}],["001531350",{"2":{"244":1}}],["0015313211",{"2":{"77":1}}],["001541014",{"2":{"244":1}}],["001513747",{"2":{"244":1}}],["001510418",{"2":{"244":1}}],["001517879",{"2":{"244":1}}],["001560361",{"2":{"244":1}}],["001566568",{"2":{"244":1}}],["001564218",{"2":{"244":1}}],["001",{"2":{"208":1}}],["001093546",{"2":{"244":1}}],["001014936",{"2":{"244":1}}],["001011056",{"2":{"244":1}}],["00101147",{"2":{"179":1}}],["001068081",{"2":{"244":1}}],["001060384",{"2":{"244":1}}],["001060399",{"2":{"244":1}}],["00106574",{"2":{"77":1}}],["001085206",{"2":{"244":1}}],["001085730",{"2":{"244":1}}],["001085884",{"2":{"244":1}}],["001089040",{"2":{"244":1}}],["001008533",{"2":{"244":1}}],["001009728",{"2":{"244":1}}],["001049175",{"2":{"244":1}}],["001044580",{"2":{"244":1}}],["001041825",{"2":{"244":1}}],["001047856",{"2":{"244":1}}],["001053359",{"2":{"244":1}}],["001050717",{"2":{"244":1}}],["001058994",{"2":{"244":1}}],["001059842",{"2":{"244":1}}],["0010350421",{"2":{"232":1}}],["001f0",{"2":{"78":1,"107":1,"217":1,"228":1}}],["0012",{"2":{"249":1}}],["001202180",{"2":{"244":1}}],["001209691",{"2":{"244":1}}],["001203537",{"2":{"244":1}}],["001257007",{"2":{"244":1}}],["001258897",{"2":{"244":1}}],["001272172",{"2":{"244":1}}],["001279382",{"2":{"244":1}}],["001228882",{"2":{"244":1}}],["001224264",{"2":{"244":1}}],["001240770",{"2":{"244":1}}],["001243982",{"2":{"244":1}}],["001243673",{"2":{"244":1}}],["001241080",{"2":{"244":1}}],["001245214",{"2":{"244":1}}],["001218763",{"2":{"244":1}}],["001215764",{"2":{"244":1}}],["001292084",{"2":{"244":1}}],["00126273",{"2":{"77":1}}],["001289588",{"2":{"244":1}}],["00128162",{"2":{"77":1}}],["0012843215",{"2":{"77":1}}],["001886566",{"2":{"244":1}}],["001884541",{"2":{"244":1}}],["001864895",{"2":{"244":1}}],["001825325",{"2":{"244":1}}],["001837625",{"2":{"244":1}}],["001894138",{"2":{"244":1}}],["001845157",{"2":{"244":1}}],["001849154",{"2":{"244":1}}],["001848093",{"2":{"244":1}}],["001879478",{"2":{"244":1}}],["001871931",{"2":{"244":1}}],["00187748",{"2":{"77":1}}],["0018058",{"2":{"77":1}}],["0018510937",{"2":{"77":1}}],["001448896",{"2":{"244":1}}],["001448493",{"2":{"244":1}}],["001439294",{"2":{"244":1}}],["001479216",{"2":{"244":1}}],["001498525",{"2":{"244":1}}],["001494603",{"2":{"244":1}}],["001492234",{"2":{"244":1}}],["001485110",{"2":{"244":1}}],["001454655",{"2":{"244":1}}],["00145586",{"2":{"77":1}}],["001404967",{"2":{"244":1}}],["001409405",{"2":{"244":1}}],["001402911",{"2":{"244":1}}],["00140763051",{"2":{"78":1}}],["0014175134",{"2":{"77":1}}],["001185708",{"2":{"244":1}}],["001182971",{"2":{"244":1}}],["001168622",{"2":{"244":1}}],["001160439",{"2":{"244":1}}],["001101400",{"2":{"244":1}}],["001103305",{"2":{"244":1}}],["001110448",{"2":{"244":1}}],["001113559",{"2":{"244":1}}],["001141232",{"2":{"244":1}}],["001143823",{"2":{"244":1}}],["001148389",{"2":{"244":1}}],["001171167",{"2":{"244":1}}],["001179150",{"2":{"244":1}}],["00117228",{"2":{"77":1}}],["001126162",{"2":{"244":1}}],["001191893",{"2":{"244":1}}],["001197650",{"2":{"244":1}}],["001136703",{"2":{"244":1}}],["001135292",{"2":{"244":1}}],["001135202",{"2":{"244":1}}],["001138862",{"2":{"244":1}}],["001153235",{"2":{"244":1}}],["001154157",{"2":{"244":1}}],["00115296",{"2":{"77":1}}],["001720798",{"2":{"244":1}}],["001728887",{"2":{"244":1}}],["001789054",{"2":{"244":1}}],["001770784",{"2":{"244":1}}],["001747428",{"2":{"244":1}}],["001717428",{"2":{"244":1}}],["001714717",{"2":{"244":1}}],["001715113",{"2":{"244":1}}],["001764537",{"2":{"244":1}}],["001731674",{"2":{"244":1}}],["001730468",{"2":{"244":1}}],["0017369908",{"2":{"77":1}}],["001798752",{"2":{"244":1}}],["001798573",{"2":{"244":1}}],["001792779",{"2":{"244":1}}],["0017900232",{"2":{"77":1}}],["00175152",{"2":{"77":1}}],["00170922",{"2":{"77":1}}],["001930058",{"2":{"244":1}}],["001994215",{"2":{"244":1}}],["001959185",{"2":{"244":1}}],["0019504696",{"2":{"77":1}}],["001972227",{"2":{"244":1}}],["001975382",{"2":{"244":1}}],["001970405",{"2":{"244":1}}],["001973281",{"2":{"77":1}}],["001917289057025e",{"2":{"258":1}}],["001919697",{"2":{"244":1}}],["001913586",{"2":{"244":1}}],["0019153744",{"2":{"77":1}}],["001940841",{"2":{"244":1}}],["001982885",{"2":{"244":1}}],["00198415",{"2":{"179":1}}],["00198767",{"2":{"77":1}}],["0019262917",{"2":{"77":1}}],["002989073",{"2":{"244":1}}],["002972272",{"2":{"244":1}}],["002935605",{"2":{"244":1}}],["002511334",{"2":{"244":1}}],["002511769",{"2":{"244":1}}],["002553018",{"2":{"244":1}}],["002556076",{"2":{"244":1}}],["002343122452919e",{"2":{"258":1}}],["002343247",{"2":{"244":1}}],["002371583",{"2":{"244":1}}],["002318957",{"2":{"244":1}}],["002301548",{"2":{"244":1}}],["002303589",{"2":{"244":1}}],["002304791",{"2":{"244":1}}],["002395016",{"2":{"244":1}}],["002358936",{"2":{"244":1}}],["002834306",{"2":{"244":1}}],["002806861",{"2":{"244":1}}],["002853032",{"2":{"244":1}}],["0028297466",{"2":{"106":1}}],["0028297475",{"2":{"106":1}}],["002670718",{"2":{"244":1}}],["002677995",{"2":{"244":1}}],["002668741",{"2":{"244":1}}],["002668936",{"2":{"244":1}}],["002606506",{"2":{"244":1}}],["002613829",{"2":{"244":1}}],["002696904",{"2":{"244":1}}],["002695386",{"2":{"244":1}}],["0026271285",{"2":{"110":1}}],["002236427",{"2":{"244":1}}],["002234935",{"2":{"244":1}}],["0022308934",{"2":{"77":1}}],["002276814",{"2":{"244":1}}],["002210422",{"2":{"244":1}}],["002218451",{"2":{"244":1}}],["002247394",{"2":{"244":1}}],["0022497",{"2":{"232":1}}],["002262217",{"2":{"244":1}}],["002201537",{"2":{"244":1}}],["002071f",{"2":{"256":1}}],["002068657",{"2":{"244":1}}],["002089650",{"2":{"244":1}}],["002081697",{"2":{"244":1}}],["002059042",{"2":{"244":1}}],["0020579377",{"2":{"106":1}}],["002057937",{"2":{"106":1}}],["002023818",{"2":{"244":1}}],["002015869",{"2":{"244":1}}],["002019164",{"2":{"244":1}}],["002037087",{"2":{"244":1}}],["00209504",{"2":{"77":1}}],["002752088",{"2":{"244":1}}],["00270753",{"2":{"77":1}}],["0027386106",{"2":{"77":1}}],["0027955552",{"2":{"77":1}}],["002155375",{"2":{"244":1}}],["002129085",{"2":{"244":1}}],["002125610",{"2":{"244":1}}],["002123826",{"2":{"244":1}}],["0021203319",{"2":{"77":1}}],["00219681",{"2":{"77":1}}],["00214946",{"2":{"77":1}}],["002106787",{"2":{"77":1}}],["002417301",{"2":{"244":1}}],["002489866",{"2":{"244":1}}],["002474443",{"2":{"244":1}}],["002448601",{"2":{"244":1}}],["0024971329",{"2":{"77":1}}],["0024323382",{"2":{"77":1}}],["0032324314120804e",{"2":{"258":1}}],["003299790",{"2":{"244":1}}],["003847855",{"2":{"244":1}}],["003813077",{"2":{"244":1}}],["0038745",{"2":{"77":1}}],["003463188",{"2":{"244":1}}],["003488734",{"2":{"244":1}}],["0034101289",{"2":{"77":1}}],["003329928",{"2":{"244":1}}],["0031484f",{"2":{"256":1}}],["003180855",{"2":{"244":1}}],["003136263",{"2":{"244":1}}],["003132161",{"2":{"244":1}}],["0031376407",{"2":{"77":1}}],["003162773",{"2":{"244":1}}],["003125851",{"2":{"244":1}}],["003911218",{"2":{"244":1}}],["003920683",{"2":{"244":1}}],["00390526722",{"2":{"78":1}}],["003504427",{"2":{"244":1}}],["003519468",{"2":{"244":1}}],["00351666",{"2":{"77":1}}],["003553676",{"2":{"244":1}}],["003585588",{"2":{"244":1}}],["003658900",{"2":{"244":1}}],["003630795",{"2":{"244":1}}],["0036810287",{"2":{"106":1}}],["0036810301",{"2":{"106":1}}],["00364828",{"2":{"77":1}}],["003095849",{"2":{"244":1}}],["003031955",{"2":{"244":1}}],["003061282",{"2":{"244":1}}],["00306169",{"2":{"77":1}}],["003071612",{"2":{"244":1}}],["003027094",{"2":{"77":1}}],["0037725805",{"2":{"77":1}}],["003750000000000005",{"2":{"53":1}}],["000",{"2":{"250":1}}],["0008565912204912905",{"2":{"258":1}}],["000857553",{"2":{"244":1}}],["000846881",{"2":{"244":1}}],["000847362",{"2":{"244":1}}],["0008246203059352311",{"2":{"258":1}}],["0008249927572664053",{"2":{"258":1}}],["000824789",{"2":{"244":1}}],["000826713",{"2":{"244":1}}],["000826687",{"2":{"244":1}}],["0008646959428964223",{"2":{"258":1}}],["000864926",{"2":{"244":1}}],["000868335",{"2":{"244":1}}],["000866074",{"2":{"244":1}}],["000835373",{"2":{"244":1}}],["000835131",{"2":{"244":1}}],["000831873",{"2":{"244":1}}],["000887047",{"2":{"244":1}}],["000883626",{"2":{"244":1}}],["000881248",{"2":{"244":1}}],["000873007",{"2":{"244":1}}],["000871024327",{"2":{"78":1}}],["000892342",{"2":{"244":1}}],["000892691",{"2":{"244":1}}],["000890729",{"2":{"244":1}}],["000893347",{"2":{"244":1}}],["000894285",{"2":{"244":1}}],["000809421",{"2":{"244":1}}],["000805100",{"2":{"244":1}}],["000807529",{"2":{"244":1}}],["0008191210063249355",{"2":{"258":1}}],["0008172507531907208",{"2":{"258":1}}],["000816396",{"2":{"244":1}}],["000814481",{"2":{"244":1}}],["000043311",{"2":{"244":1}}],["000046194",{"2":{"244":1}}],["000044412",{"2":{"244":1}}],["000099929125259e",{"2":{"258":1}}],["000098905",{"2":{"244":1}}],["000098134",{"2":{"244":1}}],["000092742",{"2":{"244":1}}],["000094398",{"2":{"244":1}}],["000097348",{"2":{"244":1}}],["000097578",{"2":{"244":1}}],["000097049",{"2":{"244":1}}],["000095254",{"2":{"244":1}}],["0000508f",{"2":{"256":1}}],["000058640",{"2":{"244":1}}],["000056836",{"2":{"244":1}}],["000059547",{"2":{"244":1}}],["000052356",{"2":{"244":1}}],["000067247",{"2":{"244":1}}],["000088090",{"2":{"244":1}}],["000086152",{"2":{"244":1}}],["000089840",{"2":{"244":1}}],["000083818",{"2":{"244":1}}],["000083883",{"2":{"244":1}}],["000085040",{"2":{"244":1}}],["000081115",{"2":{"244":1}}],["000087234",{"2":{"244":1}}],["000079111",{"2":{"244":1}}],["000070152",{"2":{"244":1}}],["000075086",{"2":{"244":1}}],["000077215",{"2":{"244":1}}],["000071670",{"2":{"244":1}}],["00000",{"2":{"187":100,"217":18,"219":3}}],["0002478710205721238",{"2":{"258":1}}],["00024047601132138021",{"2":{"258":1}}],["0002402046544920797",{"2":{"258":1}}],["00024020273",{"2":{"256":1}}],["00024900811024501776",{"2":{"258":1}}],["0002493166046591676",{"2":{"258":1}}],["0002493159",{"2":{"256":1}}],["00024803920225336914",{"2":{"258":1}}],["00024803935",{"2":{"256":1}}],["0002444638630047816",{"2":{"258":1}}],["00024446193",{"2":{"256":1}}],["00024311330118210217",{"2":{"258":1}}],["00024311207",{"2":{"256":1}}],["00024368877005781378",{"2":{"258":1}}],["00024368793",{"2":{"256":1}}],["00024250686729200373",{"2":{"258":1}}],["00024251099",{"2":{"256":1}}],["000242183",{"2":{"244":1}}],["000242905",{"2":{"244":1}}],["000242692",{"2":{"244":1}}],["00022585658063969594",{"2":{"258":1}}],["00022747977289500564",{"2":{"258":1}}],["00022747654",{"2":{"256":1}}],["00022024982202853847",{"2":{"258":1}}],["00022025035",{"2":{"256":1}}],["00022943318",{"2":{"256":1}}],["000229884",{"2":{"244":1}}],["0002289486765587484",{"2":{"258":1}}],["0002289462",{"2":{"256":1}}],["00022895353047546222",{"2":{"258":1}}],["00022895339",{"2":{"256":1}}],["00022421298252437347",{"2":{"258":1}}],["00022429263095819746",{"2":{"258":1}}],["00022429347",{"2":{"256":1}}],["00022400460020439108",{"2":{"258":1}}],["00022400402",{"2":{"256":1}}],["00022498223211867276",{"2":{"258":1}}],["00022498281",{"2":{"256":1}}],["000269425354240891",{"2":{"258":1}}],["000269295",{"2":{"244":1}}],["0002605033113944275",{"2":{"258":1}}],["000263541",{"2":{"244":1}}],["000264404",{"2":{"244":1}}],["0002732296908907871",{"2":{"258":1}}],["000273393",{"2":{"244":1}}],["00027652478",{"2":{"256":1}}],["0002716497787107297",{"2":{"258":1}}],["0002716475",{"2":{"256":1}}],["000271556",{"2":{"244":1}}],["000275985",{"2":{"244":1}}],["000277635",{"2":{"244":1}}],["0002702743585450876",{"2":{"258":1}}],["00027027391",{"2":{"256":1}}],["000270049",{"2":{"244":1}}],["000270824",{"2":{"244":1}}],["00020750172390734595",{"2":{"258":1}}],["0002022635925814998",{"2":{"258":1}}],["00020226216",{"2":{"256":1}}],["00020466443684761645",{"2":{"258":1}}],["00020466343",{"2":{"256":1}}],["00020403395",{"2":{"256":1}}],["000204869",{"2":{"244":1}}],["00020523853473720257",{"2":{"258":1}}],["000205284",{"2":{"244":1}}],["00020540271245962234",{"2":{"258":1}}],["00020540101",{"2":{"256":1}}],["00020515204127527292",{"2":{"258":1}}],["00020515101",{"2":{"256":1}}],["00020892571212278687",{"2":{"258":1}}],["000208925",{"2":{"256":1}}],["00020849705",{"2":{"146":1}}],["00020116539401022254",{"2":{"258":1}}],["00020116469",{"2":{"256":1}}],["000201183",{"2":{"244":1}}],["00020963441425483624",{"2":{"258":1}}],["00020963635",{"2":{"256":1}}],["000209925",{"2":{"244":1}}],["00020617064265970482",{"2":{"258":1}}],["00020616876",{"2":{"256":1}}],["00020690380387774357",{"2":{"258":1}}],["00020690112",{"2":{"256":1}}],["000206944",{"2":{"244":1}}],["000206055",{"2":{"244":1}}],["00023791945092090395",{"2":{"258":1}}],["0002378841309108547",{"2":{"258":1}}],["00023788413",{"2":{"256":1}}],["00023533272563963474",{"2":{"258":1}}],["00023533189",{"2":{"256":1}}],["0002387293296309346",{"2":{"258":1}}],["00023872862",{"2":{"256":1}}],["00023213715415931616",{"2":{"258":1}}],["00023213631",{"2":{"256":1}}],["0002324687375221788",{"2":{"258":1}}],["00023246874",{"2":{"256":1}}],["00023605301863994222",{"2":{"258":1}}],["0002360519",{"2":{"256":1}}],["00023694786630550335",{"2":{"258":1}}],["00023694857",{"2":{"256":1}}],["000230181",{"2":{"244":1}}],["00023950750399452675",{"2":{"258":1}}],["000239237",{"2":{"244":1}}],["000239130808",{"2":{"78":1}}],["00025559112170524283",{"2":{"258":1}}],["0002557868714673563",{"2":{"258":1}}],["00025578658",{"2":{"256":1}}],["0002548531",{"2":{"256":1}}],["00025161545",{"2":{"256":1}}],["000250679",{"2":{"244":1}}],["000257633",{"2":{"244":1}}],["0002533831052302163",{"2":{"258":1}}],["0002533797",{"2":{"256":1}}],["000253915",{"2":{"244":1}}],["00025326014",{"2":{"146":1}}],["00028384866901007055",{"2":{"258":1}}],["000288360",{"2":{"244":1}}],["000280093",{"2":{"244":1}}],["000287677",{"2":{"244":1}}],["000289123",{"2":{"244":1}}],["000286281",{"2":{"244":1}}],["000212844829094219",{"2":{"258":1}}],["0002109584224696663",{"2":{"258":1}}],["00021095501",{"2":{"256":1}}],["00021031103658934895",{"2":{"258":1}}],["00021031174",{"2":{"256":1}}],["0002189304385414387",{"2":{"258":1}}],["00021892734",{"2":{"256":1}}],["000218087",{"2":{"244":1}}],["00021563369839539874",{"2":{"258":1}}],["000215632",{"2":{"256":1}}],["00021518579044466895",{"2":{"258":1}}],["00021518409",{"2":{"256":1}}],["00021525445866880696",{"2":{"258":1}}],["00021525362",{"2":{"256":1}}],["0002157027790765166",{"2":{"258":1}}],["0002157011",{"2":{"256":1}}],["000215771",{"2":{"244":1}}],["00021138838317157782",{"2":{"258":1}}],["00021139008",{"2":{"256":1}}],["00021126755692494242",{"2":{"258":1}}],["00021126727",{"2":{"256":1}}],["000211123",{"2":{"244":1}}],["00021786714100393905",{"2":{"258":1}}],["00021786611",{"2":{"256":1}}],["000217459",{"2":{"244":1}}],["000213564",{"2":{"244":1}}],["000213855",{"2":{"244":1}}],["000216420",{"2":{"244":1}}],["000214095",{"2":{"244":1}}],["000214067404",{"2":{"78":1}}],["00029132406089327705",{"2":{"258":1}}],["00029132818",{"2":{"256":1}}],["000291289",{"2":{"244":1}}],["000298148",{"2":{"244":1}}],["000296087",{"2":{"244":1}}],["00029203",{"2":{"179":1}}],["000410463",{"2":{"244":1}}],["000412207",{"2":{"244":1}}],["000412151",{"2":{"244":1}}],["000412629",{"2":{"244":1}}],["00045962503873739547",{"2":{"258":1}}],["000454265",{"2":{"244":1}}],["000454922556",{"2":{"78":1}}],["000455096",{"2":{"244":1}}],["000405329",{"2":{"244":1}}],["000401171",{"2":{"244":1}}],["000408040",{"2":{"244":1}}],["000490567",{"2":{"244":1}}],["000485339",{"2":{"244":1}}],["000489585",{"2":{"244":1}}],["000486189",{"2":{"244":1}}],["000482833",{"2":{"244":1}}],["000482373",{"2":{"244":1}}],["000484853",{"2":{"244":1}}],["0004200473523943576",{"2":{"258":1}}],["000420685",{"2":{"244":1}}],["000421408",{"2":{"244":1}}],["000421422",{"2":{"244":1}}],["000426714",{"2":{"244":1}}],["000426463",{"2":{"244":1}}],["000425117",{"2":{"244":1}}],["000422994",{"2":{"244":1}}],["0004671387862449773",{"2":{"258":1}}],["000461797",{"2":{"244":1}}],["000461412",{"2":{"244":1}}],["000460401",{"2":{"244":1}}],["00046014786",{"2":{"145":1}}],["000477057",{"2":{"244":1}}],["000474958",{"2":{"244":1}}],["000473843",{"2":{"244":1}}],["000435325",{"2":{"244":1}}],["000432069",{"2":{"244":1}}],["000431564",{"2":{"244":1}}],["000438132",{"2":{"77":1}}],["000444956728781826",{"2":{"258":1}}],["000444564",{"2":{"244":1}}],["000449657",{"2":{"244":1}}],["000445312",{"2":{"244":1}}],["000448536",{"2":{"244":1}}],["000447986",{"2":{"244":1}}],["000442930",{"2":{"244":1}}],["000441076729",{"2":{"78":1}}],["0006220253485736737",{"2":{"258":1}}],["0006224015980308057",{"2":{"258":1}}],["000622923",{"2":{"244":1}}],["000620959",{"2":{"244":1}}],["000624455",{"2":{"244":1}}],["000603749788953262",{"2":{"258":1}}],["000600817",{"2":{"244":1}}],["000609521",{"2":{"77":1}}],["0006720663781192687",{"2":{"258":1}}],["000674710999483002",{"2":{"258":1}}],["000673423",{"2":{"244":1}}],["000675569",{"2":{"244":1}}],["0006965721831938202",{"2":{"258":1}}],["000695536447742295",{"2":{"258":1}}],["0006929477876594768",{"2":{"258":1}}],["000691815",{"2":{"244":1}}],["000694873",{"2":{"244":1}}],["000699265",{"2":{"77":1}}],["000635906",{"2":{"244":1}}],["000637475",{"2":{"244":1}}],["0006472975633747372",{"2":{"258":1}}],["000641108",{"2":{"244":1}}],["000645338",{"2":{"244":1}}],["000649345",{"2":{"244":1}}],["000640270",{"2":{"244":1}}],["000648518",{"2":{"244":1}}],["000652327",{"2":{"244":1}}],["000652603",{"2":{"244":1}}],["000650020",{"2":{"244":1}}],["000659882",{"2":{"244":1}}],["000664698",{"2":{"244":1}}],["000667115",{"2":{"244":1}}],["000667028",{"2":{"77":1}}],["000666251",{"2":{"244":1}}],["000666125",{"2":{"244":1}}],["0006172854121024927",{"2":{"258":1}}],["000613936",{"2":{"244":1}}],["000612554",{"2":{"244":1}}],["000612726",{"2":{"77":1}}],["000619220",{"2":{"244":1}}],["000618138",{"2":{"244":1}}],["0006836683670708925",{"2":{"258":1}}],["0006811380285880877",{"2":{"258":1}}],["000687853",{"2":{"244":1}}],["000685339",{"2":{"244":1}}],["000685782",{"2":{"244":1}}],["000688489",{"2":{"244":1}}],["00068473816",{"2":{"145":1}}],["000689046",{"2":{"77":1}}],["000382126",{"2":{"244":1}}],["000386392",{"2":{"244":1}}],["000381213",{"2":{"244":1}}],["000389668",{"2":{"244":1}}],["000387762",{"2":{"244":1}}],["00031084107835364813",{"2":{"258":1}}],["0003108386",{"2":{"256":1}}],["000310122",{"2":{"244":1}}],["000314013",{"2":{"244":1}}],["000314716",{"2":{"244":1}}],["000312205",{"2":{"244":1}}],["000317277",{"2":{"244":1}}],["000317301",{"2":{"244":1}}],["000313576",{"2":{"244":1}}],["000311768",{"2":{"244":1}}],["0003600593464662572",{"2":{"258":1}}],["000362759",{"2":{"244":1}}],["000367492",{"2":{"244":1}}],["000367264",{"2":{"244":1}}],["000369106",{"2":{"244":1}}],["000363892",{"2":{"244":1}}],["000368751",{"2":{"244":1}}],["000366831",{"2":{"244":1}}],["000366380",{"2":{"244":1}}],["00036629336",{"2":{"78":1}}],["0003212005468321654",{"2":{"258":1}}],["000321201",{"2":{"256":1}}],["000321921",{"2":{"244":1}}],["000322635",{"2":{"244":1}}],["000324281",{"2":{"244":1}}],["000326662",{"2":{"244":1}}],["000326938",{"2":{"244":1}}],["000323999",{"2":{"244":1}}],["000376061",{"2":{"244":1}}],["0003799759317775172",{"2":{"258":1}}],["000379668",{"2":{"244":1}}],["000379169",{"2":{"244":1}}],["000378937",{"2":{"244":1}}],["000378884",{"2":{"244":1}}],["000374305",{"2":{"244":1}}],["000374875",{"2":{"77":1}}],["000377450",{"2":{"244":1}}],["000372513",{"2":{"244":1}}],["00035561306879658736",{"2":{"258":1}}],["00035844953141623366",{"2":{"258":1}}],["00035845037",{"2":{"256":1}}],["000350503",{"2":{"244":1}}],["000350560",{"2":{"244":1}}],["000353724",{"2":{"244":1}}],["000354412",{"2":{"244":1}}],["000351864",{"2":{"244":1}}],["000359855",{"2":{"244":1}}],["0003091439472017924",{"2":{"258":1}}],["000301691",{"2":{"244":1}}],["000306839",{"2":{"244":1}}],["000306047",{"2":{"244":1}}],["000305183",{"2":{"244":1}}],["000300986",{"2":{"244":1}}],["000300984",{"2":{"244":1}}],["000308547",{"2":{"244":1}}],["000307154",{"2":{"244":2}}],["0003328812460768363",{"2":{"258":1}}],["000335865",{"2":{"244":1}}],["000334467",{"2":{"244":1}}],["000330688",{"2":{"244":1}}],["000330550",{"2":{"244":1}}],["000333685",{"2":{"244":1}}],["00034917132288773224",{"2":{"258":1}}],["000346362",{"2":{"244":1}}],["000348117",{"2":{"244":1}}],["000344165",{"2":{"244":1}}],["000344849",{"2":{"244":1}}],["000345872",{"2":{"244":1}}],["000347842",{"2":{"244":1}}],["000392804",{"2":{"244":1}}],["0003920444",{"2":{"77":1}}],["000395994",{"2":{"244":1}}],["000391877",{"2":{"244":1}}],["000391076",{"2":{"77":1}}],["000559556",{"2":{"244":1}}],["000554417",{"2":{"244":1}}],["000554349",{"2":{"244":1}}],["000554918",{"2":{"77":1}}],["000506370",{"2":{"244":1}}],["000503808",{"2":{"244":1}}],["000545968",{"2":{"244":1}}],["000545245",{"2":{"244":1}}],["000549527",{"2":{"244":1}}],["000540564",{"2":{"244":1}}],["000547043",{"2":{"244":1}}],["000547293",{"2":{"244":1}}],["000543516",{"2":{"244":1}}],["000590442",{"2":{"244":1}}],["0005953459865030687",{"2":{"258":1}}],["000595116",{"2":{"244":1}}],["000595709",{"2":{"244":1}}],["000599332",{"2":{"244":1}}],["0005922087",{"2":{"106":1}}],["00059220614",{"2":{"106":1}}],["000583868",{"2":{"244":1}}],["000583094",{"2":{"244":1}}],["000580873",{"2":{"244":1}}],["000580056",{"2":{"244":1}}],["000580705",{"2":{"77":1}}],["000525872",{"2":{"244":1}}],["000525994",{"2":{"244":1}}],["000522715",{"2":{"244":1}}],["000521560",{"2":{"244":1}}],["000512687",{"2":{"244":1}}],["000511645",{"2":{"244":1}}],["000516831",{"2":{"244":1}}],["000516864",{"2":{"244":1}}],["000510494",{"2":{"244":1}}],["000510198",{"2":{"77":1}}],["0005787889621067207",{"2":{"258":1}}],["000575209",{"2":{"244":1}}],["000579479",{"2":{"244":1}}],["000577084",{"2":{"244":1}}],["000574894",{"2":{"77":1}}],["000560744",{"2":{"244":1}}],["000566683",{"2":{"244":1}}],["000561465",{"2":{"244":1}}],["000564703",{"2":{"244":1}}],["000563284",{"2":{"77":1}}],["000539345",{"2":{"244":1}}],["000530431",{"2":{"244":1}}],["000537486",{"2":{"244":1}}],["000536315",{"2":{"244":1}}],["000535839",{"2":{"244":1}}],["00053332",{"2":{"179":1}}],["0005f0",{"2":{"244":1}}],["0007669096947228696",{"2":{"258":1}}],["000764329",{"2":{"244":1}}],["0007243385789572986",{"2":{"258":1}}],["0007264717282625864",{"2":{"258":1}}],["000726597",{"2":{"244":1}}],["000723986",{"2":{"244":1}}],["000723805",{"2":{"244":1}}],["0007196817701465205",{"2":{"258":1}}],["000710499",{"2":{"244":1}}],["000714949",{"2":{"244":1}}],["0007467360952999541",{"2":{"258":1}}],["000746255",{"2":{"244":1}}],["000740137",{"2":{"244":1}}],["000744279",{"2":{"244":1}}],["0007867009066537449",{"2":{"258":1}}],["000788119",{"2":{"244":1}}],["000780730",{"2":{"244":1}}],["000784697",{"2":{"244":1}}],["000781053",{"2":{"244":1}}],["0007372375652553526",{"2":{"258":1}}],["000734937",{"2":{"244":1}}],["000732894",{"2":{"244":1}}],["000731462",{"2":{"77":1}}],["000792289",{"2":{"244":1}}],["000792221",{"2":{"244":1}}],["000798627",{"2":{"244":1}}],["000798481",{"2":{"244":1}}],["000798861",{"2":{"244":1}}],["000796866",{"2":{"244":1}}],["000756769",{"2":{"244":1}}],["000754971",{"2":{"244":1}}],["000752965",{"2":{"244":1}}],["000753467",{"2":{"244":1}}],["0007093526384052144",{"2":{"258":1}}],["000709375",{"2":{"244":1}}],["0007080899236081684",{"2":{"257":1}}],["000708074",{"2":{"244":1}}],["000705982",{"2":{"244":1}}],["000707158",{"2":{"244":1}}],["0007741699374504568",{"2":{"258":1}}],["000778712",{"2":{"244":1}}],["000778960",{"2":{"244":1}}],["0007766818734608853",{"2":{"258":1}}],["000776893",{"2":{"244":1}}],["00077649",{"2":{"77":1}}],["0007736237",{"2":{"179":1}}],["00077338",{"2":{"77":1}}],["000925605",{"2":{"244":1}}],["000924496",{"2":{"244":1}}],["000924424",{"2":{"244":1}}],["000963809",{"2":{"244":1}}],["000964463",{"2":{"244":1}}],["000944194",{"2":{"244":1}}],["000942026",{"2":{"77":1}}],["000907963",{"2":{"244":1}}],["000908168",{"2":{"244":1}}],["000901216",{"2":{"244":1}}],["000906830",{"2":{"244":1}}],["000939374",{"2":{"244":1}}],["000938448",{"2":{"244":1}}],["000936727",{"2":{"77":1}}],["000952036",{"2":{"244":1}}],["000994050",{"2":{"244":1}}],["000992391663909964",{"2":{"53":2}}],["000972152",{"2":{"244":1}}],["000971555",{"2":{"244":1}}],["000975201",{"2":{"244":1}}],["000983244",{"2":{"244":1}}],["000987124",{"2":{"244":1}}],["000910802",{"2":{"77":1}}],["0001841979643931747",{"2":{"258":1}}],["0001896060107507815",{"2":{"258":1}}],["00018960478",{"2":{"256":1}}],["00018072935657733683",{"2":{"258":1}}],["00018072716",{"2":{"256":1}}],["00018003360684332612",{"2":{"258":1}}],["00018003177",{"2":{"256":1}}],["00018392716073959746",{"2":{"258":1}}],["00018398354714342134",{"2":{"258":1}}],["00018398082",{"2":{"256":1}}],["00018373390934195612",{"2":{"258":1}}],["00018373223",{"2":{"256":1}}],["0001835237729894497",{"2":{"258":1}}],["0001835215",{"2":{"256":1}}],["00018356194787536639",{"2":{"258":1}}],["00018356094",{"2":{"256":1}}],["00018645044527984222",{"2":{"258":1}}],["00018645074",{"2":{"256":1}}],["000186362",{"2":{"244":1}}],["00018713126367945243",{"2":{"258":1}}],["0001871318",{"2":{"256":1}}],["00018736748861457612",{"2":{"258":1}}],["000187367",{"2":{"256":1}}],["000182339",{"2":{"244":1}}],["00014684055717249084",{"2":{"258":1}}],["00014683808",{"2":{"256":1}}],["0001420752248470576",{"2":{"258":1}}],["00014207355",{"2":{"256":1}}],["000142089973898955",{"2":{"258":1}}],["00014208801",{"2":{"256":1}}],["0001406573295786636",{"2":{"258":1}}],["0001406546",{"2":{"256":1}}],["00014048820538111552",{"2":{"258":1}}],["00014048498",{"2":{"256":1}}],["00014077283677615214",{"2":{"258":1}}],["00014077095",{"2":{"256":1}}],["00014000128773023168",{"2":{"258":1}}],["00014000213",{"2":{"256":1}}],["00014037393362512332",{"2":{"258":1}}],["00014037451",{"2":{"256":1}}],["00014790775097958878",{"2":{"258":1}}],["00014790527",{"2":{"256":1}}],["00014748457724355295",{"2":{"258":1}}],["00014748046",{"2":{"256":1}}],["0001471106521323205",{"2":{"258":1}}],["00014711094",{"2":{"256":1}}],["000147129955021222",{"2":{"258":1}}],["00014713164",{"2":{"256":1}}],["00014950335615383206",{"2":{"258":1}}],["00014959404106874985",{"2":{"258":1}}],["00014959453",{"2":{"256":1}}],["00014967567678637593",{"2":{"258":1}}],["00014967582",{"2":{"256":1}}],["00014949995",{"2":{"256":1}}],["00014390179316628636",{"2":{"258":1}}],["00014390076",{"2":{"256":1}}],["0001435068065732527",{"2":{"258":1}}],["00014350485",{"2":{"256":1}}],["0001458435253514241",{"2":{"258":1}}],["000145846",{"2":{"256":1}}],["00014524687479645457",{"2":{"258":1}}],["00014524818",{"2":{"256":1}}],["00014500531522536515",{"2":{"258":1}}],["00014500602",{"2":{"256":1}}],["00014533368570486814",{"2":{"258":1}}],["00014533561",{"2":{"256":1}}],["00014535845601967287",{"2":{"258":1}}],["00014535875",{"2":{"256":1}}],["00014165386899630094",{"2":{"258":1}}],["0001416553",{"2":{"256":1}}],["00014184437427534283",{"2":{"258":1}}],["00014184127",{"2":{"256":1}}],["00014197554854278972",{"2":{"258":1}}],["00014197214",{"2":{"256":1}}],["00014193265710141196",{"2":{"258":1}}],["00014193162",{"2":{"256":1}}],["00014102640496356923",{"2":{"258":1}}],["00014102783",{"2":{"256":1}}],["00014100888082791929",{"2":{"258":1}}],["00014100932",{"2":{"256":1}}],["0001417396025222873",{"2":{"258":1}}],["00014176820133833353",{"2":{"258":1}}],["0001417675",{"2":{"256":1}}],["00014174149",{"2":{"256":1}}],["0001444508757388977",{"2":{"258":1}}],["00014445429",{"2":{"256":1}}],["00014408118157250598",{"2":{"258":1}}],["00014407777",{"2":{"256":1}}],["00014427890815191073",{"2":{"258":1}}],["00014427994",{"2":{"256":1}}],["00014469436428041194",{"2":{"258":1}}],["00014469436",{"2":{"256":1}}],["000144504",{"2":{"244":1}}],["0001484359344974525",{"2":{"258":1}}],["00014843841",{"2":{"256":1}}],["000148993",{"2":{"244":1}}],["00014856220460460773",{"2":{"258":1}}],["00014856191",{"2":{"256":1}}],["000148591",{"2":{"244":1}}],["000148517",{"2":{"244":1}}],["000148092",{"2":{"244":1}}],["00011953957636680179",{"2":{"258":1}}],["0001195369",{"2":{"256":1}}],["00011020193216643964",{"2":{"258":1}}],["000110201225",{"2":{"256":1}}],["000114721893929503",{"2":{"258":1}}],["000114722425",{"2":{"256":1}}],["00011677485737647723",{"2":{"258":1}}],["00011677827",{"2":{"256":1}}],["000116407",{"2":{"244":1}}],["0001110199960913336",{"2":{"258":1}}],["000111023226",{"2":{"256":1}}],["00011187947169698394",{"2":{"258":1}}],["000111876994",{"2":{"256":1}}],["00011171042569901284",{"2":{"258":1}}],["000111710426",{"2":{"256":1}}],["00011769480156576498",{"2":{"258":1}}],["0001176917",{"2":{"256":1}}],["00011791795633571468",{"2":{"258":1}}],["00011791612",{"2":{"256":1}}],["0001177831",{"2":{"256":1}}],["000117897",{"2":{"244":1}}],["00011235110961097501",{"2":{"258":1}}],["00011235082",{"2":{"256":1}}],["000112325",{"2":{"244":1}}],["00011349621618470833",{"2":{"258":1}}],["000113493494",{"2":{"256":1}}],["00011325116567307211",{"2":{"258":1}}],["00011324993",{"2":{"256":1}}],["000113002246273502",{"2":{"258":1}}],["00011300167",{"2":{"256":1}}],["00011351757938636337",{"2":{"258":1}}],["00011351562",{"2":{"256":1}}],["00011318093813018094",{"2":{"258":1}}],["00011314247593445556",{"2":{"258":1}}],["00011314431",{"2":{"256":1}}],["0001131797",{"2":{"256":1}}],["00011860123435053287",{"2":{"258":1}}],["000118600394",{"2":{"256":1}}],["00011834211550667836",{"2":{"258":1}}],["00011834405",{"2":{"256":1}}],["00011883301309719392",{"2":{"258":1}}],["00011883248",{"2":{"256":1}}],["00011518689895296008",{"2":{"258":1}}],["00011518578",{"2":{"256":1}}],["00011588570321656757",{"2":{"258":1}}],["00011588641",{"2":{"256":1}}],["0001155409067835426",{"2":{"258":1}}],["00011553895",{"2":{"256":1}}],["00011547126992784686",{"2":{"258":1}}],["00011547375",{"2":{"256":1}}],["00011549952614571395",{"2":{"258":1}}],["000115499526",{"2":{"256":1}}],["000115742",{"2":{"244":1}}],["00017378003779430537",{"2":{"258":1}}],["000173648",{"2":{"244":1}}],["00017862167326355638",{"2":{"258":1}}],["00017861997",{"2":{"256":1}}],["0001715199494814266",{"2":{"258":1}}],["00017151685",{"2":{"256":1}}],["00017926344626274568",{"2":{"258":1}}],["00017926394",{"2":{"256":1}}],["00017587639239744914",{"2":{"258":1}}],["00017587443",{"2":{"256":1}}],["00017679114662646396",{"2":{"258":1}}],["00017678842",{"2":{"256":1}}],["00017645621383999068",{"2":{"258":1}}],["00017645433",{"2":{"256":1}}],["000176415",{"2":{"244":1}}],["00017755863734712595",{"2":{"258":1}}],["00017755636",{"2":{"256":1}}],["0001778669066706221",{"2":{"258":1}}],["00017786834",{"2":{"256":1}}],["00017712906605313057",{"2":{"258":1}}],["00017712849",{"2":{"256":1}}],["00017258334318211984",{"2":{"258":1}}],["00017258363",{"2":{"256":1}}],["00017261645583491674",{"2":{"258":1}}],["00017261699",{"2":{"256":1}}],["00017448405291111741",{"2":{"258":1}}],["00017448186",{"2":{"256":1}}],["00017465767725467546",{"2":{"258":1}}],["00017465782",{"2":{"256":1}}],["00017423770613624525",{"2":{"258":1}}],["00017423871",{"2":{"256":1}}],["0001742295643544126",{"2":{"258":1}}],["00017422912",{"2":{"256":1}}],["0001707022859018242",{"2":{"258":1}}],["00017070145",{"2":{"256":1}}],["0001706641062207913",{"2":{"258":1}}],["00017066523",{"2":{"256":1}}],["0001704501483856543",{"2":{"258":1}}],["00017041730661249363",{"2":{"258":1}}],["00017041463",{"2":{"256":1}}],["00017044914",{"2":{"256":1}}],["000170466",{"2":{"244":1}}],["000170909",{"2":{"244":1}}],["00015311768273859223",{"2":{"258":1}}],["00015311797",{"2":{"256":1}}],["00015020162503220176",{"2":{"258":1}}],["00015020274",{"2":{"256":1}}],["00015755462",{"2":{"256":1}}],["00015796158",{"2":{"256":1}}],["00015525591161911432",{"2":{"258":1}}],["00015525491",{"2":{"256":1}}],["000155360",{"2":{"244":1}}],["00015216271118495206",{"2":{"258":1}}],["00015216439",{"2":{"256":1}}],["0001524763183963476",{"2":{"258":1}}],["00015247412",{"2":{"256":1}}],["000152438",{"2":{"244":1}}],["00015470543068577935",{"2":{"258":1}}],["000154704",{"2":{"256":1}}],["00015452885534060902",{"2":{"258":1}}],["00015453056",{"2":{"256":1}}],["0001541797354414483",{"2":{"258":1}}],["00015418157",{"2":{"256":1}}],["00015499095352523564",{"2":{"258":1}}],["00015499145",{"2":{"256":1}}],["00015951260887843164",{"2":{"258":1}}],["00015903904680613504",{"2":{"258":1}}],["00015904017",{"2":{"256":1}}],["000159178",{"2":{"244":1}}],["0001564378934294019",{"2":{"258":1}}],["00015643479",{"2":{"256":1}}],["000156290270588567",{"2":{"258":1}}],["00015628956",{"2":{"256":1}}],["0001560697491277318",{"2":{"258":1}}],["0001560693",{"2":{"256":1}}],["000156979688795996",{"2":{"258":1}}],["00015691447302744298",{"2":{"258":1}}],["00015691279",{"2":{"256":1}}],["00015698072",{"2":{"256":1}}],["000158824",{"2":{"244":1}}],["000158011",{"2":{"244":1}}],["00015189332571507132",{"2":{"258":1}}],["000151896",{"2":{"256":1}}],["0001510564520755974",{"2":{"258":1}}],["00015105841",{"2":{"256":1}}],["00015158951166639118",{"2":{"258":1}}],["00015158781",{"2":{"256":1}}],["00015159335953988042",{"2":{"258":1}}],["00015159532",{"2":{"256":1}}],["00015113165126769704",{"2":{"258":1}}],["00015113266",{"2":{"256":1}}],["00015113852",{"2":{"78":1}}],["000151653384923252",{"2":{"258":1}}],["00015165155",{"2":{"256":1}}],["00015167990563421714",{"2":{"258":1}}],["00015167797",{"2":{"256":1}}],["0001516661527532875",{"2":{"258":1}}],["0001516642",{"2":{"256":1}}],["000151947",{"2":{"244":1}}],["0001288668983303012",{"2":{"258":1}}],["00012887",{"2":{"256":1}}],["00012804814",{"2":{"256":1}}],["00012842058",{"2":{"256":1}}],["00012860816527754854",{"2":{"258":1}}],["0001286435746757973",{"2":{"258":1}}],["0001286409",{"2":{"256":1}}],["00012861127",{"2":{"256":1}}],["00012499302045573937",{"2":{"258":1}}],["00012498979",{"2":{"256":1}}],["00012465669771524689",{"2":{"258":1}}],["00012465993",{"2":{"256":1}}],["0001247548386680384",{"2":{"258":1}}],["00012475614",{"2":{"256":1}}],["0001243740871558316",{"2":{"258":1}}],["00012437423",{"2":{"256":1}}],["00012438484",{"2":{"256":1}}],["0001245337878049409",{"2":{"258":1}}],["00012453321",{"2":{"256":1}}],["000124517",{"2":{"244":1}}],["00012018754565336047",{"2":{"258":1}}],["00012019002",{"2":{"256":1}}],["00012088524332279989",{"2":{"258":1}}],["00012088332",{"2":{"256":1}}],["00012052433643480654",{"2":{"258":1}}],["00012052626",{"2":{"256":1}}],["0001207773698145222",{"2":{"258":1}}],["000120778604",{"2":{"256":1}}],["000120435832983497",{"2":{"258":1}}],["00012043471",{"2":{"256":1}}],["00012033503",{"2":{"256":1}}],["00012069307359096032",{"2":{"258":1}}],["00012069575",{"2":{"256":1}}],["0001206786",{"2":{"256":1}}],["00012006429676623617",{"2":{"258":1}}],["00012006202",{"2":{"256":1}}],["0001200500033519782",{"2":{"258":1}}],["00012004942",{"2":{"256":1}}],["000120209",{"2":{"244":1}}],["00012396446310782835",{"2":{"258":1}}],["0001239677",{"2":{"256":1}}],["00012313984860331249",{"2":{"258":1}}],["00012313662",{"2":{"256":1}}],["0001234756815402654",{"2":{"258":1}}],["00012347617",{"2":{"256":1}}],["00012372035047955503",{"2":{"258":1}}],["00012371842",{"2":{"256":1}}],["00012962901750167164",{"2":{"258":1}}],["00012962654",{"2":{"256":1}}],["00012981763942686558",{"2":{"258":1}}],["00012981516",{"2":{"256":1}}],["00012901733300203338",{"2":{"258":1}}],["00012901663",{"2":{"256":1}}],["000129240",{"2":{"244":1}}],["00012162500041050782",{"2":{"258":1}}],["00012162571",{"2":{"256":1}}],["00012175496021181036",{"2":{"258":1}}],["000121753",{"2":{"256":1}}],["0001223582340001114",{"2":{"258":1}}],["00012235604",{"2":{"256":1}}],["000122899645329742",{"2":{"258":1}}],["0001220733963839318",{"2":{"258":1}}],["00012207228",{"2":{"256":1}}],["00012259212593998138",{"2":{"258":1}}],["0001225902",{"2":{"256":1}}],["00012254887",{"2":{"256":1}}],["00012290911436426698",{"2":{"258":1}}],["00012290153",{"2":{"256":1}}],["00012291323",{"2":{"256":1}}],["000122752",{"2":{"244":1}}],["0001266499888613928",{"2":{"258":1}}],["0001266497",{"2":{"256":1}}],["00012601838526281895",{"2":{"258":1}}],["00012601794",{"2":{"256":1}}],["000126034",{"2":{"244":1}}],["000126193",{"2":{"244":1}}],["00012569470097689044",{"2":{"258":1}}],["0001256964",{"2":{"256":1}}],["0001259864585244711",{"2":{"258":1}}],["0001259869",{"2":{"256":1}}],["00012599879378322225",{"2":{"258":1}}],["00012599683622492008",{"2":{"258":1}}],["00012599569",{"2":{"256":1}}],["00012599488",{"2":{"256":1}}],["000125504",{"2":{"244":1}}],["000125214",{"2":{"244":1}}],["00012743253578299793",{"2":{"258":1}}],["00012742943",{"2":{"256":1}}],["00012707317248540205",{"2":{"258":1}}],["00012707147",{"2":{"256":1}}],["00012706775996495492",{"2":{"258":1}}],["00012706364",{"2":{"256":1}}],["0001270878416079679",{"2":{"258":1}}],["00012708372",{"2":{"256":1}}],["000127040",{"2":{"244":1}}],["000127500",{"2":{"244":1}}],["00019458882288741277",{"2":{"258":1}}],["0001945894",{"2":{"256":1}}],["00019908673084726726",{"2":{"258":1}}],["00019981400564241558",{"2":{"258":1}}],["00019981673",{"2":{"256":1}}],["0001992857793629967",{"2":{"258":1}}],["0001992841",{"2":{"256":1}}],["00019730367283343157",{"2":{"258":1}}],["00019730491",{"2":{"256":1}}],["00019775689034130065",{"2":{"258":1}}],["00019776101",{"2":{"256":1}}],["00019350280352469694",{"2":{"258":1}}],["0001935173529664127",{"2":{"258":1}}],["0001935182",{"2":{"256":1}}],["00019349939",{"2":{"256":1}}],["000193357",{"2":{"244":1}}],["000195880274450976",{"2":{"258":1}}],["00019587998",{"2":{"256":1}}],["00019523252218566324",{"2":{"258":1}}],["00019523122",{"2":{"256":1}}],["000195276",{"2":{"244":1}}],["0001988037280716143",{"2":{"258":1}}],["0001988018",{"2":{"256":1}}],["00019841713842119285",{"2":{"258":1}}],["0001984161",{"2":{"256":1}}],["00019824269808307467",{"2":{"258":1}}],["00019824217",{"2":{"256":1}}],["00019086928500806669",{"2":{"258":1}}],["0001908205796822708",{"2":{"258":1}}],["00019082129",{"2":{"256":1}}],["0001901418340891797",{"2":{"258":1}}],["0001901438",{"2":{"256":1}}],["00019273749398282397",{"2":{"258":1}}],["00019273692",{"2":{"256":1}}],["0001928729288418158",{"2":{"258":1}}],["000192871",{"2":{"256":1}}],["000192454",{"2":{"244":1}}],["000191858",{"2":{"244":1}}],["000191290",{"2":{"244":1}}],["00013635585733932896",{"2":{"258":1}}],["00013635393",{"2":{"256":1}}],["000136768",{"2":{"244":1}}],["0001332074155123078",{"2":{"258":1}}],["00013320909",{"2":{"256":1}}],["00013338797028697872",{"2":{"258":1}}],["0001333885",{"2":{"256":1}}],["00013065672792155194",{"2":{"258":1}}],["00013065261",{"2":{"256":1}}],["00013001370713595114",{"2":{"258":1}}],["00013001259",{"2":{"256":1}}],["00013078399943531718",{"2":{"258":1}}],["000130783",{"2":{"256":1}}],["00013130464961910515",{"2":{"258":1}}],["00013130245",{"2":{"256":1}}],["00013161742528371577",{"2":{"258":1}}],["00013169591165451328",{"2":{"258":1}}],["00013169323",{"2":{"256":1}}],["00013162084",{"2":{"256":1}}],["00013121",{"2":{"77":1}}],["00013795642381837713",{"2":{"258":1}}],["00013795459",{"2":{"256":1}}],["00013727077421109564",{"2":{"258":1}}],["00013727028",{"2":{"256":1}}],["00013706251806068834",{"2":{"258":1}}],["0001370642",{"2":{"256":1}}],["00013919706143595875",{"2":{"258":1}}],["00013919755",{"2":{"256":1}}],["00013982989343114906",{"2":{"258":1}}],["00013982822",{"2":{"256":1}}],["0001398982530597972",{"2":{"258":1}}],["00013989632",{"2":{"256":1}}],["0001398492509067325",{"2":{"258":1}}],["00013985121",{"2":{"256":1}}],["00013454334479343072",{"2":{"258":1}}],["00013454087",{"2":{"256":1}}],["00013408781358486153",{"2":{"258":1}}],["00013408852",{"2":{"256":1}}],["000134948",{"2":{"244":1}}],["00013525608072101174",{"2":{"258":1}}],["00013571456381954395",{"2":{"258":1}}],["00013571442",{"2":{"256":1}}],["00013558740496225476",{"2":{"258":1}}],["00013558794",{"2":{"256":1}}],["00013548151764520408",{"2":{"258":1}}],["00013548152",{"2":{"256":1}}],["00013285097785264679",{"2":{"258":1}}],["00013284909",{"2":{"256":1}}],["00013249322259448345",{"2":{"258":1}}],["00013249219",{"2":{"256":1}}],["000132906",{"2":{"244":1}}],["000132676",{"2":{"244":1}}],["00013858931324981928",{"2":{"258":1}}],["00013858659",{"2":{"256":1}}],["0001386984855569372",{"2":{"258":1}}],["00013840894088736912",{"2":{"258":1}}],["00013840995",{"2":{"256":1}}],["00013806619213939137",{"2":{"258":1}}],["00013806451",{"2":{"256":1}}],["00013818619481746985",{"2":{"258":1}}],["0001381862",{"2":{"256":1}}],["00013870077",{"2":{"256":1}}],["000138344",{"2":{"244":1}}],["000138281",{"2":{"244":1}}],["00010993102350438319",{"2":{"258":1}}],["000109933746",{"2":{"256":1}}],["00010979878209107037",{"2":{"258":1}}],["00010950736759688676",{"2":{"258":1}}],["00010950737",{"2":{"256":1}}],["00010980106",{"2":{"256":1}}],["00010432152608951083",{"2":{"258":1}}],["00010432372",{"2":{"256":1}}],["00010477013827213785",{"2":{"258":1}}],["00010477184",{"2":{"256":1}}],["00010446825",{"2":{"256":1}}],["00010147573620435553",{"2":{"258":1}}],["00010147674",{"2":{"256":1}}],["00010194693624761005",{"2":{"258":1}}],["00010194623",{"2":{"256":1}}],["00010122614",{"2":{"256":1}}],["00010180867694196524",{"2":{"258":1}}],["00010180767",{"2":{"256":1}}],["00010184901444884173",{"2":{"258":1}}],["00010184848",{"2":{"256":1}}],["00010187309817400318",{"2":{"258":1}}],["0001018731",{"2":{"256":1}}],["00010347370924434819",{"2":{"258":1}}],["000103474944",{"2":{"256":1}}],["00010336604298537744",{"2":{"258":1}}],["00010336675",{"2":{"256":1}}],["00010375578868313764",{"2":{"258":1}}],["000103754785",{"2":{"256":1}}],["00010376478541247726",{"2":{"258":1}}],["00010376295",{"2":{"256":1}}],["000103729",{"2":{"244":1}}],["00010300754038227083",{"2":{"258":1}}],["00010300705",{"2":{"256":1}}],["00010303148186721462",{"2":{"258":1}}],["00010303316",{"2":{"256":1}}],["00010235058645694412",{"2":{"258":1}}],["00010234916",{"2":{"256":1}}],["00010246663904548159",{"2":{"258":1}}],["000102468526",{"2":{"256":1}}],["00010223080554242442",{"2":{"258":1}}],["000102230515",{"2":{"256":1}}],["00010264589545939696",{"2":{"258":1}}],["00010264639",{"2":{"256":1}}],["00010280328211156535",{"2":{"258":1}}],["000102805214",{"2":{"256":1}}],["00010207414153895346",{"2":{"258":1}}],["00010207755",{"2":{"256":1}}],["00010204861825989456",{"2":{"258":1}}],["00010204634",{"2":{"256":1}}],["00010201420308684551",{"2":{"258":1}}],["00010201224",{"2":{"256":1}}],["0001004086417647983",{"2":{"258":1}}],["000100411744",{"2":{"256":1}}],["00010019202940388812",{"2":{"258":1}}],["00010019395",{"2":{"256":1}}],["0001003058989041292",{"2":{"258":1}}],["000100306606",{"2":{"256":1}}],["00010023864857796439",{"2":{"258":1}}],["00010023812",{"2":{"256":1}}],["00010061746934998852",{"2":{"258":1}}],["00010061943",{"2":{"256":1}}],["00010002195",{"2":{"256":1}}],["00010077263899800934",{"2":{"258":1}}],["00010077403340593758",{"2":{"258":1}}],["00010077432",{"2":{"256":1}}],["00010077599",{"2":{"256":1}}],["0001081674018812073",{"2":{"258":1}}],["000108165565",{"2":{"256":1}}],["00010838345154639748",{"2":{"258":1}}],["00010838457",{"2":{"256":1}}],["00010889856866223033",{"2":{"258":1}}],["00010889661",{"2":{"256":1}}],["00010899481616663255",{"2":{"258":1}}],["00010899424",{"2":{"256":1}}],["00010899297522072281",{"2":{"258":1}}],["00010899347",{"2":{"256":1}}],["000108091",{"2":{"244":1}}],["0001076371262514288",{"2":{"258":1}}],["00010763813",{"2":{"256":1}}],["0001072664006335285",{"2":{"258":1}}],["00010726528",{"2":{"256":1}}],["00010711853246899926",{"2":{"258":1}}],["00010711853",{"2":{"256":1}}],["00010799095616670282",{"2":{"258":1}}],["000107990956",{"2":{"256":1}}],["00010650497807117498",{"2":{"258":1}}],["00010650766",{"2":{"256":1}}],["00010643072779770449",{"2":{"258":1}}],["00010642961",{"2":{"256":1}}],["000106173",{"2":{"244":1}}],["000106987",{"2":{"244":1}}],["000106323",{"2":{"244":1}}],["0001051048244499431",{"2":{"258":1}}],["000105108054",{"2":{"256":1}}],["0001053717727699719",{"2":{"258":1}}],["00010537235",{"2":{"256":1}}],["00010593192406465257",{"2":{"258":1}}],["0001059336",{"2":{"256":1}}],["00010557082660925735",{"2":{"258":1}}],["000105570536",{"2":{"256":1}}],["00010551",{"2":{"179":1}}],["000105879",{"2":{"244":1}}],["00010542904541584576",{"2":{"258":1}}],["00010542153402936693",{"2":{"258":1}}],["00010542277",{"2":{"256":1}}],["00010543097",{"2":{"256":1}}],["000105440",{"2":{"244":1}}],["00010546855",{"2":{"179":1}}],["000105296",{"2":{"77":1}}],["00016779499629383977",{"2":{"258":1}}],["00016779696",{"2":{"256":1}}],["0001691603719510617",{"2":{"258":1}}],["00016915894",{"2":{"256":1}}],["00016950217490102439",{"2":{"258":1}}],["00016950133",{"2":{"256":1}}],["0001698451632771909",{"2":{"258":1}}],["00016984323",{"2":{"256":1}}],["0001682267913164207",{"2":{"258":1}}],["00016822782",{"2":{"256":1}}],["00016812379",{"2":{"256":1}}],["00016878",{"2":{"179":1}}],["00016414398814676247",{"2":{"258":1}}],["00016413987",{"2":{"256":1}}],["00016463631845207847",{"2":{"258":1}}],["00016463436",{"2":{"256":1}}],["00016427322279927948",{"2":{"258":1}}],["00016427264",{"2":{"256":1}}],["0001642029965294685",{"2":{"258":1}}],["000164203",{"2":{"256":1}}],["000164069",{"2":{"244":1}}],["0001623345884553702",{"2":{"258":1}}],["00016233047",{"2":{"256":1}}],["00016278515",{"2":{"256":1}}],["0001629407816428321",{"2":{"258":1}}],["00016294271",{"2":{"256":1}}],["000162970",{"2":{"244":1}}],["000162085",{"2":{"244":1}}],["00016633264583980773",{"2":{"258":1}}],["00016633335",{"2":{"256":1}}],["00016674744975718865",{"2":{"258":1}}],["000166747",{"2":{"256":1}}],["000166804",{"2":{"244":1}}],["000166462",{"2":{"244":1}}],["00016129921949133171",{"2":{"258":1}}],["00016129599",{"2":{"256":1}}],["00016146013855451884",{"2":{"258":1}}],["00016146098",{"2":{"256":1}}],["00016179558928054307",{"2":{"258":1}}],["0001617969",{"2":{"256":1}}],["000161607",{"2":{"244":1}}],["000161545",{"2":{"244":1}}],["00016001904",{"2":{"256":1}}],["000160591",{"2":{"244":1}}],["000160725",{"2":{"244":1}}],["00016329006294691238",{"2":{"258":1}}],["00016328818",{"2":{"256":1}}],["00016309728881215715",{"2":{"258":1}}],["00016309925",{"2":{"256":1}}],["0001631283993620916",{"2":{"258":1}}],["00016312697",{"2":{"256":1}}],["000163182",{"2":{"244":1}}],["000163823",{"2":{"244":1}}],["00016510924884892",{"2":{"258":1}}],["00016515781657519486",{"2":{"258":1}}],["00016515724",{"2":{"256":1}}],["00016511144",{"2":{"256":1}}],["000165743",{"2":{"244":1}}],["000165069",{"2":{"244":1}}],["000165657",{"2":{"77":1}}],["0001",{"2":{"77":7,"78":1}}],["0001f0",{"2":{"77":1}}],["01380257574673e",{"2":{"258":1}}],["013499700",{"2":{"244":1}}],["013011228",{"2":{"244":1}}],["01336",{"2":{"187":1}}],["013224f",{"2":{"256":1}}],["01324",{"2":{"187":1}}],["01328",{"2":{"187":1}}],["01371",{"2":{"187":1}}],["01372",{"2":{"187":1}}],["01375",{"2":{"187":1}}],["013515576783818e",{"2":{"258":1}}],["01358",{"2":{"187":1}}],["01357",{"2":{"187":1}}],["01350",{"2":{"187":1}}],["01362",{"2":{"187":1}}],["01368",{"2":{"187":1}}],["013616f",{"2":{"256":1}}],["01361",{"2":{"187":2}}],["01364",{"2":{"187":1}}],["01366",{"2":{"187":1}}],["01315",{"2":{"187":1}}],["01395",{"2":{"187":1}}],["01396",{"2":{"187":1}}],["013992591",{"2":{"106":1}}],["013992601",{"2":{"106":1}}],["017750096",{"2":{"244":1}}],["017799046",{"2":{"244":1}}],["01779",{"2":{"187":1}}],["017614847",{"2":{"244":1}}],["017605182",{"2":{"244":1}}],["017645847",{"2":{"244":1}}],["01767",{"2":{"187":1}}],["01785",{"2":{"187":1}}],["017197266",{"2":{"244":1}}],["01711",{"2":{"187":1}}],["01715",{"2":{"187":1}}],["0171745",{"2":{"168":1}}],["01755",{"2":{"187":1}}],["01752",{"2":{"187":1}}],["01759",{"2":{"187":1}}],["01733",{"2":{"187":1}}],["01709",{"2":{"187":1}}],["017215939",{"2":{"244":1}}],["01720",{"2":{"187":1}}],["01724",{"2":{"187":1}}],["017432783",{"2":{"106":2}}],["0162168166267354e",{"2":{"258":1}}],["016228525",{"2":{"244":1}}],["016558f",{"2":{"256":1}}],["01658",{"2":{"187":1}}],["0161936f",{"2":{"256":1}}],["016118063",{"2":{"244":1}}],["016987354",{"2":{"244":1}}],["01695",{"2":{"187":1}}],["01699",{"2":{"187":1}}],["01670",{"2":{"187":1}}],["01675",{"2":{"187":1}}],["016834570084262e",{"2":{"258":1}}],["0168346f",{"2":{"256":1}}],["01680",{"2":{"187":1}}],["0168695",{"2":{"145":1}}],["016490150",{"2":{"244":1}}],["01649",{"2":{"187":1}}],["016479947",{"2":{"106":2}}],["0163922f",{"2":{"256":1}}],["0163849f",{"2":{"256":1}}],["016386209",{"2":{"244":1}}],["0163",{"2":{"249":1}}],["01634",{"2":{"187":1}}],["01632",{"2":{"187":1}}],["0160255751533918e",{"2":{"258":1}}],["01605",{"2":{"187":1}}],["01609",{"2":{"187":1}}],["0160898448823084e",{"2":{"258":1}}],["01608",{"2":{"187":1}}],["01667",{"2":{"187":1}}],["01660",{"2":{"187":1}}],["016681549131931e",{"2":{"258":1}}],["01668",{"2":{"187":1}}],["01661",{"2":{"187":1}}],["01f0",{"2":{"145":1,"146":1,"179":2,"187":1}}],["019325165",{"2":{"244":1}}],["01935",{"2":{"187":2}}],["019859709",{"2":{"244":1}}],["01989s\\ttraining",{"2":{"228":1}}],["019567011",{"2":{"244":1}}],["01957",{"2":{"187":1}}],["01962",{"2":{"187":1}}],["01965834",{"2":{"126":1}}],["019484218",{"2":{"244":1}}],["01947",{"2":{"187":1}}],["019427970",{"2":{"244":1}}],["01942",{"2":{"187":1}}],["01997",{"2":{"187":1}}],["019155467",{"2":{"244":1}}],["01911",{"2":{"187":1}}],["01911576",{"2":{"106":1}}],["019115759",{"2":{"106":1}}],["01910",{"2":{"187":1}}],["01919",{"2":{"187":1}}],["01926",{"2":{"187":1}}],["01927",{"2":{"187":1}}],["019236464",{"2":{"77":1}}],["01975s\\ttraining",{"2":{"228":1}}],["0197287",{"2":{"168":1}}],["019791335",{"2":{"106":1}}],["019791333",{"2":{"106":1}}],["018483378",{"2":{"244":1}}],["0189805",{"2":{"232":1}}],["01892",{"2":{"187":1}}],["018073166",{"2":{"244":1}}],["01803",{"2":{"187":1}}],["018008159",{"2":{"244":1}}],["01800",{"2":{"187":1}}],["01851",{"2":{"187":1}}],["01816",{"2":{"187":1}}],["01828",{"2":{"187":1}}],["018884756",{"2":{"106":1}}],["018884761",{"2":{"106":1}}],["018338893",{"2":{"106":1}}],["018338889",{"2":{"106":1}}],["018706074",{"2":{"244":1}}],["018701272",{"2":{"77":1}}],["018723227",{"2":{"244":1}}],["01872db4",{"2":{"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["018743665453639813",{"2":{"174":1}}],["0187668",{"2":{"106":1}}],["015767f",{"2":{"256":1}}],["015748031",{"2":{"232":2}}],["015494239",{"2":{"244":1}}],["015478307",{"2":{"235":1}}],["01541",{"2":{"187":1}}],["01542492",{"2":{"107":1}}],["0152288709595648e",{"2":{"258":1}}],["015280124",{"2":{"244":1}}],["0152843",{"2":{"168":1}}],["015216017",{"2":{"244":1}}],["01526",{"2":{"187":1}}],["01502",{"2":{"187":1}}],["015340659",{"2":{"244":1}}],["01534",{"2":{"187":1}}],["01512",{"2":{"187":1}}],["01597",{"2":{"187":1}}],["01595",{"2":{"187":1}}],["015937408",{"2":{"126":1,"129":1}}],["01564",{"2":{"187":1}}],["01551",{"2":{"187":1}}],["0155126",{"2":{"106":1}}],["0158726",{"2":{"77":1}}],["010004668",{"2":{"244":1}}],["0100971125",{"2":{"77":1}}],["010971f",{"2":{"256":1}}],["010920972",{"2":{"244":1}}],["010936455",{"2":{"244":1}}],["010336538",{"2":{"244":1}}],["010518193",{"2":{"244":1}}],["010586374",{"2":{"244":1}}],["0105465",{"2":{"77":1}}],["01029",{"2":{"187":1}}],["0102189295",{"2":{"106":1}}],["010218928",{"2":{"106":1}}],["010755754",{"2":{"244":1}}],["01077",{"2":{"187":1}}],["01074",{"2":{"187":1}}],["01071",{"2":{"187":1}}],["01079",{"2":{"187":2}}],["01072",{"2":{"187":1}}],["010452257",{"2":{"244":1}}],["01044",{"2":{"187":1}}],["010404",{"2":{"145":1}}],["010629211",{"2":{"244":1}}],["01068",{"2":{"187":1}}],["010637204",{"2":{"106":1}}],["010637202",{"2":{"106":1}}],["0108878f",{"2":{"256":1}}],["010898398",{"2":{"244":1}}],["010876314",{"2":{"244":1}}],["01084",{"2":{"187":1}}],["0108668577150213",{"2":{"171":1}}],["01013",{"2":{"187":1}}],["0101",{"2":{"78":1}}],["012873f",{"2":{"256":1}}],["01288",{"2":{"187":1}}],["012775686",{"2":{"244":1}}],["01270",{"2":{"187":1}}],["012140292",{"2":{"244":1}}],["012183795",{"2":{"106":1}}],["012183794",{"2":{"106":1}}],["012323526",{"2":{"244":1}}],["0123139",{"2":{"77":1}}],["01298",{"2":{"187":1}}],["0122",{"2":{"249":1}}],["01224",{"2":{"187":1}}],["01225",{"2":{"187":1}}],["012002973",{"2":{"244":1}}],["01204",{"2":{"187":1}}],["01207",{"2":{"187":2}}],["01202",{"2":{"187":1}}],["012508835",{"2":{"244":1}}],["01250",{"2":{"187":1}}],["01259",{"2":{"187":1}}],["012522168",{"2":{"77":1}}],["01269",{"2":{"187":1}}],["01266",{"2":{"187":1}}],["01262",{"2":{"187":1}}],["01260",{"2":{"187":1}}],["01243",{"2":{"187":1}}],["014400685",{"2":{"244":1}}],["014388840086942e",{"2":{"258":1}}],["014340003",{"2":{"244":1}}],["014307282",{"2":{"244":1}}],["014098606",{"2":{"244":1}}],["0145318f",{"2":{"256":1}}],["014530762",{"2":{"244":1}}],["014555071",{"2":{"244":1}}],["01458",{"2":{"187":1}}],["0149",{"2":{"249":1}}],["01493",{"2":{"187":1}}],["01494",{"2":{"187":1}}],["0149811f",{"2":{"256":1}}],["01498",{"2":{"187":1}}],["014708381",{"2":{"244":1}}],["01472",{"2":{"187":1}}],["01479",{"2":{"187":1}}],["01467",{"2":{"187":1}}],["01464",{"2":{"187":1}}],["01463",{"2":{"187":1}}],["0146587",{"2":{"77":1}}],["014241929",{"2":{"244":1}}],["014243508",{"2":{"244":1}}],["01428",{"2":{"187":1}}],["0142814",{"2":{"77":1}}],["01422",{"2":{"187":1}}],["014815190",{"2":{"244":1}}],["01484",{"2":{"187":1}}],["01488",{"2":{"187":1}}],["014877922",{"2":{"106":1}}],["01412",{"2":{"187":1}}],["01415047",{"2":{"106":1}}],["014150472",{"2":{"106":1}}],["014100174",{"2":{"77":1}}],["011457845",{"2":{"244":1}}],["011906523",{"2":{"244":1}}],["011915382",{"2":{"244":1}}],["01193",{"2":{"187":1}}],["011937321",{"2":{"179":1}}],["011342090",{"2":{"244":1}}],["011374202",{"2":{"244":1}}],["01136",{"2":{"187":1}}],["0115952094448078e",{"2":{"258":1}}],["011598296",{"2":{"244":1}}],["011583f",{"2":{"256":1}}],["011505678",{"2":{"244":1}}],["01157",{"2":{"187":2}}],["01183",{"2":{"187":1}}],["011284535",{"2":{"244":1}}],["011247103",{"2":{"244":1}}],["01124",{"2":{"187":1}}],["01126",{"2":{"187":1}}],["01127",{"2":{"187":1}}],["011293f",{"2":{"256":1}}],["01129",{"2":{"187":1}}],["011087346",{"2":{"244":1}}],["011062279",{"2":{"244":1}}],["011037273",{"2":{"244":1}}],["01103",{"2":{"187":1}}],["011073254",{"2":{"244":1}}],["01107",{"2":{"187":1}}],["01110",{"2":{"187":1}}],["011100831f0",{"2":{"53":1}}],["01119",{"2":{"187":1}}],["0111896992904666e",{"2":{"258":1}}],["01118",{"2":{"187":2}}],["01111",{"2":{"187":1}}],["011713461786774e",{"2":{"258":1}}],["01170",{"2":{"187":1}}],["01176",{"2":{"187":1}}],["01174",{"2":{"187":2}}],["01173",{"2":{"187":1}}],["011734666",{"2":{"77":1}}],["011739168",{"2":{"77":1}}],["011611394",{"2":{"106":1}}],["011611392",{"2":{"106":1}}],["01",{"2":{"24":3,"53":2,"179":1,"208":4,"258":1}}],["0",{"2":{"2":1,"4":1,"7":3,"11":1,"19":3,"24":44,"34":2,"49":2,"53":122,"54":4,"59":16,"62":3,"64":3,"67":2,"68":10,"69":1,"74":1,"77":204,"78":14,"106":292,"107":11,"109":5,"110":37,"113":1,"121":1,"122":1,"124":2,"126":54,"129":25,"131":8,"140":6,"141":1,"142":7,"145":18,"146":5,"147":35,"148":23,"152":7,"153":358,"155":1,"168":58,"170":1,"171":155,"172":15,"173":3,"174":62,"176":12,"177":9,"178":4,"179":72,"180":11,"183":1,"186":1,"187":452,"189":11,"192":1,"193":2,"194":2,"195":7,"197":3,"206":4,"208":2,"210":11,"213":1,"214":2,"215":1,"216":2,"217":26,"218":1,"219":8,"220":19,"221":11,"224":2,"227":2,"228":105,"229":11,"232":139,"233":1,"234":4,"235":40,"236":11,"237":11,"243":7,"244":800,"245":6,"246":11,"248":4,"249":65,"251":3,"254":4,"255":9,"256":466,"257":1,"258":375,"260":3}}],["1e",{"2":{"208":1,"254":1}}],["1g",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["1×16",{"2":{"147":1,"148":1}}],["1×32",{"2":{"59":1,"78":1}}],["1`",{"2":{"141":3}}],["1st",{"2":{"127":1}}],["119375377567678e",{"2":{"258":1}}],["11984776",{"2":{"77":1}}],["1145324542991e",{"2":{"258":1}}],["1144479453016605e",{"2":{"258":1}}],["1144622f",{"2":{"256":1}}],["114121f",{"2":{"256":1}}],["1143",{"2":{"249":1}}],["114975",{"2":{"147":1}}],["1101569386649514e",{"2":{"258":1}}],["110875f",{"2":{"256":1}}],["11001",{"2":{"244":1}}],["11007148",{"2":{"232":1}}],["11023622",{"2":{"232":2}}],["11023762",{"2":{"179":1}}],["110",{"2":{"220":6}}],["11s",{"2":{"196":1}}],["1110424227072386e",{"2":{"258":1}}],["1116573f",{"2":{"256":1}}],["11161",{"2":{"187":1}}],["11111",{"2":{"219":1}}],["111715421055745e",{"2":{"258":1}}],["11171",{"2":{"187":1}}],["111552626",{"2":{"106":1}}],["115299832234561e",{"2":{"258":1}}],["115398386105128e",{"2":{"258":1}}],["1153668001507944e",{"2":{"258":1}}],["11535",{"2":{"187":1}}],["1154679f",{"2":{"256":1}}],["1151988f",{"2":{"256":1}}],["11515346",{"2":{"106":1}}],["11515345",{"2":{"106":1}}],["11501",{"2":{"244":1}}],["116989",{"2":{"171":1}}],["116796836",{"2":{"106":1}}],["118406968945824e",{"2":{"258":1}}],["11846f",{"2":{"256":1}}],["118809f",{"2":{"256":1}}],["1188200246206075",{"2":{"153":2}}],["1181102",{"2":{"232":2}}],["1187928",{"2":{"106":2}}],["112954f",{"2":{"256":1}}],["1121",{"2":{"249":1}}],["1126356",{"2":{"148":1}}],["1124609",{"2":{"106":1}}],["11322229060031e",{"2":{"258":1}}],["11352",{"2":{"187":1}}],["1137304",{"2":{"147":1}}],["11365668",{"2":{"106":2}}],["117825409116929e",{"2":{"258":1}}],["1171655702431014e",{"2":{"258":1}}],["1172363f",{"2":{"256":1}}],["117236",{"2":{"232":1}}],["1173s\\ttraining",{"2":{"219":1}}],["1173706",{"2":{"147":1}}],["117",{"2":{"217":1}}],["1175871f",{"2":{"106":1}}],["11",{"2":{"145":2,"146":2,"147":2,"148":1,"180":2,"187":14,"189":2,"210":2,"220":3,"221":2,"228":3,"229":2,"236":1,"237":2,"246":2,"249":3}}],["146893167705971e",{"2":{"258":1}}],["146316900063252e",{"2":{"258":1}}],["1461070325456786e",{"2":{"258":1}}],["1461293066412356e",{"2":{"258":1}}],["146621f",{"2":{"256":1}}],["146487f",{"2":{"256":1}}],["1462957f",{"2":{"256":1}}],["1406",{"2":{"249":2}}],["1405",{"2":{"249":1}}],["14001",{"2":{"244":1}}],["149555715169725e",{"2":{"258":1}}],["1495414f",{"2":{"256":1}}],["1490868",{"2":{"235":1}}],["1496063",{"2":{"232":2}}],["149151e",{"2":{"208":1}}],["14941669",{"2":{"77":1}}],["147694839410321e",{"2":{"258":1}}],["147583f",{"2":{"256":1}}],["14781",{"2":{"187":1}}],["147705370924899",{"2":{"153":1}}],["1477053709248992",{"2":{"153":1}}],["143404171644247e",{"2":{"258":1}}],["14340512",{"2":{"107":1}}],["143605621520986e",{"2":{"258":1}}],["1436957f",{"2":{"256":1}}],["143527677819532e",{"2":{"258":1}}],["143162f",{"2":{"256":1}}],["1431762f",{"2":{"256":1}}],["143798f",{"2":{"256":1}}],["1437841",{"2":{"179":1}}],["1439",{"2":{"249":1}}],["14388",{"2":{"187":1}}],["1458",{"2":{"249":1}}],["14501f",{"2":{"256":1}}],["14501",{"2":{"244":1}}],["145296",{"2":{"168":1}}],["14568081",{"2":{"106":1}}],["142839321734721e",{"2":{"258":1}}],["142283934572076e",{"2":{"258":1}}],["142251",{"2":{"106":1}}],["142470881033475",{"2":{"153":2}}],["148407230835695e",{"2":{"258":1}}],["1481878f",{"2":{"256":1}}],["14815",{"2":{"217":2}}],["14825",{"2":{"171":1}}],["148248",{"2":{"171":2}}],["1489007",{"2":{"147":1}}],["1488175",{"2":{"123":1}}],["14",{"2":{"145":1,"187":14,"217":1,"228":2,"249":7}}],["14445141",{"2":{"106":1}}],["14488737",{"2":{"106":1}}],["14132261",{"2":{"232":1}}],["14173229",{"2":{"232":2}}],["14118214",{"2":{"106":1}}],["1415927",{"2":{"53":1}}],["172136450476319e",{"2":{"258":1}}],["172469923189455e",{"2":{"258":1}}],["1729372f",{"2":{"256":1}}],["1720332f",{"2":{"256":1}}],["172325e",{"2":{"148":1}}],["17239982",{"2":{"77":1}}],["173944915744557e",{"2":{"258":1}}],["1731566735086957e",{"2":{"258":1}}],["173001f",{"2":{"256":1}}],["17322835",{"2":{"232":2}}],["17501",{"2":{"244":1}}],["17560",{"2":{"187":1}}],["17111921",{"2":{"235":1}}],["178097060060917e",{"2":{"258":1}}],["17816184",{"2":{"232":1}}],["17828",{"2":{"187":1}}],["177997f",{"2":{"256":1}}],["17725",{"2":{"187":1}}],["17706361",{"2":{"179":1}}],["17935471924539e",{"2":{"258":1}}],["17930",{"2":{"187":1}}],["1790317f",{"2":{"256":1}}],["1791",{"2":{"249":1}}],["179584495291061",{"2":{"153":2}}],["170725f",{"2":{"256":1}}],["170666939487488e",{"2":{"258":1}}],["1706",{"2":{"249":1}}],["17001",{"2":{"244":1}}],["170",{"2":{"229":1}}],["170342",{"2":{"171":1}}],["17048413",{"2":{"106":1}}],["1766681590665e",{"2":{"258":1}}],["1766674f",{"2":{"256":1}}],["176991f",{"2":{"256":1}}],["1769385",{"2":{"147":1}}],["1763",{"2":{"249":1}}],["176",{"2":{"208":1}}],["17602",{"2":{"187":1}}],["1764085",{"2":{"126":1}}],["17",{"2":{"110":1,"148":1,"187":14,"196":11,"217":2,"228":5,"233":1,"249":4,"258":16}}],["17441",{"2":{"171":1}}],["17449912",{"2":{"77":1}}],["17435724",{"2":{"106":2}}],["1746128",{"2":{"77":1}}],["1th",{"2":{"69":3}}],["13265146020744e",{"2":{"258":1}}],["13327349021029e",{"2":{"258":1}}],["1330848f",{"2":{"256":1}}],["13343152",{"2":{"106":1}}],["1345386f",{"2":{"256":1}}],["134673f",{"2":{"256":1}}],["13475561",{"2":{"232":1}}],["1363285551926876e",{"2":{"258":1}}],["136356",{"2":{"168":1}}],["1366388f",{"2":{"256":1}}],["1367566f",{"2":{"256":1}}],["130987903437382e",{"2":{"258":1}}],["1305885632492694e",{"2":{"258":1}}],["13054",{"2":{"187":1}}],["1303",{"2":{"249":1}}],["13001",{"2":{"244":1}}],["13073668",{"2":{"179":1}}],["130897im",{"2":{"168":1}}],["1306755",{"2":{"148":1}}],["13025",{"2":{"187":1}}],["1302",{"2":{"68":1}}],["138688332017845e",{"2":{"258":1}}],["13864",{"2":{"106":1}}],["13805",{"2":{"187":1}}],["13819042",{"2":{"126":1}}],["13",{"2":{"109":1,"187":14,"228":3,"244":4,"249":3}}],["137727386419124e",{"2":{"258":1}}],["1377298",{"2":{"106":2}}],["1378113f",{"2":{"256":1}}],["137417f",{"2":{"256":1}}],["13701",{"2":{"187":1}}],["13725637",{"2":{"106":1}}],["13725635",{"2":{"106":1}}],["13760868",{"2":{"106":1}}],["13760866",{"2":{"106":1}}],["13559",{"2":{"171":1}}],["135084952820556e",{"2":{"258":1}}],["13501",{"2":{"244":1}}],["1350521",{"2":{"147":1}}],["13504265",{"2":{"106":1}}],["13518013",{"2":{"106":1}}],["13518015",{"2":{"106":1}}],["139312f",{"2":{"256":1}}],["1395766",{"2":{"232":1}}],["13994",{"2":{"187":1}}],["139433",{"2":{"148":1}}],["13913931",{"2":{"106":1}}],["13988f",{"2":{"77":1}}],["131311f",{"2":{"256":1}}],["1310",{"2":{"249":1}}],["131726",{"2":{"147":1,"148":1}}],["131583631",{"2":{"78":1}}],["13168f",{"2":{"77":1}}],["1312",{"2":{"24":1}}],["1c",{"2":{"66":1}}],["1b",{"2":{"66":3}}],["1a",{"2":{"66":3}}],["189576818839681e",{"2":{"258":1}}],["189577f",{"2":{"256":1}}],["18941",{"2":{"168":1}}],["18481395",{"2":{"235":1}}],["1846",{"2":{"249":1}}],["18468",{"2":{"187":1}}],["184668",{"2":{"168":1}}],["1846486482317626",{"2":{"153":2}}],["1828",{"2":{"249":1}}],["1828456",{"2":{"232":1}}],["18276",{"2":{"187":1}}],["18501",{"2":{"244":1}}],["18535946",{"2":{"235":1}}],["18534",{"2":{"187":1}}],["18564",{"2":{"232":1}}],["18519",{"2":{"217":2}}],["185124",{"2":{"147":1}}],["1801805143487045e",{"2":{"258":1}}],["180348f",{"2":{"256":1}}],["18001",{"2":{"244":1}}],["18042",{"2":{"187":1}}],["18056087",{"2":{"179":1}}],["18089005",{"2":{"106":1}}],["1871622",{"2":{"176":4}}],["18794",{"2":{"168":1}}],["187646",{"2":{"168":1}}],["1867412622953456e",{"2":{"258":1}}],["186",{"2":{"256":1}}],["1860842f",{"2":{"256":1}}],["186571f",{"2":{"256":1}}],["18652338",{"2":{"148":1}}],["18695377",{"2":{"179":1}}],["186451",{"2":{"148":1}}],["1832",{"2":{"249":1}}],["18365",{"2":{"168":1}}],["1836515766351254",{"2":{"153":2}}],["183307",{"2":{"147":1}}],["18308182",{"2":{"126":1}}],["1887817",{"2":{"232":1}}],["188992e",{"2":{"208":1}}],["18841726",{"2":{"179":1}}],["18860114",{"2":{"126":1}}],["18807876",{"2":{"106":2}}],["18811704",{"2":{"77":1}}],["181246307532888e",{"2":{"258":1}}],["181208",{"2":{"77":1}}],["1811064f",{"2":{"256":1}}],["1811024",{"2":{"232":2}}],["181076f",{"2":{"256":1}}],["18102339",{"2":{"77":1}}],["1814s\\ttraining",{"2":{"217":1}}],["18",{"2":{"59":1,"187":14,"196":1,"217":1,"228":2,"249":5,"258":9}}],["1−z",{"2":{"66":1}}],["1−2∑yy^+α∑y2+∑y^2+α",{"2":{"53":1}}],["1−α",{"2":{"53":2}}],["1−y",{"2":{"53":1}}],["1−yy^",{"2":{"53":2}}],["1−y^+ϵ",{"2":{"53":1}}],["1−y~",{"2":{"53":2}}],["1671129438060196e",{"2":{"258":1}}],["1661803266749204e",{"2":{"258":1}}],["166129530056265e",{"2":{"258":1}}],["1661295f",{"2":{"256":1}}],["166389f",{"2":{"256":1}}],["1634160361519357e",{"2":{"258":1}}],["163950406014773e",{"2":{"258":1}}],["1639504f",{"2":{"256":1}}],["1637022907966216e",{"2":{"258":1}}],["1637023f",{"2":{"256":1}}],["16300425f",{"2":{"256":1}}],["16241",{"2":{"220":12}}],["16240",{"2":{"220":12}}],["165908f",{"2":{"256":1}}],["1652",{"2":{"249":1}}],["16526",{"2":{"187":1}}],["16501",{"2":{"244":1}}],["16505",{"2":{"187":1}}],["16566901",{"2":{"145":1}}],["1617228180412864",{"2":{"153":2}}],["16187939",{"2":{"106":1}}],["16450",{"2":{"220":6}}],["16486",{"2":{"187":1}}],["164786",{"2":{"171":1}}],["164379",{"2":{"168":1}}],["1642133",{"2":{"147":1}}],["1641532f",{"2":{"106":1}}],["16407",{"2":{"106":1}}],["16409892",{"2":{"106":1}}],["1693669411222534e",{"2":{"258":1}}],["169309f",{"2":{"256":1}}],["169817707313807e",{"2":{"258":1}}],["169227288980036e",{"2":{"258":1}}],["16965f",{"2":{"256":1}}],["169174f",{"2":{"256":1}}],["1691s\\ttraining",{"2":{"217":1}}],["16946f",{"2":{"77":1}}],["16909766",{"2":{"77":1}}],["1603039129696805e",{"2":{"258":1}}],["16034",{"2":{"187":1}}],["160258023912445e",{"2":{"258":1}}],["160413329608878e",{"2":{"258":1}}],["160474f",{"2":{"256":1}}],["160428f",{"2":{"256":1}}],["1605165f",{"2":{"256":1}}],["16001",{"2":{"244":1}}],["160876",{"2":{"168":1}}],["16083185f0",{"2":{"53":1}}],["160832f0",{"2":{"53":1}}],["16094846",{"2":{"145":1}}],["1607",{"2":{"19":2,"69":1}}],["16",{"2":{"39":1,"59":3,"109":13,"110":14,"121":1,"147":1,"148":1,"171":1,"187":14,"193":3,"228":3,"233":4,"236":2,"243":1,"244":8,"249":4,"251":4,"258":7,"260":4}}],["10f",{"2":{"257":1}}],["104465346083729e",{"2":{"258":1}}],["104298f",{"2":{"256":1}}],["1049",{"2":{"249":1}}],["10458f",{"2":{"77":1}}],["101",{"2":{"208":1,"220":6,"236":1}}],["10140238162746013",{"2":{"153":2}}],["108157604876883e",{"2":{"258":1}}],["1081045f",{"2":{"256":1}}],["108145",{"2":{"232":1}}],["108",{"2":{"196":1}}],["1057017f",{"2":{"256":1}}],["10572",{"2":{"187":1}}],["1058904237291846e",{"2":{"258":1}}],["1058",{"2":{"249":1}}],["10501",{"2":{"244":1}}],["10552457",{"2":{"106":1}}],["105524555",{"2":{"106":1}}],["10395",{"2":{"187":1}}],["10313",{"2":{"187":1}}],["10333",{"2":{"187":1}}],["1032465",{"2":{"106":1}}],["107883934623298e",{"2":{"258":1}}],["107888",{"2":{"232":1}}],["1077605f",{"2":{"256":1}}],["1079326",{"2":{"147":1}}],["10754459",{"2":{"106":1}}],["102150377145509e",{"2":{"258":1}}],["1024",{"2":{"153":7}}],["102",{"2":{"109":1,"217":1}}],["1099894f",{"2":{"256":1}}],["1090",{"2":{"254":2}}],["109",{"2":{"196":1}}],["109205071819074e",{"2":{"258":1}}],["109203495",{"2":{"106":1}}],["109261915",{"2":{"126":1}}],["1095082",{"2":{"106":1}}],["10950818",{"2":{"106":1}}],["106823550168255e",{"2":{"258":1}}],["106838f",{"2":{"256":1}}],["10688411",{"2":{"106":1}}],["106884085",{"2":{"106":1}}],["106069",{"2":{"145":1}}],["1069",{"2":{"106":1}}],["10",{"2":{"34":2,"39":2,"49":1,"62":1,"69":12,"74":2,"76":1,"77":5,"78":1,"106":5,"123":2,"124":1,"145":2,"146":2,"148":2,"171":4,"174":2,"179":3,"180":5,"187":14,"189":5,"193":5,"195":1,"196":22,"197":2,"210":5,"215":5,"220":30,"221":5,"226":1,"228":2,"229":5,"237":5,"246":5,"249":4,"250":1,"251":2,"254":2,"258":9,"260":2}}],["1001",{"2":{"179":1,"244":1}}],["1003911",{"2":{"142":1}}],["10001",{"2":{"244":1}}],["10000",{"2":{"179":2,"244":1}}],["1000",{"2":{"59":1,"78":3,"107":14,"179":1}}],["100",{"2":{"24":1,"49":2,"63":6,"78":1,"107":2,"180":2,"189":2,"195":2,"197":1,"210":2,"217":2,"220":6,"221":2,"228":32,"229":2,"237":2,"246":2,"251":1,"255":2,"260":1}}],["1220037199041655e",{"2":{"258":1}}],["12755343796869e",{"2":{"258":1}}],["127781945322305e",{"2":{"258":1}}],["1277556f0",{"2":{"53":1}}],["1272304f",{"2":{"256":1}}],["12001",{"2":{"244":1}}],["12096125",{"2":{"235":1}}],["1204217935313214",{"2":{"153":2}}],["12s",{"2":{"228":2}}],["1265",{"2":{"249":1}}],["126",{"2":{"208":1}}],["1261215f",{"2":{"256":1}}],["12611",{"2":{"187":1}}],["1261990389976131",{"2":{"174":1}}],["12183",{"2":{"187":1}}],["12156",{"2":{"187":1}}],["12193",{"2":{"187":1}}],["12501",{"2":{"244":1}}],["12533",{"2":{"187":1}}],["12589",{"2":{"179":1}}],["12556516",{"2":{"106":2}}],["124682054587932e",{"2":{"258":1}}],["124638f",{"2":{"256":1}}],["12477568",{"2":{"179":1}}],["12442",{"2":{"168":1}}],["1245008",{"2":{"147":1}}],["12abac4f24f6",{"2":{"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["12902719",{"2":{"106":2}}],["1293236",{"2":{"106":1}}],["12992674",{"2":{"106":1}}],["129",{"2":{"59":1,"69":1}}],["1280099f",{"2":{"256":1}}],["1287014f",{"2":{"256":1}}],["1286439",{"2":{"232":1}}],["12827216",{"2":{"106":1}}],["128",{"2":{"39":2,"59":12,"69":3,"77":4,"193":6,"195":1,"217":1,"232":4,"251":1,"260":1}}],["12",{"2":{"34":2,"140":1,"147":6,"148":7,"152":9,"171":2,"180":6,"187":14,"189":6,"197":1,"210":6,"221":6,"228":14,"229":6,"237":6,"246":6,"249":5,"251":1,"254":1,"260":1}}],["1234",{"2":{"248":1}}],["12345",{"2":{"232":1}}],["12367",{"2":{"187":1}}],["1239749674027375",{"2":{"153":2}}],["123",{"2":{"24":3,"62":1,"153":1}}],["1=dense",{"2":{"32":1}}],["1d",{"2":{"24":1,"66":1}}],["1f0",{"2":{"19":3,"53":2,"59":1,"69":2,"78":1,"232":1}}],["1946446036671373e",{"2":{"258":1}}],["1943",{"2":{"249":1}}],["1965621215445414e",{"2":{"258":1}}],["19641835432398e",{"2":{"258":1}}],["1964474f",{"2":{"256":1}}],["1963698f",{"2":{"256":1}}],["196703",{"2":{"131":2}}],["1988",{"2":{"254":2}}],["1984668",{"2":{"145":1}}],["1979838107002867e",{"2":{"258":1}}],["19791",{"2":{"187":1}}],["197045",{"2":{"171":1}}],["19501",{"2":{"244":1}}],["195746",{"2":{"232":1}}],["1953009",{"2":{"177":1,"178":1}}],["19597391412112541",{"2":{"174":1}}],["1958431",{"2":{"179":1}}],["1958",{"2":{"17":1,"249":1}}],["1914",{"2":{"249":1}}],["19193",{"2":{"187":1}}],["191956",{"2":{"172":1}}],["19124654",{"2":{"145":1}}],["19397983",{"2":{"179":1}}],["193291",{"2":{"171":1}}],["1936812",{"2":{"145":1}}],["19001",{"2":{"244":1}}],["1904755",{"2":{"148":1}}],["19084926",{"2":{"106":1}}],["199821f",{"2":{"256":1}}],["1990",{"2":{"149":1}}],["199387",{"2":{"145":1}}],["19975941",{"2":{"77":1}}],["19",{"2":{"122":1,"187":14,"196":1,"217":1,"228":5,"249":5}}],["192001543404211e",{"2":{"258":1}}],["19209f",{"2":{"106":2}}],["192641783709016e",{"2":{"258":1}}],["1926574f",{"2":{"256":1}}],["1924650916664847e",{"2":{"258":1}}],["1923136156973306e",{"2":{"258":1}}],["192343f",{"2":{"256":1}}],["1928305f",{"2":{"256":1}}],["192836",{"2":{"168":1}}],["1922104f",{"2":{"256":1}}],["1921s\\ttraining",{"2":{"217":1}}],["1921258",{"2":{"147":1}}],["1929",{"2":{"17":1}}],["1596533621909674e",{"2":{"258":1}}],["159416419616535e",{"2":{"258":1}}],["1594696f",{"2":{"256":1}}],["159547f",{"2":{"256":1}}],["159042358",{"2":{"244":2}}],["15s",{"2":{"196":1}}],["156228191004157e",{"2":{"258":1}}],["156144f",{"2":{"256":1}}],["1565",{"2":{"249":1}}],["15681",{"2":{"220":6}}],["15680",{"2":{"220":6}}],["15686738",{"2":{"106":1}}],["156",{"2":{"193":2}}],["15675",{"2":{"187":1}}],["1550",{"2":{"249":2}}],["15501",{"2":{"244":1}}],["1557574",{"2":{"148":1}}],["15544322",{"2":{"106":1}}],["15542f",{"2":{"77":1}}],["1576735437242696e",{"2":{"258":1}}],["1571982f",{"2":{"256":1}}],["1577266f",{"2":{"256":1}}],["1577715",{"2":{"77":1}}],["1570552561265787e",{"2":{"258":1}}],["15701",{"2":{"220":6}}],["15700",{"2":{"220":12}}],["15707",{"2":{"187":1}}],["15707001",{"2":{"147":1}}],["15792",{"2":{"187":1}}],["15276335",{"2":{"140":1}}],["1511",{"2":{"249":1}}],["1515",{"2":{"249":1}}],["151",{"2":{"208":1,"236":1}}],["15107603",{"2":{"106":1}}],["15107606",{"2":{"106":1}}],["15133555",{"2":{"106":1}}],["158594733371077e",{"2":{"258":1}}],["1585735",{"2":{"106":1}}],["1586391f",{"2":{"256":1}}],["1586047939092094",{"2":{"153":2}}],["15802455",{"2":{"106":1}}],["154041",{"2":{"171":1}}],["1545162",{"2":{"148":1}}],["15451166",{"2":{"77":1}}],["154",{"2":{"122":1,"193":2}}],["15493166",{"2":{"106":1}}],["1549292",{"2":{"106":1}}],["1501",{"2":{"244":1}}],["15001",{"2":{"244":1}}],["1500",{"2":{"213":2,"250":1}}],["15076",{"2":{"187":1}}],["150",{"2":{"69":1}}],["15",{"2":{"17":1,"53":1,"147":1,"148":3,"171":4,"180":3,"187":14,"189":3,"197":1,"210":3,"221":3,"228":2,"229":3,"237":3,"246":3,"249":5,"251":1,"260":1}}],["1",{"2":{"2":1,"4":1,"5":2,"7":1,"11":4,"17":8,"19":10,"24":48,"32":6,"39":5,"48":6,"49":1,"52":1,"53":110,"54":1,"59":24,"62":27,"63":9,"64":1,"65":24,"67":7,"68":12,"69":13,"70":3,"77":57,"78":13,"86":2,"93":1,"102":1,"106":36,"107":5,"109":35,"110":47,"121":6,"123":1,"124":2,"126":4,"127":2,"128":2,"129":9,"131":4,"134":2,"140":3,"141":11,"142":1,"145":13,"147":23,"148":19,"150":3,"151":3,"152":3,"153":88,"168":4,"171":45,"173":4,"174":5,"176":4,"177":6,"178":1,"179":6,"180":4,"183":11,"185":1,"186":1,"187":116,"189":4,"192":5,"193":8,"194":1,"195":5,"196":2,"197":1,"206":12,"208":17,"209":7,"210":4,"213":5,"214":3,"216":1,"217":6,"218":1,"219":1,"220":118,"221":4,"224":4,"225":1,"226":1,"227":1,"228":7,"229":4,"232":90,"233":5,"234":1,"235":4,"236":17,"237":4,"241":1,"242":6,"243":3,"244":6,"245":5,"246":4,"248":6,"249":69,"250":9,"251":1,"254":37,"255":7,"256":122,"257":2,"258":121,"259":5,"260":1}}],["e^2",{"2":{"255":1}}],["err",{"2":{"245":3}}],["errs",{"2":{"245":3}}],["errored",{"2":{"108":1}}],["errors",{"2":{"33":3,"111":1,"208":1}}],["error",{"2":{"2":3,"3":1,"31":1,"33":6,"50":2,"52":1,"53":3,"55":1,"57":2,"60":1,"69":1,"95":2,"102":1,"109":3,"145":1,"179":1,"245":1,"249":2}}],["ess",{"2":{"249":3}}],["essentially",{"2":{"59":1,"62":1,"141":1,"166":1,"236":1}}],["estimated",{"2":{"149":1}}],["estimate",{"2":{"149":1,"152":1}}],["estimation",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":1,"208":4}}],["established",{"2":{"81":1,"174":1}}],["especially",{"2":{"88":1,"108":1}}],["epyc",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["epochs",{"2":{"208":2,"236":3}}],["epoch",{"2":{"59":1,"187":353,"195":2,"217":2,"228":2,"236":11}}],["eps",{"2":{"19":8,"53":1}}],["epsilon=1f",{"2":{"69":4}}],["epsilon=0",{"2":{"53":2}}],["epsilon=nothing",{"2":{"53":1}}],["epsilon",{"2":{"19":9,"53":9,"69":4}}],["emp",{"2":{"145":2,"146":2}}],["empting",{"2":{"63":1}}],["empty",{"2":{"7":1,"8":2,"52":2,"53":1,"63":2,"65":9,"67":4,"68":5,"69":5,"70":1,"140":1,"179":1,"256":1}}],["embeddings",{"2":{"67":2}}],["embedding",{"2":{"67":2,"104":1,"153":1,"226":1}}],["embed",{"2":{"59":8,"78":8,"153":1}}],["either",{"2":{"53":1,"63":4,"65":6,"254":1}}],["eagerly",{"2":{"160":1}}],["easily",{"2":{"82":1,"145":1,"157":1}}],["easier",{"2":{"52":1,"68":1,"250":1}}],["easy",{"2":{"62":1,"71":1,"83":1,"108":1,"155":1}}],["eachindex",{"2":{"249":1}}],["eachslice",{"2":{"54":2,"184":1,"185":1}}],["each",{"2":{"5":1,"19":2,"24":1,"34":4,"62":15,"63":8,"65":12,"66":4,"67":2,"68":4,"69":2,"146":3,"171":2,"207":1,"249":1,"255":1}}],["evolved",{"2":{"250":1}}],["evalpoly",{"2":{"232":2,"236":1}}],["eval=32",{"2":{"224":1}}],["eval",{"2":{"153":4,"224":5}}],["evaluate",{"2":{"50":1}}],["eventually",{"2":{"153":1}}],["even",{"2":{"3":1,"7":1,"8":1,"55":1,"82":1,"83":1,"84":1,"108":1,"110":1,"156":1,"171":1,"175":1,"196":1}}],["everystep",{"2":{"220":13}}],["everystep=false",{"2":{"215":1}}],["everything",{"2":{"83":2,"134":1}}],["every",{"2":{"48":1,"64":1,"119":1,"120":1,"160":1,"171":1}}],["ever",{"2":{"3":1,"83":1}}],["ecosystem",{"2":{"37":1,"78":1,"85":1}}],["eccv",{"2":{"19":1}}],["edges",{"2":{"144":1}}],["edge",{"2":{"30":1,"83":1,"88":1,"89":1,"111":1,"153":1}}],["effort",{"2":{"106":1}}],["efficient",{"2":{"53":1,"55":1,"74":1,"105":1,"143":1,"145":1,"146":1,"175":2,"240":1}}],["efficiently",{"2":{"27":2,"53":1}}],["effectively",{"2":{"208":2}}],["effect",{"2":{"1":1,"60":1,"70":1}}],["european",{"2":{"19":1}}],["equilibrium",{"2":{"83":1}}],["equivalent",{"2":{"8":1,"39":1,"53":2,"55":4,"59":1,"68":1,"96":1,"171":1}}],["equations",{"2":{"74":1,"257":1}}],["equals",{"2":{"68":1}}],["equal",{"2":{"24":1,"34":1,"53":1,"63":2,"65":3}}],["equally",{"2":{"19":1,"54":1}}],["et",{"2":{"17":2,"24":2,"53":2,"252":1}}],["etc",{"2":{"3":2,"7":1,"20":1,"83":1,"133":1,"134":1,"136":1,"140":1,"143":1,"145":2,"174":1,"204":1}}],["elem",{"2":{"243":2,"245":2}}],["element",{"0":{"57":1},"2":{"17":2,"24":1,"34":1,"53":1,"55":1,"57":10,"59":2,"62":1,"66":6,"69":1,"153":4,"171":4,"176":1,"177":1,"179":1,"184":1,"188":1}}],["elementwise",{"2":{"15":1,"19":1,"68":2,"69":5}}],["elements",{"2":{"8":1,"24":9,"32":1,"63":2,"65":3,"67":1,"81":1,"171":6,"184":1}}],["elman",{"2":{"66":1}}],["eltypes",{"2":{"155":1,"165":1}}],["eltype",{"0":{"165":1},"2":{"19":9,"55":6,"56":3,"57":3,"121":1,"155":5,"165":2,"173":1}}],["elseif",{"2":{"132":1,"142":1,"254":1}}],["else",{"2":{"8":1,"17":1,"28":1,"37":1,"50":1,"52":1,"54":1,"55":1,"62":1,"64":3,"66":1,"68":1,"132":1,"254":3}}],["eg",{"2":{"8":1,"63":2,"65":3}}],["energy",{"2":{"249":2}}],["environment",{"2":{"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["engine",{"2":{"173":1}}],["enforce",{"2":{"173":1}}],["enough",{"2":{"133":1,"160":1,"208":1}}],["encountered",{"2":{"110":1}}],["encounter",{"2":{"89":1,"106":1,"144":1}}],["enhance",{"2":{"88":1}}],["entries",{"2":{"146":1}}],["entry",{"2":{"125":1}}],["entropy",{"2":{"53":3}}],["enter",{"2":{"76":1}}],["entire",{"2":{"64":2,"66":5,"69":1,"120":1,"208":1}}],["entirely",{"2":{"6":1,"63":2,"67":3,"218":1,"220":1}}],["enzymemlir",{"2":{"106":1}}],["enzyme",{"2":{"49":3,"52":1,"83":1,"86":1,"87":4,"89":1,"93":2,"106":10,"144":1,"153":1,"173":1,"175":1}}],["enable",{"2":{"157":1}}],["enables",{"2":{"18":1,"83":1,"247":1}}],["enabled",{"2":{"7":1,"33":2,"102":1,"247":1}}],["ensuring",{"2":{"17":1}}],["ensures",{"2":{"113":1}}],["ensure",{"2":{"2":1,"8":1,"24":1,"83":1,"89":1,"113":1,"149":1,"184":1,"235":1}}],["endpoints",{"2":{"254":2}}],["end",{"2":{"5":2,"11":1,"24":2,"32":2,"59":14,"62":4,"63":2,"65":3,"77":1,"78":7,"106":2,"107":4,"109":2,"110":4,"121":3,"122":1,"123":3,"126":1,"127":2,"128":2,"132":1,"134":3,"140":4,"141":2,"142":3,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"153":1,"160":2,"172":1,"174":2,"179":3,"180":3,"181":1,"183":3,"184":4,"185":3,"186":1,"187":4,"189":3,"192":1,"194":2,"195":3,"197":3,"206":2,"207":1,"208":4,"209":1,"210":3,"213":1,"214":7,"215":1,"216":2,"217":3,"218":3,"221":3,"224":2,"225":3,"226":1,"227":2,"228":4,"229":3,"232":2,"236":4,"237":3,"241":3,"242":3,"243":1,"244":3,"245":2,"246":3,"248":1,"249":4,"250":3,"251":3,"254":26,"255":2,"256":2,"257":2,"259":2,"260":3}}],["enumerate",{"2":{"5":2}}],["execution",{"2":{"174":1}}],["executables",{"2":{"106":1}}],["exhaustive",{"2":{"161":1}}],["existent",{"2":{"116":1}}],["exists",{"2":{"115":1,"214":1}}],["exist",{"2":{"59":2}}],["external",{"2":{"249":1}}],["extensively",{"2":{"83":1,"88":2,"89":1,"175":1}}],["extensive",{"2":{"82":1,"83":1}}],["extensible",{"2":{"82":1}}],["extensions",{"0":{"50":1}}],["extending",{"2":{"173":1,"184":1}}],["extended",{"2":{"31":1,"32":1,"33":1,"53":2,"57":1,"59":1,"62":6,"63":2,"65":3,"66":2,"68":1,"69":4,"70":1}}],["extend",{"2":{"7":1,"31":1,"83":1}}],["ext",{"2":{"122":2}}],["extrema",{"2":{"243":2}}],["extremely",{"2":{"33":1,"52":1,"130":1,"134":1}}],["extract",{"2":{"249":2,"250":1}}],["extra",{"2":{"24":1}}],["exact",{"2":{"24":1,"141":1,"187":1}}],["exactly",{"2":{"16":1,"54":1,"70":1,"96":1,"136":1,"140":1,"141":1}}],["examples",{"0":{"169":1},"2":{"5":1,"24":2,"53":2,"59":4,"84":1,"169":1,"202":1}}],["example",{"0":{"120":1},"1":{"121":1,"122":1,"123":1,"124":1},"2":{"2":1,"3":2,"4":1,"8":1,"19":1,"24":1,"28":1,"31":3,"32":2,"34":3,"37":1,"39":1,"48":1,"49":1,"52":1,"53":14,"58":1,"62":3,"68":5,"69":3,"84":1,"96":1,"99":1,"108":1,"112":2,"126":1,"127":1,"133":1,"134":1,"142":1,"143":1,"146":4,"147":1,"148":1,"155":1,"166":1,"176":1,"177":1,"256":1}}],["excellent",{"2":{"190":1}}],["except",{"2":{"7":1,"66":2,"68":1,"156":1}}],["exceptions",{"2":{"24":1}}],["exception",{"2":{"3":2,"50":1}}],["exciting",{"2":{"90":1}}],["exclusively",{"2":{"32":1,"47":1}}],["excluding",{"2":{"31":1,"39":1,"68":1,"69":1}}],["exclude",{"2":{"10":2,"92":1}}],["exclude=internal",{"2":{"10":1}}],["exp",{"2":{"243":1}}],["exposing",{"2":{"220":1}}],["exported",{"2":{"3":1,"6":1,"30":1,"172":1}}],["exploiting",{"2":{"160":1}}],["explore",{"2":{"144":2}}],["explicit",{"2":{"74":1,"83":1,"168":2,"247":1}}],["explicitly",{"2":{"7":1,"59":2}}],["express",{"2":{"249":1}}],["expressed",{"2":{"107":1}}],["expression",{"2":{"48":1,"59":1}}],["expr",{"2":{"50":4,"59":1}}],["expected",{"2":{"53":4,"62":1,"63":2,"67":1,"83":1,"110":1,"116":1,"174":1}}],["expect",{"2":{"30":1,"83":1,"106":1,"116":1,"153":1,"247":1}}],["expects",{"2":{"28":1,"39":1,"48":1,"63":1,"65":3,"70":1,"142":1}}],["experimental",{"0":{"30":1},"1":{"31":1,"32":1,"33":1,"34":1},"2":{"2":1,"4":2,"7":1,"30":2,"31":10,"32":2,"33":6,"34":2,"102":5,"103":5,"106":2,"108":1,"109":2,"110":2,"111":1,"117":1,"125":1,"126":3,"127":2,"128":2,"129":1,"130":1,"153":2,"155":1}}],["e",{"2":{"2":1,"11":1,"19":1,"24":1,"31":1,"37":1,"39":1,"53":3,"57":1,"58":1,"63":2,"64":2,"65":9,"69":3,"95":2,"106":1,"109":4,"110":4,"113":1,"122":2,"143":1,"149":1,"254":4,"255":6,"256":3}}],["gw",{"2":{"255":1}}],["gc",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["gnu",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["guess",{"2":{"258":1}}],["guide",{"0":{"113":1,"114":1},"1":{"115":1,"116":1},"2":{"114":1,"140":1}}],["guarantee",{"2":{"103":1,"166":1}}],["guarantees",{"2":{"30":1}}],["guaranteed",{"2":{"16":1,"18":1,"145":2}}],["gdev",{"2":{"100":1,"131":1,"132":4,"153":2,"160":6,"205":1,"207":3,"208":3,"209":1,"239":1,"244":3}}],["gif",{"2":{"245":1,"250":2}}],["gib",{"2":{"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["gigantic",{"2":{"108":1}}],["github",{"2":{"66":1,"71":2,"84":2,"111":1,"144":1,"153":1}}],["give",{"2":{"183":1,"258":2}}],["gives",{"2":{"54":1,"59":1,"140":1}}],["given",{"2":{"2":1,"4":2,"19":1,"20":1,"24":6,"25":24,"37":1,"42":6,"43":2,"44":5,"53":3,"62":2,"67":3,"69":5,"82":1,"83":1,"208":1}}],["g=tanh",{"2":{"66":1}}],["gs",{"2":{"59":2,"77":3,"195":1}}],["globals",{"2":{"256":1}}],["globally",{"2":{"247":1}}],["globallppool",{"2":{"65":1,"105":1}}],["globalmeanpool",{"2":{"65":1}}],["globalmaxpool",{"2":{"65":1}}],["global",{"2":{"48":1,"65":3,"153":1,"155":1,"207":1,"247":1}}],["glorot",{"2":{"24":4,"140":2}}],["gaussian",{"2":{"249":1}}],["gaussadjoint",{"2":{"217":1}}],["gated",{"2":{"66":1}}],["gamma=0",{"2":{"53":1}}],["gamma",{"2":{"53":2}}],["ganguli",{"2":{"24":1}}],["gain=1",{"2":{"24":1,"168":2}}],["gain",{"2":{"24":15,"63":4,"67":2,"104":2}}],["grid",{"2":{"243":5,"245":8}}],["green",{"2":{"236":1}}],["great",{"2":{"204":1}}],["ground",{"2":{"179":3}}],["group",{"2":{"19":3,"69":1,"249":1}}],["groups=1",{"2":{"63":2}}],["groups",{"2":{"19":4,"63":8,"69":4}}],["groupnorm",{"2":{"19":1,"69":7,"136":1}}],["gravitational",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["gravitate",{"2":{"55":1}}],["graph",{"2":{"248":1,"250":1}}],["grads",{"2":{"52":11,"107":1,"179":1}}],["gradient|jacobian",{"2":{"144":2}}],["gradient",{"0":{"49":1,"147":1},"2":{"27":2,"47":1,"49":1,"52":1,"106":8,"110":4,"133":2,"134":2,"145":6,"146":3,"147":6,"148":6,"152":3,"176":9,"179":1,"204":1,"242":3}}],["gradients",{"0":{"176":1},"2":{"27":2,"45":1,"49":10,"52":29,"53":2,"54":1,"62":1,"77":6,"115":1,"144":1,"145":1,"152":1,"176":1,"195":1,"240":1}}],["grucell",{"2":{"66":6,"104":1}}],["gru",{"2":{"7":1,"66":1}}],["günter",{"2":{"17":1}}],["gt",{"2":{"5":1,"11":1,"24":1,"53":1,"54":1,"59":2,"63":2,"65":3,"68":2,"69":2,"70":3,"79":1,"95":2,"100":1,"131":1,"141":2,"144":12,"184":1}}],["geared",{"2":{"204":1}}],["gelu",{"2":{"106":6,"107":2,"155":1}}],["geoffrey",{"2":{"19":1}}],["genericlossfunction",{"2":{"53":2}}],["genericbroadcastop",{"2":{"20":1}}],["generic",{"2":{"13":1,"16":2,"18":2,"78":1,"102":1,"106":1,"110":1,"131":1,"150":1,"151":1,"152":1,"153":1,"183":1,"185":1,"186":1,"192":1,"194":1,"195":1,"205":1,"213":1,"214":2,"215":1,"216":1,"224":1,"225":1,"226":1,"227":1,"232":1,"239":1,"242":2,"249":2,"250":1,"254":5,"256":1,"257":2}}],["generating",{"0":{"248":1},"2":{"70":1}}],["generation",{"2":{"66":3}}],["generator=lux",{"2":{"225":1}}],["generator",{"2":{"17":4,"24":3,"64":3,"140":1,"170":1,"174":2,"225":4,"226":2,"232":1}}],["generates",{"2":{"248":1}}],["generated",{"2":{"17":1,"24":1,"59":1,"64":1,"144":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"230":1,"237":1,"246":1,"250":1,"251":1,"260":1}}],["generate",{"0":{"206":1,"243":1},"2":{"3":1,"9":1,"10":1,"59":1,"66":1,"143":1,"168":2,"179":2,"183":1,"225":1,"232":4,"243":1,"248":2}}],["generalization",{"2":{"100":1}}],["generalized",{"2":{"5":1}}],["general",{"0":{"8":1},"2":{"62":1,"71":1,"79":1,"154":1,"256":1}}],["generally",{"2":{"5":1}}],["getaxes",{"2":{"225":1}}],["getindex",{"2":{"207":1}}],["getproperty",{"2":{"54":3,"142":2,"220":3}}],["getting",{"0":{"75":1},"1":{"76":1,"77":1,"78":1,"79":1,"80":1},"2":{"8":1}}],["getfield",{"2":{"7":1,"121":3,"142":3}}],["get",{"0":{"84":1},"2":{"3":5,"13":1,"14":1,"20":1,"41":2,"42":2,"43":2,"54":1,"77":1,"113":2,"133":1,"142":1,"149":2,"153":2,"165":1,"169":1,"171":2,"174":2,"183":3,"184":1,"187":2,"203":1,"249":2,"254":1}}],["g",{"2":{"2":1,"62":4,"64":2,"69":4,"106":1,"254":1}}],["goal",{"2":{"248":1}}],["goodies",{"2":{"177":1}}],["good",{"2":{"138":1,"145":1,"208":1,"249":1,"258":2}}],["goodfellow",{"2":{"68":1}}],["going",{"2":{"37":1,"147":1}}],["go",{"2":{"2":1,"84":2,"110":1,"120":1,"125":1,"141":1,"142":1,"154":1,"181":1,"238":1}}],["gpuci",{"2":{"109":1,"220":3,"244":4}}],["gpuarray",{"2":{"54":1}}],["gpuarrayscore",{"2":{"156":2}}],["gpuarrays",{"2":{"22":7,"54":1}}],["gpubroadcastop",{"2":{"20":1}}],["gpusintel",{"2":{"72":2}}],["gpusmetal",{"2":{"72":2}}],["gpusamd",{"2":{"72":2}}],["gpus",{"2":{"13":1,"42":1,"72":2,"130":1,"155":1,"156":1,"160":1}}],["gpu",{"0":{"4":1,"72":1,"80":1,"130":1,"153":1,"156":1,"163":1,"164":1},"1":{"131":1,"132":1},"2":{"1":8,"2":7,"3":10,"5":2,"16":1,"20":1,"49":1,"65":3,"72":4,"73":1,"77":2,"78":2,"80":1,"86":1,"87":3,"103":2,"106":2,"116":4,"121":1,"130":4,"131":9,"132":5,"134":1,"153":15,"160":4,"164":3,"168":1,"172":3,"187":1,"205":1,"207":3,"217":3,"220":1,"228":1,"235":3,"236":2,"239":1}}],["nparameters",{"2":{"249":3}}],["nvml",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["nvidia",{"2":{"3":1,"5":1,"72":2,"180":4,"189":4,"197":2,"210":4,"221":4,"229":4,"237":4,"246":4}}],["ndrange=length",{"2":{"153":2}}],["ndims",{"2":{"11":1,"28":4,"63":4,"65":9,"68":4,"69":2,"70":2,"121":2,"150":2,"151":2}}],["nicely",{"2":{"145":1}}],["nilarray",{"2":{"95":1}}],["nitish",{"2":{"17":1}}],["n=tanh⁡",{"2":{"66":1}}],["nn",{"2":{"63":2,"67":1,"153":7,"249":12,"250":11,"256":14,"257":1,"259":9}}],["nnlib",{"2":{"13":2,"14":2,"63":1,"70":6,"102":3,"133":2,"134":2,"158":3,"236":1}}],["ntuple",{"2":{"62":1,"63":2,"69":2,"121":1,"220":13,"244":1}}],["nlayers",{"2":{"59":2,"78":2}}],["nccl",{"2":{"41":1,"42":4,"115":1}}],["ncclbackend",{"2":{"41":2,"42":2,"113":4}}],["nabla",{"2":{"176":1}}],["native",{"2":{"137":1,"204":1}}],["naming",{"2":{"62":9,"68":2}}],["name>=",{"2":{"161":3}}],["namefreezing",{"2":{"127":1}}],["name=",{"2":{"59":1,"62":1}}],["name=nothing",{"2":{"59":1,"62":11}}],["names",{"2":{"31":1,"59":1,"69":1}}],["named",{"2":{"19":2,"31":2,"53":1,"142":1,"215":2,"217":1,"249":1}}],["namedtuples",{"2":{"142":1}}],["namedtuple",{"2":{"7":4,"8":1,"10":3,"31":5,"32":1,"52":1,"54":1,"55":1,"59":3,"62":10,"63":2,"65":9,"66":2,"67":4,"68":9,"69":5,"70":1,"77":24,"78":1,"95":1,"106":7,"109":3,"110":6,"119":3,"121":3,"122":7,"123":3,"126":3,"129":4,"140":4,"141":3,"143":1,"153":2,"184":2,"220":147,"235":2,"236":1,"249":1,"256":6}}],["name",{"0":{"127":1},"2":{"7":2,"8":5,"31":1,"32":1,"59":3,"68":1,"79":1,"102":1,"121":5,"127":5,"128":6,"131":1,"142":1,"228":4}}],["navab",{"2":{"53":1}}],["nassir",{"2":{"53":1}}],["nans",{"0":{"110":1},"2":{"33":3,"110":7,"111":1}}],["nan",{"2":{"33":5,"110":31,"244":1}}],["nₙ",{"2":{"28":2}}],["n₂",{"2":{"28":2}}],["n₁",{"2":{"28":2}}],["n",{"2":{"19":5,"54":2,"59":5,"62":23,"63":27,"65":54,"67":2,"68":11,"69":11,"70":8,"78":5,"107":1,"179":3,"187":2,"192":3,"195":1,"208":1,"213":3,"214":5,"217":1,"218":5,"224":12,"228":2,"236":1,"244":1,"248":2,"249":3,"250":2,"257":1}}],["numer",{"2":{"255":3,"256":3}}],["numeric",{"2":{"24":2}}],["numerically",{"2":{"186":1}}],["numerical",{"2":{"19":4,"53":1,"69":4,"155":1,"249":1}}],["num",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"250":2,"251":1,"260":1}}],["number=0",{"2":{"24":2}}],["number=1",{"2":{"24":2}}],["numbers",{"2":{"24":6,"25":12,"168":1,"171":3}}],["number",{"2":{"9":1,"10":1,"17":4,"19":1,"24":8,"43":1,"52":1,"53":4,"54":3,"55":1,"59":3,"62":3,"63":10,"64":3,"65":3,"67":7,"68":2,"69":3,"70":3,"140":1,"170":1,"171":1,"174":2,"175":1,"232":1,"248":1,"249":4}}],["nepochs",{"2":{"179":3,"195":3,"217":3,"228":3}}],["nest",{"2":{"145":1}}],["nested",{"0":{"29":1,"144":1,"162":1},"1":{"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1},"2":{"3":1,"29":2,"34":2,"55":5,"58":2,"86":1,"144":1,"147":2,"162":3,"238":1,"240":1,"245":1}}],["next",{"2":{"113":1,"140":1,"249":1,"250":1,"254":1,"256":1,"257":1}}],["nearest",{"2":{"70":7}}],["necessary",{"2":{"65":3,"140":1}}],["negative",{"2":{"53":1,"254":1}}],["net",{"2":{"53":1,"242":11,"249":1,"256":1,"259":2}}],["network",{"0":{"233":1,"247":1,"249":1,"256":1,"257":1,"258":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"19":1,"24":1,"78":1,"81":2,"83":1,"120":1,"145":1,"146":2,"153":2,"170":1,"181":2,"225":5,"226":3,"233":1,"248":1,"249":6,"250":4,"256":6}}],["networks",{"0":{"153":1,"241":1},"2":{"17":3,"24":3,"39":1,"53":2,"68":1,"74":1,"83":3,"152":1,"153":1,"190":1,"204":2,"241":2,"249":1}}],["never",{"2":{"30":1,"37":1,"68":1,"256":1}}],["neuralpde",{"2":{"238":2}}],["neuralode",{"2":{"214":7,"217":4,"220":5}}],["neuralodecompact",{"2":{"214":2,"217":1,"220":1}}],["neural",{"0":{"153":1,"211":1,"214":1,"215":1,"219":1,"233":1,"241":1,"247":1,"249":1,"252":1,"256":1,"257":1,"258":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"248":1,"249":1,"250":1,"251":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"17":4,"24":4,"39":1,"53":1,"58":1,"74":2,"78":1,"81":2,"83":4,"120":3,"145":3,"146":2,"152":1,"153":3,"170":1,"181":2,"190":1,"204":5,"211":2,"214":1,"215":1,"233":1,"248":1,"249":7,"250":2,"256":7,"257":1,"259":2}}],["needed",{"2":{"13":2,"27":2,"28":1,"52":1,"102":1,"115":1,"134":1,"144":1,"168":2}}],["need",{"2":{"6":1,"8":1,"31":1,"34":1,"37":1,"59":2,"68":1,"77":1,"90":1,"102":1,"106":3,"107":1,"109":1,"113":1,"114":2,"119":1,"134":1,"140":3,"141":2,"144":1,"153":1,"166":1,"169":1,"171":1,"183":1,"184":1,"193":2,"208":1,"214":1,"226":1,"235":1,"242":1,"254":1,"256":1}}],["needs",{"2":{"3":1,"7":1,"10":1,"13":2,"63":1,"64":1,"114":1,"121":1,"148":1,"164":1}}],["newtonian",{"2":{"254":2,"256":1,"258":1}}],["new",{"0":{"93":1,"96":1,"100":1,"105":1,"219":1},"2":{"1":1,"32":9,"34":5,"52":1,"59":1,"64":1,"66":4,"68":5,"90":1,"92":1,"103":1,"106":1,"114":1,"119":1,"130":1,"138":1,"140":1,"141":1,"144":1,"153":1,"225":2,"249":3}}],["nom",{"2":{"249":1}}],["noisy",{"2":{"183":2}}],["noise",{"2":{"179":1}}],["now",{"2":{"93":1,"100":1,"103":1,"104":4,"105":2,"106":4,"107":1,"109":2,"110":2,"114":1,"116":3,"134":1,"140":1,"141":1,"145":1,"146":2,"147":1,"148":1,"152":2,"153":2,"155":1,"170":2,"175":1,"179":1,"184":2,"186":1,"193":1,"196":1,"208":1,"236":1,"247":1,"249":2,"250":1,"254":1,"256":2,"257":1,"259":1}}],["norm",{"2":{"69":1,"145":7,"146":7,"147":6,"148":6,"152":12}}],["normally",{"2":{"106":1}}],["normal",{"2":{"22":4,"24":12,"25":6,"168":6,"249":1,"256":3}}],["normalises",{"2":{"19":2,"69":2}}],["normalize",{"2":{"69":2}}],["normalized",{"2":{"19":8,"53":3,"69":7,"245":2}}],["normalization",{"0":{"19":1,"69":1},"2":{"19":12,"24":1,"28":1,"69":11,"146":1}}],["normalizing",{"2":{"17":1}}],["nooplayer",{"2":{"62":6,"64":3,"68":3}}],["nonlinear",{"2":{"24":1}}],["non",{"0":{"124":1},"2":{"16":1,"18":1,"24":2,"31":1,"33":1,"42":2,"52":2,"53":1,"59":9,"67":1,"95":1,"98":1,"108":1,"116":1,"134":2,"203":1,"214":2,"254":1}}],["nonetheless",{"2":{"2":1}}],["none",{"2":{"2":1,"30":1,"31":1,"33":1,"57":1,"110":1}}],["node",{"2":{"3":1}}],["notice",{"2":{"140":1,"141":1,"146":2,"217":1}}],["notion",{"2":{"55":1}}],["notangent",{"2":{"49":1,"110":1}}],["not",{"2":{"2":1,"3":4,"4":2,"6":1,"7":1,"8":1,"15":1,"16":1,"17":1,"19":3,"24":2,"28":1,"33":1,"34":1,"39":3,"41":2,"47":1,"50":1,"52":1,"54":1,"57":1,"58":1,"59":1,"62":1,"66":11,"68":1,"69":1,"77":1,"83":1,"85":1,"88":1,"89":2,"95":2,"102":1,"106":1,"109":1,"110":1,"113":1,"116":1,"131":1,"134":3,"138":1,"144":1,"145":5,"149":1,"152":1,"153":1,"160":1,"167":1,"169":1,"196":2,"203":1,"208":1,"214":1,"217":2,"219":1,"220":2,"233":1,"249":1,"250":1,"254":1}}],["notes",{"2":{"16":1,"18":1,"49":1,"177":1,"211":1}}],["note",{"2":{"2":1,"3":4,"13":2,"24":1,"31":1,"40":1,"52":1,"53":1,"55":2,"58":1,"59":1,"68":1,"74":1,"89":1,"106":1,"107":1,"109":1,"113":1,"137":1,"140":1,"141":1,"144":1,"147":1,"156":1,"173":1,"176":1,"188":1,"214":1,"220":1,"247":1}}],["nothing",{"2":{"2":4,"3":5,"4":1,"8":2,"15":1,"16":1,"17":2,"18":1,"19":15,"31":5,"41":3,"42":1,"49":1,"53":12,"54":1,"55":2,"58":1,"62":4,"63":4,"64":1,"66":11,"67":4,"68":10,"69":5,"77":27,"78":1,"106":9,"110":1,"122":4,"145":2,"147":2,"148":2,"206":1,"207":4,"208":2,"220":117,"236":6,"249":1,"256":1}}],["no",{"2":{"2":2,"4":2,"5":1,"30":1,"31":3,"39":1,"48":1,"52":1,"53":2,"57":1,"60":1,"68":1,"77":1,"83":1,"97":1,"100":1,"102":1,"109":1,"110":1,"115":3,"122":1,"131":2,"132":1,"140":1}}],["u=u",{"2":{"242":1}}],["u0=res",{"2":{"208":1}}],["u0",{"2":{"206":2,"208":2,"209":1,"255":2,"256":2,"257":1,"259":2}}],["utc",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["utility",{"0":{"216":1,"227":1,"254":1}}],["utilities",{"0":{"51":1},"1":{"52":1,"53":1,"54":1,"55":1,"56":1,"57":1,"58":1,"59":1,"60":1},"2":{"47":1,"134":1}}],["utils",{"0":{"40":1},"1":{"41":1,"42":1,"43":1,"44":1,"45":1,"46":1},"2":{"31":2,"55":1,"145":4}}],["url",{"2":{"74":1}}],["url=",{"2":{"71":1}}],["u",{"2":{"27":7,"206":4,"208":11,"209":6,"214":7,"218":2,"241":4,"242":11,"244":4,"245":7,"255":3,"256":4,"258":1,"259":2}}],["ulyanov",{"2":{"19":1,"69":1}}],["upto",{"2":{"240":1}}],["upsampled",{"2":{"70":1}}],["upsample",{"2":{"70":5,"104":1}}],["upsampling",{"0":{"70":1},"2":{"70":2}}],["upscaling",{"2":{"70":1}}],["upscale",{"2":{"70":2}}],["up",{"0":{"257":1},"2":{"54":1,"149":1,"152":1,"203":2,"211":1}}],["updating",{"0":{"56":1,"90":1},"1":{"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":1,"103":1,"104":1,"105":1},"2":{"19":2,"45":1,"76":1,"179":1}}],["updates",{"2":{"34":1,"52":3,"69":1,"98":1,"113":1}}],["updated",{"2":{"15":1,"17":2,"19":2,"34":1,"52":5,"53":1,"62":6,"64":3,"66":7,"68":1,"69":1,"78":1,"82":1,"83":1,"104":1,"116":1,"140":1,"184":1,"203":1,"235":1}}],["update",{"2":{"10":2,"15":1,"17":2,"52":2,"56":1,"59":1,"64":3,"66":5,"69":3,"76":2,"90":1,"173":1}}],["upon",{"2":{"5":1}}],["untrained",{"2":{"256":2,"259":1}}],["until",{"2":{"64":1}}],["uncertain",{"2":{"250":1}}],["unchanged",{"2":{"62":1}}],["unable",{"2":{"173":1}}],["unnecessary",{"2":{"146":1}}],["unnormalized",{"2":{"69":1}}],["un",{"2":{"102":1}}],["unexpected",{"2":{"83":1}}],["unreleased",{"2":{"71":1}}],["unreasonably",{"2":{"62":1}}],["unrolls",{"2":{"62":1}}],["unroll",{"2":{"13":1}}],["unwrapped",{"2":{"55":1}}],["unwrap",{"2":{"55":2}}],["unwraps",{"2":{"31":1}}],["undone",{"2":{"34":1}}],["undef",{"2":{"174":1}}],["undefined",{"2":{"8":1,"66":1}}],["underlying",{"2":{"214":1}}],["under",{"2":{"134":1}}],["understand",{"2":{"59":1,"138":1,"184":1,"211":1,"214":1}}],["understanding",{"2":{"24":2,"225":1}}],["undesirable",{"2":{"11":1}}],["unfreezes",{"2":{"31":1}}],["unfreeze",{"2":{"31":2}}],["unlike",{"2":{"25":1,"59":1}}],["unless",{"2":{"2":1,"33":1,"225":1}}],["unified",{"2":{"204":1}}],["uniformly",{"2":{"113":1}}],["uniform",{"2":{"24":6,"25":6,"63":5,"66":6,"67":4,"77":1,"140":3}}],["uninitiated",{"0":{"170":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1}}],["universal",{"2":{"68":1}}],["unity",{"2":{"254":1}}],["unitrange",{"2":{"173":1}}],["unit",{"2":{"66":1}}],["union",{"2":{"2":3,"8":1,"17":1,"24":2,"33":2,"39":1,"53":3,"66":2,"69":1}}],["unpack",{"2":{"8":1}}],["unsupported",{"2":{"8":1,"22":1}}],["unsafe",{"2":{"5":1}}],["unknown",{"2":{"3":2}}],["unknowndevice",{"2":{"3":2}}],["usr",{"2":{"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["usacases",{"2":{"87":2}}],["usage",{"2":{"31":1,"84":1}}],["usual",{"2":{"69":3,"148":1,"153":1}}],["usually",{"2":{"53":2,"59":1,"70":1,"171":1,"208":1}}],["us",{"2":{"27":2,"30":1,"108":1,"109":1,"110":6,"134":1,"141":2,"142":3,"144":1,"170":1,"174":1,"177":1,"178":1,"179":3,"183":1,"219":1,"256":1,"257":1,"259":2}}],["usecases",{"2":{"55":1,"58":1,"153":1,"238":1}}],["uses",{"2":{"16":1,"18":2,"37":1,"55":4,"56":1,"58":1,"69":1,"95":1,"144":1,"147":1,"155":1,"158":2,"161":1,"174":1,"204":1,"254":2,"255":1,"256":1,"258":1}}],["userbase",{"2":{"37":1}}],["user",{"2":{"7":2,"8":2,"13":2,"58":1,"62":1,"81":1,"119":1,"134":1,"136":2,"147":1}}],["users",{"2":{"6":1,"7":2,"15":1,"31":1,"41":2,"42":1,"52":1,"55":1,"81":1,"83":1,"95":1,"98":1,"155":1,"173":1,"211":2,"214":1,"220":1}}],["useful",{"2":{"3":1,"20":1,"24":3,"33":2,"53":1,"54":1,"57":1,"59":2,"74":1,"108":1,"109":1}}],["use",{"0":{"83":1,"137":1},"2":{"2":1,"4":2,"5":1,"13":3,"15":1,"17":1,"18":2,"20":1,"24":1,"27":2,"28":1,"30":1,"31":2,"33":1,"41":4,"47":2,"52":4,"59":1,"63":11,"66":18,"67":16,"69":3,"71":1,"74":1,"77":2,"78":1,"83":1,"84":2,"87":3,"89":1,"98":1,"102":2,"103":4,"104":1,"105":1,"106":3,"107":2,"108":1,"109":1,"113":2,"117":1,"126":1,"131":1,"137":1,"144":3,"145":1,"146":1,"147":2,"149":2,"153":4,"155":2,"158":1,"159":1,"163":1,"164":2,"167":2,"169":1,"170":1,"171":1,"174":1,"175":1,"176":1,"177":2,"179":3,"183":2,"184":3,"186":2,"190":2,"192":1,"211":1,"213":1,"214":1,"215":2,"217":2,"234":1,"235":1,"236":1,"238":1,"242":1,"244":4,"247":1,"248":1,"249":5,"250":1,"254":1,"256":5}}],["used",{"2":{"2":2,"3":6,"8":2,"17":3,"24":1,"33":2,"34":2,"37":1,"39":2,"46":1,"52":2,"53":5,"55":1,"58":1,"59":11,"63":3,"64":3,"66":1,"67":1,"68":1,"69":7,"70":1,"95":1,"99":1,"102":1,"103":1,"105":1,"106":1,"116":1,"130":1,"131":2,"144":1,"145":2,"155":1,"166":1,"167":1,"169":1,"171":1,"218":1,"252":1}}],["using",{"0":{"38":1,"48":1,"106":1,"107":1,"124":1,"146":1,"150":1,"151":1,"152":1,"185":1,"204":1,"211":1,"218":1,"230":1},"1":{"39":1,"107":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"2":1,"3":1,"4":2,"5":2,"7":2,"20":3,"24":2,"27":2,"31":1,"32":1,"37":3,"39":4,"44":1,"45":1,"46":1,"48":2,"49":1,"52":7,"53":4,"55":1,"59":9,"62":11,"66":6,"67":1,"68":2,"69":1,"71":2,"76":1,"77":3,"78":4,"81":1,"82":1,"83":2,"95":1,"100":1,"105":1,"106":8,"107":2,"109":1,"112":1,"116":1,"130":3,"132":2,"133":2,"136":1,"137":1,"138":2,"142":2,"144":4,"145":6,"146":1,"147":1,"148":2,"149":5,"152":14,"153":3,"155":1,"156":1,"163":1,"170":1,"171":2,"174":1,"177":2,"178":1,"179":1,"180":1,"181":2,"183":1,"185":1,"188":1,"189":1,"190":1,"191":2,"196":1,"197":1,"204":1,"205":1,"208":1,"210":1,"212":2,"214":1,"218":1,"220":2,"221":1,"229":1,"231":1,"233":1,"237":1,"238":3,"239":1,"240":1,"241":1,"242":1,"246":1,"247":2,"249":1,"250":1,"251":1,"253":1,"255":1,"256":1,"260":1}}],["w=w",{"2":{"242":1}}],["w=gv∥v∥weight",{"2":{"69":1}}],["w=",{"2":{"59":1}}],["w=rand",{"2":{"59":1}}],["w=ones",{"2":{"59":3}}],["w3",{"2":{"59":3,"78":3}}],["w3=dense",{"2":{"59":1,"78":1}}],["w2",{"2":{"59":3,"78":3}}],["w2=",{"2":{"59":1,"78":1}}],["w1",{"2":{"59":3,"78":3}}],["w1=dense",{"2":{"59":1,"78":1}}],["w",{"2":{"59":21,"65":6,"67":1,"70":4,"78":4,"142":2,"158":1,"171":1,"179":5,"192":1,"213":1,"241":4,"242":10}}],["write",{"2":{"171":2,"173":1,"179":1,"225":1}}],["writing",{"0":{"123":1},"2":{"102":1,"124":1,"138":1}}],["written",{"2":{"37":1}}],["wrote",{"0":{"81":1},"1":{"82":1,"83":1},"2":{"203":1}}],["wrapping",{"2":{"95":1}}],["wrappers",{"0":{"27":1}}],["wrapper",{"2":{"14":1,"27":2,"33":1,"58":1,"66":1,"141":2,"148":1,"236":1,"241":1}}],["wrappedlayer",{"2":{"64":1}}],["wrappedfunction",{"2":{"63":1,"68":2,"110":3,"220":9,"256":1}}],["wrapped",{"2":{"7":3,"33":1,"62":8,"68":2,"141":1}}],["wrap",{"2":{"45":1,"108":1,"113":1,"141":1,"207":1}}],["wraps",{"2":{"39":1,"42":2,"66":2,"68":1}}],["wrt",{"2":{"27":2,"49":1,"52":3,"53":2,"54":1,"114":1,"148":2,"242":1}}],["wu",{"2":{"19":1}}],["wide",{"2":{"175":1}}],["wider",{"2":{"83":1,"93":1,"97":1,"177":1}}],["width",{"2":{"63":1}}],["wio",{"2":{"66":1}}],["wio×x+who×hprev+bo",{"2":{"66":1}}],["wig",{"2":{"66":1}}],["wig×x+whg×hprev+bg",{"2":{"66":1}}],["wif",{"2":{"66":1}}],["wif×x+whf×hprev+bf",{"2":{"66":1}}],["wii",{"2":{"66":1}}],["wii×x+whi×hprev+bi",{"2":{"66":1}}],["wiz",{"2":{"66":1}}],["wiz×x+biz+whz×hprev+bhz",{"2":{"66":1}}],["wir",{"2":{"66":1}}],["wir×x+bir+whr×hprev+bhr",{"2":{"66":1}}],["win",{"2":{"66":1}}],["win×x+bin+r⋅",{"2":{"66":1}}],["window",{"2":{"65":18}}],["will",{"2":{"3":2,"5":1,"11":1,"19":1,"24":3,"28":4,"30":1,"33":2,"34":3,"37":4,"39":1,"44":2,"48":2,"49":1,"50":4,"52":1,"55":4,"57":4,"58":1,"59":8,"62":5,"64":2,"66":3,"68":1,"69":3,"71":1,"77":1,"78":1,"95":1,"106":5,"108":1,"109":1,"110":2,"120":1,"125":1,"130":1,"133":1,"134":2,"138":1,"141":1,"144":2,"145":3,"147":1,"149":2,"150":1,"153":8,"155":2,"157":1,"167":1,"169":1,"171":1,"172":1,"173":1,"174":2,"175":4,"176":1,"177":1,"179":2,"181":2,"183":3,"184":6,"186":1,"190":2,"196":2,"203":1,"204":2,"207":4,"208":1,"214":1,"217":1,"230":1,"234":1,"235":1,"236":2,"238":3,"240":1,"241":1,"242":2,"243":3,"247":2,"249":4,"256":1}}],["withgradient",{"2":{"59":1}}],["within",{"2":{"17":2,"19":2,"59":1,"145":2}}],["without",{"2":{"2":1,"6":1,"59":1,"83":2,"95":2,"106":1,"107":1,"138":1,"145":2,"173":1,"184":1,"208":2}}],["with",{"0":{"190":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1},"2":{"1":1,"2":1,"3":1,"5":2,"6":1,"7":2,"8":1,"10":2,"11":1,"13":7,"15":1,"16":2,"18":1,"20":1,"24":5,"28":2,"31":5,"32":2,"33":3,"34":1,"37":1,"39":1,"42":2,"45":1,"46":2,"48":2,"49":1,"52":2,"53":6,"54":2,"55":2,"56":3,"57":1,"59":3,"62":14,"63":11,"64":6,"65":12,"66":10,"67":12,"68":5,"69":3,"70":5,"78":3,"81":1,"83":7,"85":2,"87":1,"88":2,"89":2,"92":1,"93":1,"95":2,"102":3,"104":1,"105":1,"106":2,"107":1,"108":3,"109":1,"110":1,"113":2,"115":2,"116":1,"117":1,"124":2,"131":2,"133":1,"138":1,"140":1,"141":1,"142":6,"144":3,"145":1,"146":4,"147":1,"150":1,"151":1,"152":1,"153":5,"155":1,"157":1,"158":6,"163":1,"167":2,"168":2,"169":2,"170":1,"171":5,"172":2,"173":1,"174":2,"175":1,"177":2,"179":4,"181":1,"183":1,"185":1,"186":1,"187":1,"192":1,"194":1,"195":1,"203":1,"204":2,"205":1,"208":1,"213":1,"214":3,"215":1,"216":1,"224":1,"225":2,"226":1,"227":1,"232":1,"239":1,"240":1,"242":2,"247":2,"248":1,"249":6,"250":2,"254":5,"256":2,"257":2}}],["what",{"2":{"109":1,"110":1,"136":1,"140":1,"142":1,"148":1,"165":1,"171":2}}],["whatever",{"2":{"52":1,"68":1}}],["why",{"0":{"81":1,"83":1},"1":{"82":1,"83":1},"2":{"140":1}}],["whole",{"2":{"69":2}}],["whose",{"2":{"67":1,"69":1}}],["who",{"2":{"66":1}}],["whg",{"2":{"66":1}}],["whf",{"2":{"66":1}}],["whn",{"2":{"66":1}}],["whn×hprev+bhn",{"2":{"66":1}}],["whz",{"2":{"66":1}}],["whr",{"2":{"66":1}}],["whcn",{"2":{"63":1,"64":2,"69":3}}],["whi",{"2":{"66":1}}],["while",{"2":{"3":1,"32":1,"39":1,"58":1,"62":1,"70":1,"83":1,"107":1,"128":1,"138":1,"140":1,"155":1,"177":1}}],["which",{"0":{"119":1},"2":{"2":1,"3":1,"7":1,"19":1,"24":1,"30":1,"31":14,"34":1,"52":1,"53":5,"58":1,"59":5,"62":3,"63":1,"65":3,"66":1,"68":3,"69":4,"83":2,"84":1,"104":1,"105":1,"108":2,"136":1,"138":1,"140":1,"141":2,"142":1,"149":1,"155":1,"170":1,"171":2,"173":1,"179":1,"185":1,"208":1,"214":1,"218":1,"236":1,"247":1,"249":1,"252":1,"255":1,"256":1}}],["whether",{"2":{"33":2,"39":1,"64":1,"183":1}}],["whereas",{"2":{"63":1}}],["where",{"2":{"3":1,"13":2,"16":1,"18":1,"19":1,"20":1,"24":7,"32":1,"33":1,"53":4,"58":1,"63":5,"65":6,"66":6,"67":4,"68":3,"69":3,"70":1,"89":1,"110":2,"134":1,"159":1,"176":1,"184":1,"185":1,"224":1,"249":2,"250":2,"254":3}}],["when",{"2":{"3":2,"16":1,"24":2,"39":2,"48":1,"53":2,"54":1,"59":2,"63":1,"67":1,"68":1,"70":1,"88":1,"115":1,"127":1,"140":2,"144":1,"156":1,"165":1,"175":2,"184":1,"196":1,"214":1,"254":1,"257":1}}],["world",{"2":{"208":1}}],["worthwhile",{"2":{"83":1}}],["word",{"2":{"67":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["words",{"2":{"63":1}}],["workaround",{"2":{"153":1}}],["worker",{"2":{"160":1}}],["workers",{"2":{"43":3,"44":5}}],["worked",{"2":{"93":1,"147":1}}],["working",{"2":{"33":1,"85":1,"87":2,"89":1,"106":1,"116":1,"117":1,"131":1,"169":1,"248":1}}],["work",{"2":{"2":1,"5":1,"31":1,"68":1,"70":1,"74":1,"88":1,"89":1,"111":1,"113":1,"116":1,"119":1,"134":1,"138":1,"144":1,"167":1,"171":2,"176":1}}],["works",{"2":{"2":1,"33":1,"37":1,"55":1,"58":1,"70":2,"83":1,"89":1,"100":1,"115":1,"124":1,"140":2,"142":1,"144":1,"145":1,"153":1,"160":1,"171":1,"214":1}}],["would",{"2":{"7":2,"8":1,"63":2,"68":3,"81":1,"83":1,"95":1,"106":2,"109":1,"127":1,"134":1}}],["wondered",{"2":{"83":1}}],["won",{"2":{"2":1,"8":1,"59":1,"98":1,"120":1,"184":1,"204":1}}],["waveform",{"2":{"254":5,"255":7,"256":9,"257":4,"259":12}}],["waveforms",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["wall",{"2":{"249":1}}],["wan",{"2":{"144":1}}],["wants",{"2":{"136":1}}],["wanted",{"2":{"83":1}}],["want",{"0":{"72":1,"73":1},"2":{"8":2,"52":2,"71":1,"78":1,"107":1,"120":1,"127":1,"138":1,"145":1,"148":1,"171":1,"177":1,"204":1,"207":1,"243":1}}],["warmup",{"2":{"195":1,"257":1}}],["warde",{"2":{"68":1}}],["warntype",{"2":{"220":3}}],["warn",{"2":{"57":3,"60":1}}],["warning",{"2":{"2":3,"3":1,"4":2,"8":1,"16":1,"27":2,"37":3,"39":2,"47":1,"52":1,"53":1,"57":5,"58":1,"63":1,"69":1,"125":1,"131":1,"144":1,"145":2,"153":1,"202":1,"203":1,"244":4}}],["way",{"2":{"17":2,"37":3,"39":2,"62":1,"83":1,"108":1,"138":2,"150":1,"161":1,"185":2,"249":1}}],["wasteful",{"2":{"140":1}}],["was",{"2":{"2":1,"5":3,"7":1,"11":1,"24":1,"34":1,"37":1,"63":1,"95":3,"97":1,"98":1,"104":1,"106":1,"115":2,"155":2,"180":1,"189":1,"197":1,"208":1,"210":1,"214":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["welcome",{"2":{"144":1}}],["well",{"2":{"39":1,"53":1,"81":1,"85":1,"87":1,"106":1,"145":1,"146":1,"170":1}}],["weird",{"2":{"140":1,"142":2}}],["weightnorm",{"2":{"69":4,"83":1,"140":1}}],["weightih×x+biasih+weighthh×hprev+biashh",{"2":{"66":1}}],["weighting",{"2":{"53":1}}],["weightinitializers",{"0":{"21":1,"97":1},"1":{"22":1,"23":1,"24":1,"25":1},"2":{"3":1,"24":8,"25":24,"140":3,"168":2,"256":1}}],["weight=truncated",{"2":{"256":3}}],["weight=l",{"2":{"140":1}}],["weight=randn32",{"2":{"104":1}}],["weight=rand32",{"2":{"67":1,"104":1}}],["weight=ones32",{"2":{"67":1}}],["weight=glorot",{"2":{"63":1,"140":1}}],["weight=nothing",{"2":{"63":1,"66":3,"67":3}}],["weight=ps",{"2":{"59":1}}],["weight=zero",{"2":{"32":1}}],["weights",{"0":{"168":1},"1":{"169":1},"2":{"53":1,"66":4,"168":10,"169":5,"225":1,"249":2,"250":3}}],["weight",{"2":{"16":4,"18":4,"21":1,"24":3,"31":3,"32":4,"34":4,"63":6,"66":12,"67":24,"69":5,"77":15,"106":12,"110":4,"122":2,"126":4,"127":2,"128":4,"140":6,"142":4,"145":2,"147":4,"148":4,"168":1,"179":1,"220":30,"225":5,"226":2,"235":2,"249":1,"256":3,"258":3}}],["weak",{"2":{"47":1}}],["weren",{"2":{"102":1}}],["were",{"2":{"11":1,"90":1,"97":1,"102":2,"144":1}}],["we",{"0":{"81":1,"137":1},"1":{"82":1,"83":1},"2":{"2":4,"3":3,"7":2,"8":4,"13":2,"16":2,"18":3,"30":2,"37":1,"42":1,"47":1,"52":2,"53":2,"55":5,"57":1,"59":3,"62":3,"63":4,"66":6,"67":4,"77":4,"78":2,"83":4,"85":1,"87":2,"88":3,"89":3,"90":2,"95":1,"97":1,"98":2,"102":1,"103":1,"104":1,"106":13,"107":2,"109":3,"110":6,"111":2,"114":1,"116":3,"117":2,"119":1,"120":3,"124":1,"125":1,"126":1,"128":1,"133":2,"134":2,"137":1,"138":1,"140":5,"141":4,"142":4,"143":1,"144":3,"145":2,"146":3,"147":3,"148":3,"149":3,"152":2,"153":11,"155":3,"159":1,"160":1,"170":1,"171":11,"173":4,"174":9,"175":6,"177":2,"179":3,"181":1,"183":3,"184":11,"185":2,"186":1,"187":1,"188":3,"190":2,"193":2,"196":3,"203":1,"204":4,"207":4,"208":5,"211":1,"214":3,"217":1,"219":1,"220":2,"225":2,"230":1,"234":1,"235":2,"236":2,"238":3,"240":2,"241":2,"242":3,"243":3,"247":3,"248":1,"249":8,"250":7,"254":3,"256":9,"257":1}}],[">randn32",{"2":{"59":2}}],[">",{"2":{"2":3,"3":8,"8":2,"24":8,"25":24,"53":3,"59":2,"66":1,"106":1,"110":2,"121":1,"133":2,"134":4,"145":2,"146":2,"147":2,"148":2,"152":1,"186":1,"232":1,"236":1,"244":2,"254":1,"258":1}}],["cycle",{"2":{"145":2,"244":2}}],["cdev",{"2":{"131":1,"132":1,"205":1,"209":1,"239":1,"244":2}}],["cdims",{"2":{"16":3}}],["circle",{"2":{"255":1,"256":2,"259":4}}],["circumvented",{"2":{"132":1}}],["circular",{"2":{"24":1,"63":1}}],["ci",{"2":{"83":1,"202":1}}],["cite",{"2":{"74":2}}],["citation",{"0":{"74":1}}],["cnew",{"2":{"66":2}}],["cnew=f⋅cprev+i⋅ghnew=o⋅tanh",{"2":{"66":1}}],["cvpr",{"2":{"53":1}}],["cluster",{"2":{"250":1}}],["clockwise",{"2":{"181":1,"183":4}}],["closest",{"2":{"122":1}}],["closures",{"2":{"59":1}}],["cl",{"2":{"141":3,"168":4}}],["classifying",{"2":{"250":1}}],["classify",{"2":{"181":1,"248":1}}],["classifier=st",{"2":{"184":1}}],["classifier",{"0":{"184":1},"2":{"184":11,"185":3,"249":1}}],["classifiers",{"2":{"53":2}}],["classified",{"2":{"53":1}}],["classification",{"0":{"190":1,"211":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1},"2":{"24":2,"53":2,"247":1}}],["classes",{"2":{"53":1}}],["class",{"0":{"88":1},"2":{"53":1,"86":1,"89":1,"194":5,"216":5,"227":5,"247":1}}],["clear",{"2":{"34":1,"110":1}}],["c",{"2":{"32":3,"63":4,"65":21,"66":1,"70":5,"184":2,"192":1,"213":1,"220":9,"250":2,"254":4}}],["cell=st",{"2":{"184":1}}],["cell`",{"2":{"184":1}}],["cells",{"2":{"66":1,"104":1}}],["cell",{"2":{"66":45,"184":10,"185":4}}],["central",{"2":{"24":1,"220":13}}],["certain",{"0":{"135":1},"1":{"136":1},"2":{"7":1,"8":2,"13":1,"54":1,"64":2,"81":1,"95":1,"102":1,"106":1,"113":1,"117":1,"125":1,"155":1,"167":1,"214":1}}],["ca",{"2":{"208":2,"225":3}}],["cairomakie",{"2":{"205":1,"206":1,"209":1,"231":1,"232":1,"236":1,"239":1,"245":2,"247":1,"248":1,"253":1,"255":1,"256":1,"259":2}}],["capabilities",{"2":{"172":1,"238":1}}],["capture",{"2":{"144":1}}],["causing",{"2":{"155":1}}],["causes",{"2":{"117":1}}],["cause",{"2":{"8":1,"13":1,"34":1,"155":1}}],["cat",{"2":{"121":1,"183":1}}],["catch",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"52":1,"109":2,"110":2,"122":1,"157":2}}],["care",{"2":{"174":1}}],["cartesian",{"2":{"67":1}}],["carry",{"2":{"66":4,"184":3,"185":3}}],["caching",{"2":{"52":1}}],["cached",{"2":{"52":2}}],["cache",{"2":{"52":2,"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["case",{"2":{"31":1,"57":1,"66":19,"115":1,"137":1,"140":1,"145":1,"179":1,"208":1,"225":1,"250":1,"256":1}}],["cases",{"2":{"8":3,"24":1,"30":1,"37":1,"52":2,"55":6,"57":1,"58":1,"59":1,"71":1,"83":1,"88":1,"89":1,"111":2,"133":1,"146":1,"153":1,"159":1,"173":1}}],["caveats",{"2":{"24":1}}],["calculation",{"2":{"145":1,"146":3}}],["calculations",{"2":{"39":1}}],["calculates",{"2":{"65":3}}],["calculate",{"2":{"63":4,"65":3,"67":1}}],["calculated",{"2":{"7":2,"17":1,"53":7}}],["caller",{"2":{"244":4}}],["called",{"2":{"3":1,"17":2,"19":2,"42":1,"62":2,"64":3,"127":1,"134":2,"136":1,"140":1,"169":1,"171":2,"184":1}}],["callback",{"2":{"208":4,"257":3,"258":1}}],["callable",{"2":{"62":2}}],["calls",{"2":{"7":1,"8":2,"144":2}}],["call",{"2":{"7":1,"8":1,"17":1,"32":1,"48":6,"58":1,"64":5,"69":3,"77":1,"83":1,"144":1,"145":2,"153":1,"168":4,"174":2,"184":1}}],["calling",{"2":{"5":1,"7":1,"8":1,"42":1,"48":1,"107":1,"113":1,"116":1,"249":1}}],["candidates",{"2":{"122":1}}],["cannot",{"2":{"24":1,"59":1,"82":1,"134":1,"140":1,"141":1,"153":1,"202":1}}],["can",{"0":{"137":1},"2":{"2":2,"3":4,"4":2,"7":3,"8":3,"11":1,"13":1,"15":1,"16":1,"17":2,"18":1,"19":14,"20":1,"31":1,"33":1,"42":3,"48":1,"49":1,"52":1,"53":2,"55":1,"58":1,"59":5,"60":1,"62":6,"63":6,"65":3,"66":7,"67":3,"68":3,"71":2,"76":2,"77":3,"78":1,"79":1,"103":1,"106":2,"108":2,"109":1,"110":2,"113":1,"124":1,"126":1,"128":1,"130":1,"132":1,"134":1,"136":1,"137":1,"141":1,"142":3,"144":2,"145":2,"146":3,"148":1,"149":2,"153":1,"155":2,"156":1,"157":1,"160":1,"163":1,"166":1,"167":1,"168":1,"169":1,"171":11,"172":2,"174":1,"176":1,"179":1,"184":1,"185":1,"187":1,"188":1,"208":1,"214":2,"217":1,"218":1,"226":1,"240":1,"241":1,"249":2,"250":3,"254":1,"256":1}}],["coordinates",{"2":{"250":1}}],["cos",{"2":{"147":1,"243":1,"254":2,"255":3,"256":8}}],["cover",{"2":{"133":1}}],["covered",{"2":{"111":1}}],["covariate",{"2":{"19":1}}],["could",{"2":{"140":1,"141":1,"249":1}}],["course",{"2":{"124":1,"145":1}}],["courville",{"2":{"68":1}}],["coupled",{"2":{"81":1,"167":1}}],["counterpart",{"2":{"137":1}}],["count",{"2":{"59":1}}],["colormap=",{"2":{"250":3}}],["colorbar",{"2":{"245":1}}],["color=",{"2":{"206":2,"209":4,"232":2,"236":2,"248":2}}],["columns",{"2":{"171":1}}],["column",{"2":{"67":1,"171":4}}],["collects",{"2":{"62":1}}],["collect",{"2":{"59":1,"78":1,"183":2,"192":2,"213":2,"232":1,"243":4,"245":1,"250":4}}],["cols",{"2":{"24":1}}],["coefficient",{"2":{"53":2}}],["corner",{"2":{"70":1}}],["corners",{"2":{"70":1,"104":1}}],["corners=false",{"2":{"70":1}}],["cores",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["core",{"2":{"37":1,"48":3,"83":1,"133":1,"173":1,"180":1,"189":1,"197":1,"210":1,"220":2,"221":1,"225":5,"226":3,"229":1,"237":1,"244":4,"246":1,"251":1,"260":1}}],["correlation=true",{"2":{"63":2,"103":1,"105":1}}],["correlation=false",{"2":{"63":2}}],["correlation",{"2":{"63":7}}],["corresponding",{"2":{"4":3,"32":2,"34":2,"53":3,"67":1,"70":2,"99":1,"102":1,"131":1,"146":1}}],["corresponds",{"2":{"2":1,"208":1}}],["corrections",{"2":{"124":1}}],["correctness",{"0":{"49":1},"2":{"47":1,"55":1}}],["correct",{"0":{"123":1},"2":{"3":2,"49":1,"155":2,"194":3,"216":3,"227":3}}],["correctly",{"2":{"2":1,"33":1,"111":1,"113":1}}],["codebases",{"2":{"58":1}}],["code",{"0":{"113":1},"2":{"30":2,"37":1,"39":1,"55":4,"59":4,"78":1,"87":1,"90":1,"106":1,"113":1,"130":1,"136":1,"137":1,"143":1,"145":2,"155":1,"156":1,"157":1,"161":1,"170":1,"185":1,"187":1,"220":3,"248":1,"252":2}}],["combination",{"2":{"144":1}}],["combined",{"2":{"66":2,"77":1}}],["combines",{"2":{"62":1}}],["come",{"2":{"109":1,"155":1,"177":1}}],["comes",{"2":{"62":1,"81":1,"83":2,"145":1,"147":1}}],["coming",{"2":{"78":1,"140":1,"155":1}}],["com",{"2":{"66":1,"71":1}}],["community",{"2":{"203":1}}],["communication",{"0":{"44":1},"2":{"115":1,"116":1}}],["communications",{"2":{"42":2}}],["commit",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["command",{"2":{"71":2,"161":2}}],["comm",{"2":{"41":2}}],["common",{"2":{"21":1,"34":1,"55":4,"141":1,"160":1,"166":1,"168":1}}],["comprises",{"2":{"141":1}}],["compilable",{"2":{"106":1}}],["compilation",{"2":{"106":1}}],["compiling",{"0":{"106":1},"1":{"107":1},"2":{"83":1}}],["compiled",{"2":{"87":1,"106":7}}],["compiler",{"2":{"81":2,"106":1}}],["compile=true",{"2":{"52":1}}],["compile",{"2":{"3":1,"48":1,"52":1,"62":1,"106":8}}],["completeness",{"2":{"152":1,"217":1}}],["completely",{"2":{"66":1,"81":1,"102":1}}],["complete",{"2":{"65":3,"66":1,"168":1}}],["complexity",{"2":{"84":1,"175":1}}],["complexf64",{"2":{"25":8}}],["complexf32",{"2":{"25":8,"168":2}}],["complexf16",{"2":{"25":8}}],["complex",{"2":{"5":1,"52":1,"59":1,"168":1}}],["compare",{"2":{"114":1,"152":1}}],["comparison",{"2":{"62":1}}],["compactmacroimpl",{"2":{"220":3}}],["compactluxlayer",{"2":{"59":2,"220":3,"225":2}}],["compact",{"0":{"59":1,"185":1},"2":{"58":1,"59":20,"78":3,"103":2,"138":2,"144":2,"185":2,"187":1,"214":4,"218":1,"220":7,"225":1}}],["compatibility",{"2":{"37":1,"89":1,"138":1}}],["compatible",{"2":{"6":1,"16":1,"18":1,"33":2,"45":1,"46":2,"53":2,"138":1,"144":1,"153":2,"217":1}}],["components",{"2":{"254":4}}],["component",{"2":{"254":7}}],["componentvector",{"2":{"179":1,"220":6,"258":1}}],["componentarray",{"2":{"34":1,"145":2,"146":2,"147":2,"148":3,"152":4,"179":1,"208":2,"215":1,"217":1,"225":2,"256":1}}],["componentarrays",{"2":{"34":2,"142":1,"144":1,"175":1,"205":1,"212":1,"220":12,"223":1,"249":1,"253":1}}],["composability",{"2":{"247":1}}],["composes",{"2":{"85":1}}],["composedlinear",{"2":{"141":3}}],["composed",{"2":{"52":1,"66":1}}],["compose",{"2":{"39":1,"249":1}}],["composition",{"2":{"19":1,"141":1}}],["computing",{"0":{"148":1,"150":1,"151":1,"152":1},"2":{"27":2,"28":1,"146":1,"150":1,"176":1}}],["computed",{"2":{"17":1,"19":1,"52":3,"53":3,"62":4,"63":2,"67":1,"68":1,"69":2,"176":4,"235":1}}],["computes",{"2":{"14":1,"16":1,"19":3,"24":1,"28":1,"52":2,"69":4,"134":1,"149":1}}],["compute",{"2":{"13":2,"18":1,"27":6,"28":1,"52":10,"53":1,"77":3,"95":1,"104":2,"106":1,"110":1,"148":3,"149":5,"152":1,"175":1,"176":1,"177":1,"178":1,"186":1,"195":1,"208":1,"240":1,"242":2,"249":1,"254":4,"255":1,"256":1,"257":1,"259":1}}],["computer",{"2":{"5":1,"19":1,"24":2,"53":4}}],["computationally",{"2":{"175":1}}],["computational",{"2":{"175":1}}],["computation",{"0":{"145":1,"147":1},"1":{"146":1},"2":{"3":1,"13":1,"49":1,"53":1,"62":2,"69":2,"95":1,"160":2}}],["copying",{"2":{"184":1}}],["copyto",{"2":{"55":3}}],["copy",{"2":{"8":1,"52":2,"55":1,"134":2,"173":5}}],["copied",{"2":{"5":1,"66":4}}],["concise",{"2":{"138":1,"185":1}}],["concatenate",{"2":{"121":1}}],["concatenated",{"2":{"66":7}}],["conclusion",{"0":{"111":1}}],["concrete",{"0":{"120":1},"1":{"121":1,"122":1,"123":1,"124":1},"2":{"90":1,"134":1,"142":1}}],["conjunction",{"2":{"83":1}}],["connection",{"2":{"62":26,"66":4}}],["connected",{"0":{"18":1},"2":{"24":1,"67":3}}],["confusing",{"2":{"208":1}}],["confusion",{"2":{"95":1}}],["conform",{"2":{"62":1}}],["conference",{"2":{"19":2,"24":6,"53":4}}],["convtranspose",{"2":{"63":1,"104":1,"105":2}}],["convolutions",{"2":{"63":5}}],["convolution",{"2":{"24":1,"63":9}}],["convolutional",{"0":{"16":1,"63":1},"2":{"24":2,"53":1,"63":4}}],["convdims",{"2":{"16":2}}],["conv",{"2":{"16":3,"39":2,"63":4,"103":1,"104":1,"133":1,"158":2,"193":6}}],["convention",{"2":{"173":1}}],["conveniently",{"2":{"171":1}}],["convenience",{"0":{"25":1},"2":{"7":1,"58":1,"85":1,"236":1}}],["conversely",{"2":{"175":1}}],["converse",{"2":{"63":1}}],["conversions",{"2":{"57":2}}],["conversion",{"0":{"165":1},"2":{"5":1}}],["converts",{"2":{"56":3,"70":1}}],["converted",{"2":{"39":2,"57":1}}],["converting",{"2":{"34":1,"39":3}}],["convert",{"2":{"5":1,"8":1,"37":2,"39":4,"57":3,"134":1,"137":1,"153":1,"193":1,"208":1,"209":1,"214":1}}],["cond",{"2":{"8":3}}],["conditions",{"2":{"196":1,"242":1,"255":1,"256":1}}],["condition",{"2":{"8":2,"24":1}}],["continua",{"2":{"254":1}}],["continue",{"2":{"116":1}}],["contour",{"2":{"245":1,"250":6}}],["contrib",{"2":{"109":1}}],["control",{"2":{"70":2,"106":2,"155":1,"170":1}}],["controlled",{"2":{"57":1,"82":1,"105":1}}],["controlling",{"2":{"57":1,"165":1,"166":1}}],["controls",{"2":{"24":1,"53":1,"63":4,"66":3,"69":8}}],["contrastive",{"2":{"53":1}}],["contrast",{"2":{"2":1}}],["context",{"2":{"17":1,"19":2,"54":1}}],["context`",{"2":{"17":1}}],["contained",{"2":{"62":1}}],["containerlayer",{"2":{"214":1}}],["containers",{"0":{"62":1},"2":{"95":2,"220":3}}],["container",{"0":{"141":1},"2":{"7":2,"8":1,"33":1,"46":1,"96":1,"113":1,"141":2,"184":2}}],["contains",{"0":{"147":1},"2":{"8":1,"28":1,"54":2,"64":1,"68":1}}],["contain",{"2":{"7":1,"57":1,"59":1,"83":1,"140":2,"184":1}}],["containing",{"0":{"145":1},"1":{"146":1},"2":{"3":1,"7":3,"19":2,"24":4,"25":24,"52":2,"53":2,"66":14,"67":1,"82":1}}],["consensus",{"2":{"147":1}}],["consecutive",{"2":{"62":1}}],["consequence",{"2":{"32":1}}],["consult",{"2":{"29":1}}],["considering",{"2":{"175":1}}],["consider",{"2":{"120":1,"140":1,"142":1,"155":1,"176":1,"179":1}}],["considered",{"2":{"3":1,"4":2,"49":1,"52":1,"125":1,"130":1}}],["consistent",{"2":{"66":1}}],["consistency",{"2":{"24":1}}],["consists",{"2":{"62":1,"146":1}}],["constrained",{"2":{"143":1}}],["constructured",{"2":{"134":1}}],["constructing",{"2":{"52":1,"113":1}}],["construction",{"2":{"7":1,"70":1,"102":1}}],["constructor",{"2":{"31":1}}],["construct",{"2":{"24":1,"31":1,"33":2,"59":2,"77":2,"82":1,"107":1,"108":1,"141":2,"208":1,"215":1,"249":2}}],["constructed",{"2":{"17":1,"59":1,"140":1}}],["constructs",{"2":{"7":1,"24":2,"31":1}}],["const",{"2":{"72":4,"73":3,"106":5,"110":1,"205":2,"206":2,"220":2,"235":2,"239":2,"254":6,"255":1,"256":2}}],["constants",{"2":{"17":1,"255":1,"256":1}}],["constant",{"2":{"3":1}}],["cupti",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["cusparse",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["cusolver",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["customparamtype",{"2":{"31":2}}],["customabstractluxlayer",{"2":{"7":4}}],["customize",{"2":{"3":1}}],["custom",{"0":{"78":1,"118":1,"134":1},"1":{"119":1,"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"3":1,"7":2,"8":1,"11":1,"20":1,"31":1,"59":1,"78":1,"83":1,"110":2,"123":1,"138":1,"142":1,"144":1,"163":1,"173":1,"181":1,"184":1,"185":1,"214":1,"238":1,"242":1}}],["cufft",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["curand",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["currently",{"2":{"4":4,"18":2,"20":1,"33":1,"65":3,"70":1,"88":2,"89":1,"106":1,"107":1,"117":1,"130":1,"144":1,"153":2,"177":1}}],["current",{"2":{"2":1,"5":1,"7":1,"31":2,"52":1,"64":1,"85":1,"120":1,"173":1,"249":1,"250":1}}],["cublas",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["cublaslt",{"2":{"18":1}}],["cu",{"2":{"172":4}}],["cuarray",{"2":{"5":3,"22":3,"77":18,"131":1,"153":2,"168":2,"172":1,"220":18}}],["cuiterator",{"2":{"5":2,"100":1,"160":1}}],["cudevice",{"2":{"4":1}}],["cudnn",{"2":{"2":2,"3":1,"16":1,"72":1}}],["cudadevice",{"2":{"2":2,"4":2,"5":1,"78":1,"99":1,"116":1,"132":2,"180":1,"189":1,"197":1,"207":2,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["cuda",{"0":{"172":1},"2":{"2":6,"3":2,"4":1,"5":5,"16":1,"18":1,"22":1,"42":7,"72":1,"77":19,"78":1,"80":1,"93":2,"100":1,"115":1,"116":1,"117":1,"130":1,"131":1,"153":2,"160":1,"163":3,"164":1,"168":5,"172":7,"180":10,"189":10,"197":3,"210":10,"212":1,"220":36,"221":10,"223":1,"229":10,"237":10,"239":1,"246":10,"251":3,"260":3}}],["ch",{"2":{"249":2,"250":1}}],["cheat",{"2":{"147":1}}],["checking",{"2":{"140":1,"220":1}}],["check=",{"2":{"110":2}}],["check",{"2":{"8":4,"33":11,"42":1,"49":1,"59":1,"64":3,"69":3,"106":1,"110":3,"152":1,"157":2,"166":3}}],["checked",{"2":{"3":1}}],["checks",{"2":{"3":2,"31":2,"33":1,"59":2}}],["chs",{"2":{"63":17,"69":19}}],["chunks",{"2":{"54":1}}],["choice",{"2":{"188":1}}],["chosen",{"2":{"169":1}}],["chopra",{"2":{"53":1}}],["choose",{"2":{"2":1}}],["christian",{"2":{"19":1}}],["chains",{"0":{"39":1},"2":{"39":4,"193":1,"196":1,"249":2}}],["chain=chain",{"2":{"32":1}}],["chain",{"0":{"121":1,"129":1},"2":{"32":10,"34":3,"37":1,"39":2,"59":1,"62":7,"66":2,"68":4,"69":6,"77":9,"102":1,"106":2,"107":1,"109":8,"110":6,"120":1,"121":2,"124":3,"126":2,"127":1,"129":2,"133":2,"145":1,"146":1,"147":1,"148":1,"152":1,"153":1,"193":6,"208":1,"215":2,"220":18,"226":2,"233":2,"236":1,"241":1,"249":5,"250":2,"256":2}}],["chainrulescore",{"2":{"110":2}}],["chainrules",{"2":{"16":1,"18":1,"33":2,"53":1,"86":1,"89":1,"144":1}}],["channel",{"2":{"65":3,"69":12,"70":3}}],["channels",{"2":{"19":1,"63":6,"64":2,"69":1,"70":1}}],["changed",{"2":{"92":1,"114":1,"136":1}}],["changes",{"0":{"92":1,"95":1,"99":1,"102":1,"103":1,"104":2},"2":{"7":1,"11":1,"60":1,"62":9,"68":2,"81":1,"90":1,"95":1,"97":1,"109":1,"114":1,"133":1,"144":1}}],["change",{"2":{"1":1,"2":1,"58":1,"95":1,"98":1,"113":1,"140":1,"217":1,"254":1}}],["crude",{"2":{"254":1}}],["crc",{"2":{"110":2}}],["critical",{"2":{"88":1,"130":1}}],["criteria",{"2":{"2":1,"33":1}}],["creating",{"0":{"184":1},"2":{"59":1,"214":1}}],["created",{"2":{"59":1}}],["create",{"0":{"215":1,"226":1},"2":{"2":1,"5":1,"37":1,"41":2,"52":1,"55":2,"59":1,"62":2,"67":3,"106":2,"107":1,"110":1,"123":1,"171":1,"174":1,"181":1,"183":3,"184":2,"187":1,"215":2,"217":1,"220":3,"226":2,"228":1,"236":1,"241":4,"249":2}}],["creates",{"2":{"1":1,"8":1,"24":1,"59":1,"66":5,"174":1,"249":1}}],["crosscor",{"2":{"63":1,"103":1}}],["crossentropyloss",{"2":{"53":6,"194":1,"216":1,"227":1}}],["cross",{"2":{"53":3,"63":11,"103":1,"105":1,"138":1}}],["cpu=true",{"2":{"217":2}}],["cpus",{"2":{"13":1,"14":2,"39":1,"155":1,"159":1}}],["cpudevice",{"2":{"2":4,"4":2,"131":3,"132":1,"205":1,"239":1}}],["cpu",{"2":{"2":4,"5":1,"18":1,"20":1,"49":1,"65":3,"73":2,"83":1,"86":1,"87":3,"89":1,"103":2,"106":1,"130":1,"131":6,"132":7,"134":1,"153":3,"167":1,"180":2,"187":1,"188":1,"189":2,"196":1,"197":2,"205":1,"210":2,"217":4,"221":2,"229":2,"235":2,"236":1,"237":2,"239":1,"246":2,"251":2,"260":2}}],["rk4",{"2":{"255":1,"256":1,"257":1,"259":1}}],["r₂",{"2":{"254":2}}],["r₁",{"2":{"254":2}}],["r2",{"2":{"254":1}}],["r1",{"2":{"254":1}}],["r=r1−r2",{"2":{"254":1}}],["r=σ",{"2":{"66":1}}],["rhat",{"2":{"249":1}}],["risk",{"2":{"117":1}}],["right",{"2":{"68":1}}],["rrule",{"2":{"110":1}}],["rrules",{"2":{"89":1}}],["rᴰ",{"2":{"70":1}}],["r²",{"2":{"70":1}}],["rgb",{"2":{"63":1}}],["r",{"2":{"59":9,"70":9,"254":3}}],["rnns",{"2":{"66":1}}],["rnncell",{"2":{"66":9,"104":1,"105":1}}],["rnn",{"2":{"54":1,"66":3}}],["rng=xoshiro",{"2":{"195":1}}],["rngs",{"2":{"82":1}}],["rng",{"0":{"22":1},"2":{"3":3,"7":2,"8":6,"9":1,"10":1,"11":2,"17":6,"22":10,"24":11,"25":24,"32":3,"37":1,"39":1,"59":5,"62":4,"64":6,"66":3,"67":4,"68":18,"77":7,"78":3,"97":2,"102":1,"106":1,"107":1,"109":3,"110":2,"122":2,"124":1,"126":4,"129":6,"133":7,"134":9,"140":9,"141":2,"142":5,"155":4,"168":13,"169":6,"170":2,"171":1,"174":5,"176":1,"177":1,"179":8,"187":2,"195":2,"207":1,"208":1,"215":4,"225":3,"228":2,"232":4,"235":1,"244":4,"248":11,"249":1,"256":1}}],["row",{"2":{"171":3,"250":1}}],["rows",{"2":{"24":1,"171":1}}],["routines",{"2":{"144":1}}],["route",{"2":{"141":1}}],["rough",{"2":{"144":1}}],["rounded",{"2":{"63":2,"65":3}}],["round",{"2":{"42":1,"228":4,"248":1}}],["root",{"2":{"44":7,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["robin",{"2":{"42":1}}],["rocarray",{"2":{"22":4}}],["rocrand",{"2":{"22":1}}],["rocm",{"2":{"3":1,"72":2,"116":1,"117":1,"163":3,"164":1}}],["ryan",{"2":{"19":1}}],["raw",{"2":{"192":2,"213":2}}],["rademacher",{"2":{"152":1}}],["ra",{"2":{"106":18,"107":9}}],["ratio≥0",{"2":{"254":1}}],["ratio≤1",{"2":{"254":1}}],["ratio",{"2":{"254":6,"255":2,"256":1,"257":1,"259":1}}],["rationale",{"2":{"95":1}}],["rate",{"2":{"179":1,"249":1}}],["rather",{"2":{"3":1,"16":1,"66":1,"82":1,"184":1}}],["raia",{"2":{"53":1}}],["ran",{"2":{"109":1}}],["randc64",{"2":{"25":1}}],["randc32",{"2":{"25":1}}],["randc16",{"2":{"25":1}}],["rand64",{"2":{"25":1}}],["rand32",{"2":{"25":1}}],["rand16",{"2":{"25":1}}],["randnc64",{"2":{"25":1}}],["randnc32",{"2":{"25":1}}],["randnc16",{"2":{"25":1}}],["randn64",{"2":{"25":1}}],["randn32",{"2":{"25":1}}],["randn16",{"2":{"25":1}}],["randn",{"2":{"24":1,"37":1,"39":1,"59":1,"62":2,"68":2,"106":1,"107":1,"109":1,"122":1,"126":1,"129":1,"131":1,"132":1,"133":2,"134":6,"140":1,"141":1,"142":1,"145":2,"146":2,"171":2,"174":1,"176":1,"177":1,"179":4,"195":1,"232":1}}],["randomness",{"0":{"174":1},"2":{"66":3,"77":1,"82":1,"149":1,"170":1,"174":1}}],["randomly",{"2":{"64":2}}],["random",{"2":{"17":4,"22":1,"24":8,"25":12,"32":2,"37":2,"39":2,"59":3,"62":2,"64":3,"68":8,"77":4,"78":3,"106":3,"107":1,"109":1,"121":1,"126":2,"129":4,"133":4,"134":4,"140":4,"142":3,"144":1,"149":1,"153":1,"155":2,"168":3,"170":5,"171":1,"174":8,"179":4,"182":1,"191":1,"205":1,"207":1,"208":1,"212":1,"215":2,"223":1,"225":1,"231":1,"232":2,"239":1,"243":1,"244":2,"247":1,"248":2,"253":1,"256":1}}],["rand",{"2":{"5":1,"59":1,"77":3,"78":1,"147":1,"148":1,"152":2,"153":1,"155":2,"171":5,"172":1,"174":3,"248":8}}],["rank",{"2":{"4":3,"43":3,"113":1,"114":1}}],["ranges",{"2":{"62":1}}],["range",{"2":{"3":2,"97":1,"177":1,"206":1,"232":1,"243":4,"250":23,"255":1}}],["rule",{"2":{"70":1,"236":2}}],["rules",{"0":{"123":1},"2":{"0":1,"55":1,"88":1}}],["runtime",{"2":{"49":2,"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["running",{"0":{"122":1},"2":{"17":2,"19":18,"69":16,"95":1,"109":5,"110":11,"126":2,"129":2,"136":1,"156":1,"184":1}}],["run",{"2":{"2":1,"3":1,"5":1,"48":1,"64":1,"71":2,"72":1,"73":1,"77":1,"102":1,"106":3,"110":3,"117":1,"123":1,"134":1,"140":1,"153":3,"155":1,"161":3,"174":1,"184":1,"202":1,"250":1}}],["red",{"2":{"206":1,"209":2,"248":1}}],["reduction",{"2":{"53":1,"65":3}}],["reduce",{"2":{"44":3,"47":1,"68":1,"249":1}}],["reducing",{"2":{"19":1,"62":1}}],["refresher",{"2":{"140":1}}],["referred",{"2":{"171":1}}],["refer",{"2":{"78":1,"203":1,"204":1,"238":1}}],["reference",{"0":{"23":1},"1":{"24":1,"25":1},"2":{"190":1}}],["references",{"2":{"17":2,"19":4,"24":6,"53":4,"68":1,"69":1}}],["reiterate",{"2":{"134":1}}],["reexport",{"2":{"102":2}}],["renamed",{"2":{"95":2,"98":1,"99":2}}],["renormalize",{"2":{"69":2}}],["request",{"2":{"84":1,"95":1}}],["requisites",{"2":{"77":1}}],["requirements",{"2":{"62":1,"236":1}}],["require",{"2":{"19":1,"37":1,"39":1,"59":1,"116":1,"208":1}}],["requires",{"2":{"8":1,"10":1,"42":4,"54":1,"80":1,"119":1,"208":1,"225":1}}],["rev=true",{"2":{"244":4}}],["revising",{"2":{"83":1}}],["reverses",{"2":{"68":1}}],["reversesequence",{"2":{"68":3}}],["reversed",{"2":{"68":1}}],["reversediffadjoint",{"2":{"214":1}}],["reversediff",{"2":{"8":2,"16":1,"18":1,"20":1,"49":1,"52":1,"55":1,"86":1,"87":1,"122":1,"217":1}}],["reverse",{"2":{"27":2,"33":2,"49":1,"59":1,"68":2,"86":6,"106":1,"110":1,"175":2,"176":2}}],["retcode",{"2":{"258":1}}],["retrieve",{"2":{"67":1}}],["retained",{"2":{"64":1}}],["retuened",{"2":{"3":2}}],["returned",{"2":{"17":1,"19":1,"22":1,"24":1,"34":1,"37":1,"39":1,"46":1,"52":6,"54":1,"64":4,"66":3,"67":1,"68":1}}],["returning",{"2":{"8":1,"140":1}}],["returns",{"2":{"3":3,"7":2,"8":3,"13":2,"17":2,"19":4,"20":1,"24":6,"27":2,"28":1,"34":1,"50":1,"52":2,"53":6,"54":2,"57":1,"62":7,"63":2,"64":3,"65":9,"66":8,"67":5,"68":6,"69":5,"70":2,"100":1,"131":3,"146":1,"147":1,"148":1,"208":1,"220":6,"256":1}}],["return",{"2":{"2":3,"3":5,"7":1,"9":1,"10":1,"11":2,"24":6,"25":24,"31":1,"32":2,"52":4,"53":9,"54":2,"55":2,"59":19,"62":3,"66":4,"68":1,"78":3,"82":1,"95":1,"106":2,"107":1,"110":2,"121":1,"123":2,"126":1,"127":3,"128":4,"134":1,"140":5,"141":2,"142":3,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"169":1,"179":1,"183":1,"184":2,"185":2,"186":1,"187":1,"192":1,"194":1,"195":1,"206":1,"208":3,"213":1,"214":5,"215":1,"216":1,"218":2,"220":3,"224":2,"225":3,"226":1,"227":1,"228":1,"232":1,"235":1,"236":1,"241":2,"242":3,"244":1,"245":1,"248":1,"249":2,"250":2,"254":11,"255":1,"256":1,"257":2}}],["remove",{"2":{"174":1}}],["removed",{"0":{"102":1,"115":1},"2":{"63":1,"92":1,"95":2,"99":1,"102":10,"103":4,"106":1,"115":1,"130":1,"145":2,"146":1}}],["remake",{"2":{"208":1}}],["remark",{"2":{"146":1}}],["remains",{"2":{"17":1}}],["remember",{"2":{"107":1,"108":1,"111":1,"140":1,"144":1,"153":1,"184":1,"217":1}}],["regions",{"2":{"250":1}}],["registry",{"2":{"71":1,"79":1}}],["registered",{"2":{"59":1,"71":1}}],["regression",{"0":{"179":1},"2":{"155":1,"179":1}}],["regressions",{"2":{"8":1,"130":1,"196":1}}],["reg",{"2":{"145":2,"146":2}}],["regarding",{"2":{"134":1,"175":1}}],["regularization",{"2":{"145":1,"152":1,"249":1}}],["regular",{"2":{"39":2,"59":2,"153":1}}],["reuse",{"2":{"16":1,"18":1}}],["reusing",{"2":{"16":1,"18":1}}],["reltol",{"2":{"220":13}}],["reltol=1",{"2":{"215":1}}],["reliance",{"2":{"145":2}}],["reliable",{"2":{"87":2}}],["reliability",{"2":{"83":1}}],["relies",{"2":{"134":2,"161":1,"174":1}}],["relevant",{"2":{"106":1,"247":1,"254":1}}],["release",{"2":{"90":2,"97":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["released",{"2":{"71":1}}],["relativisticorbitmodel",{"2":{"255":3}}],["relatively",{"2":{"144":1,"153":1}}],["relation",{"2":{"66":1}}],["related",{"2":{"84":1}}],["relu",{"2":{"16":1,"37":1,"39":4,"59":3,"62":4,"66":1,"68":3,"69":8,"78":1,"109":5,"110":5,"133":2,"153":1,"193":12,"226":2,"233":2,"236":2}}],["rely",{"2":{"15":1,"59":1,"95":1,"174":1}}],["readme",{"2":{"106":1}}],["read",{"2":{"84":1,"173":1}}],["reactant",{"0":{"106":1},"1":{"107":1},"2":{"73":7,"83":1,"106":9,"107":1}}],["reason",{"2":{"87":1,"155":1}}],["reasonable",{"2":{"62":1}}],["reasons",{"2":{"8":1,"214":1}}],["really",{"2":{"39":1,"59":1,"89":1,"140":1,"171":1}}],["reallocations",{"2":{"16":1,"18":1}}],["real",{"2":{"24":1,"53":4,"64":1,"123":1,"245":2}}],["real=eps",{"2":{"19":1}}],["rewrite",{"2":{"13":1,"37":1,"240":1}}],["receives",{"2":{"68":1}}],["reconstruction",{"2":{"140":1}}],["recognition",{"2":{"53":1}}],["record",{"2":{"50":1,"245":1,"250":1}}],["recorded",{"2":{"49":1,"50":4}}],["recomputing",{"2":{"17":1}}],["recommendations",{"0":{"87":1},"2":{"154":1,"155":1}}],["recommendation",{"2":{"83":1}}],["recommend",{"2":{"7":1,"83":1,"116":1,"138":1,"140":1,"173":1,"177":1,"185":1,"188":1,"211":1,"214":1,"220":1,"225":1}}],["recommended",{"0":{"131":1},"2":{"3":1,"7":2,"8":1,"15":1,"31":1,"37":1,"47":1,"55":1,"59":2,"83":1,"134":1,"138":1,"140":2,"144":1,"145":2,"150":1,"152":1,"153":1,"156":1,"164":1,"186":1,"256":1}}],["recvbuf",{"2":{"44":6}}],["rectifiers",{"2":{"24":2}}],["recur",{"2":{"66":3}}],["recurrence",{"2":{"66":4,"184":1}}],["recurrent",{"0":{"66":1},"2":{"24":1,"66":10,"104":1,"181":2}}],["recurse",{"2":{"55":3}}],["recurses",{"2":{"33":1,"56":3}}],["recursion",{"2":{"55":1}}],["recursive",{"0":{"55":1},"2":{"55":9,"153":2,"155":1}}],["recursively",{"2":{"10":1,"55":5}}],["repository",{"2":{"144":1}}],["report",{"2":{"30":1,"84":1,"89":1}}],["reproducer",{"2":{"117":1}}],["represents",{"2":{"249":1}}],["representing",{"2":{"70":1}}],["representation",{"2":{"59":1}}],["represent",{"2":{"24":1}}],["reparameterized",{"2":{"69":2}}],["reparameterization",{"2":{"69":1}}],["repack",{"2":{"8":1}}],["repeat",{"2":{"183":2}}],["repeatedly",{"2":{"62":1}}],["repeatedlayer",{"2":{"62":1}}],["repeats",{"2":{"62":6,"66":5}}],["replacing",{"2":{"133":1,"171":1}}],["replacement",{"2":{"95":1}}],["replaces",{"2":{"33":1,"65":3}}],["replaced",{"2":{"13":1}}],["replace",{"2":{"13":2,"124":1,"158":5}}],["repl",{"2":{"48":1,"71":1,"76":1,"156":2,"161":1}}],["replicated",{"2":{"174":1}}],["replicate",{"2":{"7":1,"8":1,"62":1,"174":3}}],["resolve",{"2":{"148":1}}],["resolution",{"2":{"70":1}}],["resources",{"0":{"84":1}}],["rescale",{"2":{"69":4}}],["resnet",{"2":{"62":1}}],["res",{"2":{"59":2,"62":4,"208":5,"258":1,"259":1}}],["respective",{"2":{"81":1,"113":2,"134":1,"173":1}}],["respect",{"2":{"28":2,"49":1,"146":2}}],["responsibility",{"2":{"8":1,"142":1}}],["reshapes",{"2":{"68":1}}],["reshapelayer",{"2":{"68":3}}],["reshape",{"2":{"53":4,"78":1,"150":2,"151":2,"173":2,"183":2,"192":1,"213":1,"214":4,"224":2,"232":1,"243":2,"245":2,"249":1}}],["reshaped",{"2":{"16":1,"68":1}}],["reshaping",{"2":{"24":1}}],["restricted",{"2":{"55":1}}],["rest",{"2":{"24":1,"184":2,"185":2}}],["restarted",{"2":{"1":1,"60":1}}],["reserved",{"2":{"59":1}}],["research",{"2":{"17":1}}],["resets",{"2":{"3":1}}],["reset",{"2":{"3":1}}],["results",{"0":{"209":1,"245":1,"259":1},"2":{"106":1,"152":2,"236":1,"250":2,"255":1,"256":1,"258":1,"259":1}}],["result",{"2":{"15":1,"24":1,"28":2,"44":4,"50":4,"69":1,"82":1,"142":4}}],["mtensor",{"2":{"254":4}}],["mtlarray",{"2":{"22":3}}],["mcse",{"2":{"249":1}}],["mcmcchains",{"2":{"249":1}}],["mcmc",{"2":{"249":1,"250":1}}],["mcclelland",{"2":{"24":1}}],["mvnormal",{"2":{"249":1}}],["md",{"2":{"244":8}}],["mnist\\ttraining",{"2":{"228":1}}],["mnist\\ttime",{"2":{"228":50}}],["mnist",{"0":{"190":1,"192":1,"211":1,"213":1,"222":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"191":1,"192":2,"212":1,"213":2,"224":1,"228":2}}],["my",{"2":{"196":1}}],["mybias",{"2":{"142":3}}],["myinputtype",{"2":{"119":3}}],["myfancychain",{"2":{"62":2}}],["myweight",{"2":{"31":1,"142":4}}],["mse",{"2":{"242":3}}],["mseloss",{"2":{"53":4,"77":2,"78":1,"106":1,"107":1,"179":1,"208":1,"235":1,"242":1,"257":3}}],["msleloss",{"2":{"53":2}}],["mpi",{"0":{"163":1},"2":{"41":2,"42":8,"116":2,"117":1,"163":5}}],["mpibackend",{"2":{"41":2,"42":2}}],["m2",{"2":{"37":3}}],["mₘ",{"2":{"28":2}}],["m₂",{"2":{"28":2,"254":7}}],["m₁",{"2":{"28":2,"254":6}}],["m",{"2":{"19":2,"37":3,"56":6,"62":2,"72":2,"214":2,"218":2,"248":15,"254":4,"255":5,"256":3}}],["mkl",{"2":{"18":1}}],["mldatasets",{"2":{"191":1,"212":1,"223":1}}],["mldatadevices",{"0":{"0":1,"98":1},"1":{"1":1,"2":1,"3":1,"4":1,"5":1,"99":1,"100":1},"2":{"0":1,"1":1,"2":3,"3":12,"4":1,"5":2,"98":2,"132":2,"164":2,"180":3,"189":3,"197":3,"205":1,"207":3,"210":3,"221":3,"229":3,"237":3,"239":1,"246":3,"251":3,"260":3}}],["ml",{"2":{"171":1,"208":2}}],["mlir",{"2":{"106":1}}],["mlp",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"59":1,"78":1,"106":1,"226":1,"230":1,"241":4}}],["mlutils",{"0":{"46":1},"2":{"5":3,"46":4,"100":1,"113":1,"160":2,"182":1,"183":3,"191":1,"205":1,"207":1,"212":1,"223":1,"239":1}}],["motion",{"2":{"254":1,"255":1,"256":1}}],["motivating",{"2":{"8":1,"58":1}}],["moment",{"2":{"147":1,"254":1}}],["momentum=0",{"2":{"69":2}}],["momentum",{"2":{"19":6,"69":2}}],["mooncake",{"2":{"86":1}}],["monolithic",{"2":{"83":1}}],["month",{"2":{"74":1}}],["mouthful",{"2":{"34":1}}],["moved",{"0":{"103":1},"2":{"95":1,"103":2,"106":1}}],["move",{"2":{"30":1,"106":1,"107":2,"172":1,"173":1,"207":3}}],["movement",{"2":{"3":2}}],["modivations",{"2":{"102":1}}],["modify",{"2":{"69":1}}],["modified",{"2":{"55":2,"121":1}}],["modulating",{"2":{"53":1}}],["modules=",{"2":{"48":2}}],["modules",{"2":{"48":8}}],["module",{"0":{"54":1},"2":{"30":1,"40":1,"48":1,"54":1,"102":1,"103":1}}],["modes",{"2":{"20":1,"60":1,"70":1}}],["mode",{"0":{"136":1},"2":{"17":2,"19":2,"20":5,"27":2,"33":7,"49":1,"60":2,"64":6,"66":2,"69":3,"70":5,"86":1,"87":1,"103":1,"108":3,"109":2,"110":2,"136":4,"145":2,"155":1,"156":1,"161":1,"175":1,"176":4,"177":2,"254":1}}],["model`",{"2":{"214":1}}],["models",{"0":{"37":2,"39":1,"106":1,"108":1,"124":1,"202":1,"204":1},"1":{"107":1,"109":1,"110":1,"111":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"8":1,"21":1,"37":2,"39":2,"47":1,"52":2,"59":2,"77":1,"83":6,"106":1,"108":2,"111":1,"120":1,"153":1,"168":1,"171":1,"177":1,"181":1,"185":1,"202":2,"204":1}}],["model",{"0":{"109":1,"125":1,"187":1,"188":1,"193":1,"196":1,"208":1,"252":1,"255":1,"256":1},"1":{"126":1,"127":1,"128":1,"129":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"8":8,"32":1,"33":2,"34":2,"37":6,"39":11,"52":5,"53":26,"58":3,"59":14,"62":16,"66":3,"68":12,"69":4,"77":6,"78":10,"81":1,"106":18,"107":8,"108":3,"109":17,"110":25,"111":1,"113":2,"120":1,"121":1,"122":2,"123":2,"124":2,"125":1,"126":6,"129":7,"133":9,"134":11,"136":1,"137":1,"140":1,"141":5,"145":10,"146":7,"147":7,"148":7,"150":2,"151":2,"152":9,"153":4,"155":6,"179":11,"184":2,"185":1,"186":2,"187":10,"188":6,"190":1,"193":4,"194":2,"195":7,"196":4,"202":1,"208":9,"209":2,"214":11,"215":8,"216":2,"217":10,"218":8,"220":17,"226":4,"227":2,"228":9,"235":2,"236":3,"242":4,"243":1,"244":7,"249":7,"250":1,"254":7,"255":4,"256":11,"257":1,"258":1,"259":2}}],["mostly",{"2":{"14":1,"83":1,"90":1,"102":1,"117":1,"214":1}}],["most",{"2":{"7":1,"8":1,"20":1,"24":1,"33":1,"52":2,"55":1,"71":1,"83":1,"85":1,"87":2,"98":1,"107":1,"133":1,"138":1,"148":1,"150":1,"153":1,"203":1,"256":1}}],["more",{"2":{"5":1,"7":1,"8":1,"11":1,"14":1,"24":2,"37":1,"39":2,"49":1,"54":1,"55":1,"58":1,"59":3,"62":1,"70":1,"83":1,"84":1,"92":1,"95":1,"97":1,"100":1,"102":1,"104":2,"116":1,"145":1,"146":1,"155":1,"157":1,"162":1,"165":1,"166":1,"171":3,"175":1,"184":1,"185":1,"186":1,"204":1,"250":1}}],["mechanics",{"2":{"254":1}}],["mechanism",{"2":{"83":1}}],["message",{"2":{"144":1}}],["mersennetwister",{"2":{"122":1,"168":2,"232":1}}],["merged",{"2":{"66":1}}],["merge",{"2":{"31":1,"32":1,"66":2,"121":1,"184":1}}],["mentioned",{"2":{"79":1,"140":1}}],["measure",{"2":{"53":1}}],["meanpool",{"2":{"65":1}}],["meant",{"2":{"33":1,"52":1,"58":1,"169":1}}],["means",{"2":{"24":1,"30":1,"59":1,"124":1,"140":1,"142":1,"155":1,"171":1,"173":1,"208":2}}],["meaningless",{"2":{"17":1}}],["mean",{"2":{"17":2,"19":13,"24":2,"53":17,"65":7,"69":13,"109":1,"110":2,"126":1,"129":1,"142":1,"179":1,"242":3,"244":12,"249":1,"250":1}}],["medical",{"2":{"53":1}}],["medium",{"2":{"18":1}}],["met",{"2":{"24":1,"33":1}}],["methodinstance",{"2":{"220":3}}],["methoderror",{"2":{"122":1}}],["methods",{"2":{"70":1,"78":1,"131":1,"149":1,"153":1,"169":2,"204":2,"205":1,"208":1,"214":1,"215":1,"224":1,"239":1,"249":1,"254":2}}],["method",{"2":{"24":3,"106":1,"110":1,"119":1,"122":1,"150":1,"151":1,"152":1,"183":1,"185":1,"186":1,"192":1,"194":1,"195":1,"213":1,"214":1,"216":1,"225":1,"226":1,"227":1,"232":1,"242":2,"249":1,"250":2,"254":3,"256":1,"257":2}}],["metaldevice",{"2":{"4":2}}],["metal",{"2":{"2":1,"3":2,"22":1,"72":2,"77":1,"78":1,"80":2,"130":4,"164":1}}],["memory=false",{"2":{"66":1}}],["memory=zeros32",{"2":{"66":1}}],["memory",{"2":{"5":1,"16":1,"18":1,"66":20,"180":2,"184":1,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["mutability",{"0":{"173":1}}],["mutables",{"2":{"134":1}}],["mutable",{"2":{"134":1,"140":1}}],["mutations",{"2":{"81":1,"83":2,"87":1}}],["mutation",{"2":{"16":1,"18":1,"173":2}}],["mutating",{"2":{"16":1,"18":1}}],["mutate",{"2":{"173":1}}],["mutated",{"2":{"15":1,"55":1,"173":2,"174":1}}],["mutates",{"2":{"8":1}}],["much",{"2":{"53":2,"83":1,"114":1,"250":1}}],["mul",{"2":{"14":2,"158":1}}],["multilayer",{"2":{"230":1}}],["multirnncell",{"2":{"66":1}}],["multigate",{"2":{"54":1}}],["multiply",{"2":{"171":1}}],["multiplied",{"2":{"17":1}}],["multiplication",{"2":{"14":3}}],["multiple",{"0":{"146":1},"2":{"16":1,"18":1,"62":1,"66":1,"69":1,"130":1,"153":2,"250":1}}],["multi",{"0":{"4":1},"2":{"53":1,"67":1}}],["must",{"2":{"1":1,"3":2,"4":1,"7":1,"19":1,"24":4,"28":2,"32":2,"34":2,"37":1,"39":2,"42":1,"46":2,"52":3,"60":1,"62":3,"63":2,"64":3,"66":7,"67":4,"69":1,"70":1,"82":4,"83":1,"113":1,"140":1,"143":1,"235":1,"254":4}}],["major",{"0":{"93":1,"96":1,"100":1},"2":{"90":1,"97":1}}],["masses",{"2":{"254":1}}],["mass2",{"2":{"254":5}}],["mass1",{"2":{"254":6}}],["mass",{"2":{"254":14,"255":2,"256":1,"257":1,"259":1}}],["mass=1",{"2":{"254":3}}],["massachusetts",{"2":{"74":1}}],["master",{"2":{"113":1}}],["mask",{"2":{"17":9,"64":10}}],["magnitude",{"2":{"69":2}}],["maeloss",{"2":{"53":2}}],["made",{"2":{"52":1,"90":1,"95":1,"98":1,"140":1,"144":1}}],["making",{"2":{"52":1,"81":1,"83":1,"98":1,"106":1,"153":1}}],["makes",{"2":{"32":1,"53":1,"66":1,"83":1,"105":1,"138":1}}],["make",{"2":{"10":2,"34":1,"52":1,"55":6,"66":1,"81":2,"83":1,"109":1,"114":1,"133":1,"140":1,"142":1,"145":1,"146":1,"153":1,"161":1,"171":1,"175":1,"183":1,"249":1,"256":2}}],["macro",{"2":{"48":1,"49":1,"59":4,"78":2,"106":1,"138":2,"185":1,"214":2}}],["machines",{"2":{"53":2}}],["machine",{"2":{"17":1,"19":1,"24":1,"149":1,"171":1,"196":1,"254":1}}],["maxiters",{"2":{"244":4}}],["maxiters=1000",{"2":{"258":1}}],["maxiters=500",{"2":{"208":1}}],["maxiters=epochs",{"2":{"208":1}}],["maximum",{"2":{"16":1,"18":1,"68":1,"243":2,"245":1}}],["maxout",{"2":{"68":5}}],["max",{"2":{"53":2,"65":4,"68":1,"243":8,"245":1,"250":1}}],["maxpool",{"2":{"39":2,"65":1,"193":6}}],["margin=2",{"2":{"53":1}}],["margin−y^",{"2":{"53":1}}],["margin",{"2":{"53":2}}],["marker=",{"2":{"255":1,"256":2,"259":4}}],["markersize=16",{"2":{"248":2}}],["markersize=12",{"2":{"232":1,"236":2,"255":1,"256":2,"259":4}}],["marked",{"2":{"5":1,"56":3,"59":1}}],["mark",{"2":{"59":2,"256":1}}],["marks",{"2":{"48":2,"98":1}}],["martens",{"2":{"24":1}}],["main",{"0":{"24":1},"2":{"47":1,"123":1,"124":1,"140":1,"160":1,"184":1,"187":3,"207":2,"214":1,"220":18,"236":2,"241":1}}],["maintaining",{"2":{"19":1}}],["matches",{"2":{"186":2}}],["matched",{"2":{"122":5}}],["matching",{"0":{"57":1},"2":{"122":1}}],["match",{"2":{"32":1,"57":4,"66":5,"143":2,"155":1,"165":1}}],["matrices",{"2":{"18":1,"67":1}}],["matrix",{"2":{"5":3,"14":3,"15":1,"18":2,"24":13,"53":6,"59":4,"67":6,"68":1,"78":1,"106":2,"109":5,"110":13,"122":3,"123":1,"124":1,"131":1,"146":1,"147":1,"148":1,"149":5,"153":1,"168":4,"171":13,"175":3,"207":2}}],["matmul",{"2":{"14":2,"150":1,"151":1,"158":1}}],["maps",{"2":{"24":2,"53":1,"65":3,"66":2}}],["mapping",{"2":{"24":7,"53":1}}],["map",{"0":{"32":1},"2":{"7":1,"32":4,"55":2,"66":4,"102":2,"103":1,"126":2,"127":1,"153":2,"250":1}}],["managing",{"0":{"174":1},"2":{"174":1}}],["management",{"0":{"130":1,"131":1,"132":1},"1":{"131":1,"132":1},"2":{"98":1,"130":1,"131":1}}],["manager",{"2":{"76":1}}],["manage",{"2":{"59":1}}],["many",{"2":{"146":1,"171":1,"175":1}}],["mandatorily",{"2":{"140":1}}],["manual",{"0":{"132":1},"2":{"8":1,"29":1,"31":1,"58":1,"111":1,"125":1,"144":1,"162":1,"172":1}}],["manually",{"2":{"2":1,"37":1,"39":1,"113":1,"138":1,"172":1}}],["manipulation",{"2":{"7":2,"83":1}}],["maybe",{"2":{"57":1,"175":1}}],["may",{"2":{"3":1,"59":3,"70":2,"196":1,"203":1}}],["min",{"2":{"243":10,"245":2}}],["minibatching",{"2":{"204":1,"208":1}}],["minibatch",{"2":{"183":1,"192":1,"213":1}}],["mini",{"2":{"177":1}}],["minimum",{"2":{"243":4,"245":2}}],["minimally",{"2":{"252":1}}],["minimal",{"2":{"117":1}}],["minimized",{"2":{"257":1}}],["minimize",{"2":{"18":1}}],["minimizes",{"2":{"16":1,"179":1}}],["minor",{"2":{"144":1,"153":1}}],["mig",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["migrating",{"0":{"133":1},"1":{"134":1,"135":1,"136":1,"137":1}}],["migration",{"0":{"114":1},"1":{"115":1,"116":1},"2":{"140":1}}],["might",{"2":{"2":1,"8":1,"13":1,"15":1,"37":1,"39":1,"52":3,"58":2,"59":1,"62":1,"68":1,"89":1,"106":1,"107":1,"111":1,"130":1,"140":2,"144":1,"145":2,"155":1,"159":1,"167":2,"171":1,"173":1,"196":1,"208":1,"219":1}}],["miopen",{"2":{"105":1}}],["mirza",{"2":{"68":1}}],["milletari",{"2":{"53":1}}],["mixing",{"2":{"28":1}}],["mixed",{"2":{"16":1,"27":2}}],["mistakes",{"2":{"83":1}}],["misc",{"0":{"68":1}}],["miscellaneous",{"0":{"3":1,"60":1},"2":{"62":1}}],["mismatch",{"0":{"109":1},"2":{"57":3,"116":1,"155":1,"165":1}}],["missing",{"2":{"19":1,"42":1,"69":1}}],["s3",{"2":{"259":2}}],["s0025",{"2":{"254":2}}],["s2",{"2":{"236":2,"256":2,"259":2}}],["s1",{"2":{"236":2,"256":2,"259":2}}],["sgd",{"2":{"179":1}}],["sz",{"2":{"121":2}}],["szegedy",{"2":{"19":1}}],["src",{"2":{"109":1,"122":1,"145":2,"220":3,"244":4}}],["srivastava",{"2":{"17":1}}],["switch",{"2":{"64":3,"98":1,"144":1,"175":1}}],["switching",{"0":{"36":1},"1":{"37":1},"2":{"144":4,"162":2}}],["square",{"2":{"171":2,"243":1}}],["squaredhingeloss",{"2":{"53":2}}],["squared",{"2":{"53":3,"179":1}}],["sqrt",{"2":{"24":4,"63":2,"66":6,"67":3,"249":1,"255":2}}],["sm",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["smarter",{"2":{"243":1}}],["smatrix",{"2":{"153":4}}],["smaller",{"2":{"83":1}}],["small",{"2":{"18":1,"39":1,"83":1,"106":1,"155":1,"171":1,"190":1,"208":1}}],["smodel",{"2":{"145":3,"146":4,"147":2,"148":2,"150":2,"151":2,"152":2,"208":3}}],["smooth",{"2":{"53":2}}],["smoothing=0",{"2":{"53":3}}],["smoothing",{"2":{"53":16}}],["skipped",{"2":{"62":1,"254":1}}],["skipconnection",{"2":{"62":4}}],["skip",{"2":{"49":3,"62":1,"106":1,"243":1}}],["speedup",{"2":{"196":2}}],["spectralnorm",{"2":{"83":1}}],["specify",{"2":{"53":1,"59":1,"64":2,"70":1,"128":1,"249":1}}],["specifying",{"2":{"24":1,"48":1,"59":1,"63":2,"65":3,"69":2}}],["specifies",{"2":{"63":4,"65":3,"68":1,"177":1}}],["specified",{"2":{"2":1,"24":2,"49":1,"62":8,"67":1,"68":3,"70":1,"131":1,"169":1}}],["specifically",{"2":{"136":1,"171":1}}],["specifications",{"2":{"111":1}}],["specification",{"0":{"109":1},"2":{"249":1}}],["specific",{"2":{"4":2,"8":1,"55":1,"67":1,"88":1,"164":1}}],["specialized",{"2":{"13":1,"20":1,"59":1,"153":1,"173":1}}],["special",{"2":{"3":2,"18":2,"44":2,"53":1,"59":1,"66":1,"88":1,"220":3}}],["spiralclassifiercompact",{"2":{"185":2,"187":1}}],["spiralclassifier",{"2":{"184":5,"187":1}}],["spiral",{"2":{"183":1}}],["spirals",{"2":{"181":1,"183":6}}],["spiritual",{"2":{"112":1}}],["spurious",{"0":{"155":1}}],["space",{"2":{"66":6}}],["spatial",{"2":{"63":5,"65":6,"243":1}}],["sparsity=0",{"2":{"24":1}}],["sparsity",{"2":{"24":4}}],["sparsely",{"2":{"24":2,"67":1}}],["sparse",{"2":{"24":2}}],["spliced",{"2":{"66":2}}],["split=",{"2":{"192":1,"213":1}}],["splitobs",{"2":{"183":1,"192":1,"212":1,"213":1}}],["split",{"2":{"54":1,"113":2,"183":1,"184":1,"192":2,"213":2}}],["splatted",{"2":{"59":1}}],["splatting",{"2":{"59":1}}],["scientific",{"2":{"254":1}}],["scientificmachinelearning",{"2":{"252":1}}],["scimlsensitivity",{"2":{"205":1,"212":1,"220":26,"253":1}}],["sciml",{"2":{"58":1,"59":1,"78":1,"83":2,"177":1}}],["scatter",{"2":{"232":1,"236":2,"248":2,"255":1,"256":2,"259":4}}],["scalable",{"2":{"150":1}}],["scalar",{"0":{"156":1},"2":{"3":2,"20":1,"107":1,"156":1,"176":1,"241":1}}],["scale=ones32",{"2":{"69":4}}],["scaled",{"2":{"24":3}}],["scale",{"2":{"19":15,"67":2,"69":20,"70":6,"110":1,"129":1,"145":1}}],["scaling",{"2":{"17":2,"24":2,"254":6}}],["scratch",{"2":{"179":1,"184":1,"211":1}}],["schwarzschild",{"2":{"255":1}}],["school",{"2":{"74":1}}],["schemes",{"2":{"21":1,"168":1}}],["score",{"2":{"53":1}}],["slow",{"2":{"145":2}}],["slower",{"2":{"108":1}}],["slices",{"2":{"54":1}}],["slice",{"2":{"19":2,"24":1,"66":2,"69":2}}],["slightly",{"2":{"62":1}}],["slight",{"2":{"13":1,"175":1}}],["sleefpirates",{"2":{"13":2}}],["s",{"2":{"10":1,"20":1,"57":3,"58":1,"59":1,"62":1,"64":2,"67":1,"69":3,"70":1,"72":1,"77":1,"83":1,"106":2,"107":1,"109":1,"110":1,"114":2,"126":1,"134":1,"140":1,"141":1,"144":2,"145":3,"147":1,"148":1,"149":1,"152":3,"153":1,"155":1,"170":4,"171":5,"176":1,"179":2,"184":5,"186":1,"188":1,"204":1,"208":2,"217":1,"232":3,"233":1,"236":1,"249":2,"255":3}}],["syntax",{"2":{"59":1,"62":1,"171":2,"172":1}}],["synchronized",{"2":{"113":2}}],["synchronize",{"2":{"44":2,"113":6,"114":2,"116":2,"153":2}}],["symbol",{"2":{"10":1,"33":1,"59":1,"69":1,"106":1,"142":1,"188":1,"220":29,"249":2}}],["systems",{"2":{"17":1,"77":1,"83":2,"167":1}}],["system",{"2":{"2":1,"3":1,"106":1,"130":2,"142":1,"175":1,"238":1,"255":1,"256":1}}],["safely",{"2":{"256":1}}],["saveat=tsteps",{"2":{"255":1,"256":1,"257":1,"259":1}}],["saveat=t",{"2":{"206":1,"208":1,"209":1}}],["save",{"2":{"188":4,"215":2,"220":26}}],["saving",{"0":{"188":1},"2":{"188":1}}],["sake",{"2":{"184":1,"186":1,"243":1}}],["sarray",{"2":{"153":7}}],["sarrays",{"2":{"153":1}}],["sanity",{"2":{"152":1}}],["sampling",{"2":{"243":1,"247":1}}],["sampler",{"2":{"249":1}}],["sampled",{"2":{"249":2}}],["samples",{"2":{"179":15,"249":2,"250":2}}],["sample",{"2":{"149":3,"152":1,"249":3,"250":1}}],["samepad",{"2":{"63":2,"65":3}}],["same",{"0":{"124":1},"2":{"3":1,"5":2,"7":3,"13":2,"15":1,"17":1,"19":3,"24":1,"27":2,"32":1,"33":1,"34":1,"52":2,"54":2,"57":2,"62":1,"63":2,"65":3,"66":3,"68":2,"69":1,"71":1,"78":1,"82":2,"106":1,"113":1,"141":1,"146":1,"152":2,"153":1,"172":1,"174":1,"178":1,"187":1,"190":1,"214":1}}],["saw",{"2":{"107":1}}],["say",{"2":{"83":1,"126":1}}],["satisfactory",{"2":{"250":1}}],["satisfying",{"2":{"63":2,"65":6,"140":1}}],["satisfies",{"2":{"39":2,"68":1}}],["satisfied",{"2":{"8":1,"235":1}}],["said",{"2":{"30":1}}],["saxe",{"2":{"24":1}}],["sorted",{"2":{"84":1}}],["soln",{"2":{"254":13,"255":2,"256":2,"259":2}}],["soln2orbit",{"2":{"254":3}}],["solution",{"2":{"243":5,"245":1}}],["solutions",{"2":{"24":1}}],["sol",{"2":{"209":2}}],["solver",{"2":{"214":6,"218":3,"220":7}}],["solver=tsit5",{"2":{"214":2,"218":1}}],["solvers",{"2":{"83":1}}],["solves",{"2":{"208":1}}],["solve",{"2":{"206":1,"208":4,"209":1,"214":2,"218":1,"238":1,"255":1,"256":1,"257":1,"258":1,"259":1}}],["society",{"2":{"53":1}}],["software",{"2":{"74":2}}],["softmax",{"2":{"53":5}}],["softfail",{"2":{"50":2}}],["soft",{"2":{"49":4}}],["sooner",{"2":{"30":1}}],["so",{"2":{"13":1,"77":1,"134":1,"144":3,"145":1,"146":1,"207":1,"214":2,"218":2,"235":1,"240":1,"241":1}}],["sometimes",{"2":{"171":2}}],["somewhere",{"2":{"37":1,"39":1}}],["some",{"0":{"206":1,"254":1},"2":{"8":1,"33":1,"37":2,"59":1,"66":1,"68":1,"83":1,"85":1,"87":1,"88":1,"89":1,"90":1,"102":1,"108":1,"110":1,"112":1,"116":1,"128":1,"134":1,"141":1,"144":7,"145":1,"155":1,"158":1,"160":2,"171":1,"217":1,"225":1,"243":1}}],["source",{"2":{"1":1,"2":3,"3":8,"4":2,"5":1,"7":3,"8":7,"9":2,"10":5,"11":1,"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"20":1,"24":8,"25":24,"27":2,"28":1,"31":5,"32":1,"33":2,"34":1,"37":3,"39":3,"41":2,"42":3,"43":2,"44":4,"45":1,"46":1,"48":2,"49":2,"50":1,"52":6,"53":15,"54":8,"55":6,"56":3,"57":1,"58":1,"59":3,"60":1,"62":6,"63":2,"64":3,"65":9,"66":6,"67":4,"68":7,"69":5,"70":2,"95":1,"170":2}}],["situations",{"2":{"117":1}}],["si+1",{"2":{"63":1,"65":3}}],["siamese",{"2":{"53":1}}],["siamesecontrastiveloss",{"2":{"53":3}}],["sig",{"2":{"249":2}}],["sigmoid",{"2":{"53":3,"184":1,"185":1,"249":2}}],["signify",{"2":{"90":1,"97":1}}],["significantly",{"2":{"138":1,"217":1}}],["significant",{"2":{"8":1,"196":1,"219":1}}],["signature",{"2":{"17":1,"32":1}}],["silently",{"2":{"34":1}}],["sided",{"2":{"254":2}}],["side",{"2":{"16":1,"59":1}}],["size=1000",{"2":{"183":1}}],["sizes",{"2":{"18":1,"24":2,"95":1}}],["sized",{"2":{"18":1,"19":1,"54":1}}],["size",{"0":{"11":1},"2":{"11":2,"13":2,"19":3,"24":24,"25":72,"28":2,"37":1,"39":1,"59":6,"62":1,"63":16,"65":48,"66":11,"67":16,"68":8,"69":7,"70":8,"105":1,"121":1,"150":2,"151":2,"153":2,"171":1,"179":2,"180":1,"183":7,"189":1,"192":3,"197":1,"210":1,"213":3,"214":2,"217":1,"221":1,"229":1,"237":1,"246":1,"249":4,"250":1,"251":1,"254":4,"260":1}}],["sin",{"2":{"254":1}}],["singular",{"0":{"140":1},"2":{"140":1,"141":2}}],["singleton",{"2":{"7":3,"63":2,"65":3,"95":2,"96":1}}],["single",{"2":{"7":2,"8":1,"15":1,"16":1,"52":5,"53":1,"59":1,"63":6,"65":9,"66":13,"77":2,"78":1,"95":1,"107":3,"134":1,"141":1,"146":1,"149":1,"153":4,"171":1,"179":1,"187":1,"195":1,"217":1,"228":1,"236":1,"244":1,"249":1}}],["since",{"2":{"8":1,"24":1,"32":1,"59":1,"71":1,"83":1,"95":1,"107":1,"113":1,"119":1,"142":1,"177":1,"184":1,"186":1,"208":2,"240":1}}],["simultaneously",{"2":{"142":1}}],["simulate",{"2":{"110":1,"254":1,"255":1,"256":2}}],["simulating",{"0":{"255":1},"2":{"24":1}}],["simd",{"2":{"20":1}}],["simplicity",{"2":{"149":1,"186":1,"243":1}}],["simplified",{"2":{"140":1}}],["simplest",{"2":{"62":1}}],["simplechainslayer",{"2":{"39":2,"193":1}}],["simplechains",{"0":{"190":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1},"2":{"39":14,"83":1,"190":3,"191":2,"193":1,"196":2}}],["simple",{"0":{"39":1,"181":1},"1":{"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1},"2":{"17":2,"20":1,"39":4,"52":1,"59":1,"66":1,"106":1,"131":2,"134":1,"136":1,"170":1,"176":1,"193":1,"196":1,"204":1}}],["simply",{"2":{"8":1,"13":1,"52":1,"56":1,"57":1,"59":1,"71":1,"107":1,"108":1,"124":1,"128":1,"142":1,"225":1,"249":1}}],["similarly",{"2":{"141":1}}],["similarity",{"2":{"5":1}}],["similar",{"2":{"2":1,"3":1,"7":1,"19":1,"53":1,"54":1,"55":1,"66":1,"67":1,"106":1,"107":1,"133":1,"142":1,"160":1,"175":1,"256":1}}],["ship",{"2":{"108":1}}],["shifted",{"2":{"24":1}}],["shift",{"2":{"19":1,"24":4,"69":4}}],["shuffle=true",{"2":{"183":2,"192":1,"213":1,"224":1,"244":2}}],["shuffle=false",{"2":{"5":1,"183":1,"192":1,"213":1,"224":1}}],["shuffle",{"2":{"70":1,"183":2,"192":2,"213":2}}],["shuffling",{"2":{"70":1}}],["shapedaxis",{"2":{"220":30}}],["shaped",{"2":{"65":9}}],["shape",{"2":{"63":1,"66":16,"68":1,"69":17,"109":4,"171":1,"179":4}}],["shate",{"2":{"34":1}}],["sharing",{"2":{"34":9}}],["share",{"2":{"34":4}}],["shared",{"2":{"19":2,"34":1,"98":1}}],["shooting",{"2":{"208":1}}],["shortcomings",{"0":{"117":1}}],["shortcut",{"2":{"8":1,"62":1}}],["short",{"2":{"66":1,"175":1}}],["shorthand",{"2":{"8":1}}],["shown",{"2":{"249":1}}],["showcasing",{"2":{"203":1}}],["showing",{"2":{"152":1}}],["shows",{"2":{"142":1,"148":1,"149":1,"152":1,"250":1}}],["showerror",{"2":{"122":1}}],["show",{"2":{"5":2,"77":1,"145":1,"172":1,"204":1,"247":1}}],["shouldn",{"2":{"52":1,"95":1,"142":1,"147":1}}],["should",{"0":{"119":1},"2":{"4":2,"15":1,"20":1,"31":1,"34":1,"39":1,"41":2,"52":2,"53":1,"55":1,"59":1,"63":5,"65":6,"66":1,"69":1,"70":1,"98":1,"99":1,"125":1,"130":1,"133":1,"134":2,"140":2,"141":2,"142":1,"155":1,"173":1,"174":1,"211":1,"233":1,"256":1}}],["stencils",{"2":{"254":2}}],["stepnorm=0",{"2":{"258":1}}],["steprangelen",{"2":{"207":2}}],["steps",{"2":{"52":1,"77":1,"107":1,"249":1}}],["step",{"2":{"52":10,"77":1,"78":1,"107":3,"179":1,"187":1,"195":1,"217":1,"228":1,"236":2,"244":1,"249":3}}],["stime",{"2":{"195":2,"217":2,"228":2}}],["stick",{"2":{"175":1}}],["still",{"0":{"137":1},"2":{"2":1,"8":1,"39":1,"109":1,"124":1,"130":1,"140":1,"141":1,"184":1,"220":1,"233":1,"250":1}}],["stop=6",{"2":{"250":4}}],["stochastic",{"2":{"82":1,"83":1,"179":1}}],["stores",{"2":{"58":1,"64":1,"67":1,"116":1,"134":1,"171":1,"214":1}}],["stored",{"2":{"52":3,"63":1}}],["store",{"2":{"44":2,"67":1,"82":2,"134":1,"208":1,"248":1,"256":1,"257":1}}],["storing",{"2":{"19":1,"123":1,"134":1}}],["std=1e",{"2":{"256":3}}],["std=0",{"2":{"24":1}}],["stdlib",{"2":{"174":1}}],["stdout",{"2":{"122":1}}],["std",{"2":{"19":1,"24":5,"249":1}}],["stylization",{"2":{"19":1,"69":1}}],["stage",{"2":{"144":1}}],["start=false",{"2":{"215":1}}],["start",{"2":{"106":1,"170":1,"220":13,"247":1}}],["starting",{"2":{"93":1,"130":1,"171":1,"218":1}}],["started",{"0":{"75":1,"84":1},"1":{"76":1,"77":1,"78":1,"79":1,"80":1},"2":{"133":1}}],["stacktraces",{"2":{"108":1}}],["stacktrace",{"2":{"102":2,"109":1}}],["stack",{"2":{"66":2,"243":6,"245":2}}],["standard",{"2":{"24":3,"25":6,"63":2,"69":1,"77":1,"145":1,"256":1}}],["stabilities",{"2":{"219":1}}],["stability",{"0":{"220":1},"2":{"8":1,"19":4,"53":1,"59":1,"69":4,"90":1,"97":1,"166":1}}],["stablerng",{"2":{"145":3,"146":3,"147":2,"148":2,"152":3}}],["stablerngs",{"2":{"22":1,"144":1}}],["stable",{"2":{"52":1,"55":5,"59":2,"166":1,"186":1,"214":1,"220":1}}],["stats=false",{"2":{"69":5}}],["stats=true",{"2":{"62":2,"69":7,"109":1,"110":2}}],["stats",{"2":{"52":2,"77":2,"107":1,"244":7}}],["staticint",{"2":{"193":2}}],["staticarrayscore",{"2":{"153":4}}],["staticarrays",{"2":{"153":1}}],["staticsymbol",{"2":{"33":1,"54":2}}],["static",{"2":{"20":1,"39":1,"54":1,"58":1,"77":8,"106":3,"122":2,"153":7,"193":4,"220":58,"236":2}}],["staticbool",{"2":{"17":1,"33":1,"54":1}}],["statistics",{"2":{"19":2,"24":2,"52":2,"69":7,"105":1,"182":1,"191":1,"212":1,"223":1,"231":1,"235":1,"239":1,"249":1}}],["state=false",{"2":{"66":4}}],["state=zeros32",{"2":{"66":3}}],["statements",{"2":{"59":1}}],["statefulneuralode",{"2":{"218":4,"219":1,"220":5}}],["statefulrecurrentcell",{"2":{"66":2}}],["statefulluxlayers",{"2":{"144":1}}],["statefulluxlayer",{"2":{"58":1,"103":4,"144":7,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"208":2,"218":2,"242":7,"244":2,"249":2,"256":2}}],["stateful",{"0":{"58":1,"218":1,"219":1},"2":{"66":2,"145":1,"146":1,"148":2,"220":6}}],["stateless",{"2":{"8":1,"68":1}}],["statelength",{"2":{"7":1,"10":1,"140":4,"141":1}}],["state",{"0":{"143":1},"2":{"7":1,"8":3,"10":5,"17":2,"19":1,"31":1,"37":1,"52":7,"58":4,"59":7,"62":6,"64":4,"66":64,"68":1,"69":4,"77":8,"78":4,"82":3,"83":3,"107":4,"113":5,"134":1,"140":5,"141":3,"143":4,"184":3,"187":7,"195":8,"208":6,"228":11,"235":1,"244":6,"249":2,"256":1}}],["states",{"0":{"10":1},"2":{"3":1,"7":4,"8":1,"10":2,"31":4,"32":6,"33":1,"34":1,"37":1,"39":1,"52":2,"53":2,"56":2,"57":4,"58":1,"59":5,"62":16,"64":3,"66":5,"68":2,"69":7,"77":2,"78":2,"95":2,"107":1,"109":2,"110":1,"113":2,"121":1,"126":3,"129":1,"134":3,"138":1,"140":5,"141":3,"143":1,"179":2,"184":2,"187":2,"188":1,"193":2,"195":2,"214":1,"217":2,"228":4,"233":1,"235":1,"236":4,"244":1,"249":1,"256":1}}],["st",{"2":{"7":2,"8":4,"10":6,"31":2,"32":6,"34":1,"37":4,"39":2,"52":1,"53":2,"54":2,"57":1,"58":4,"59":19,"62":6,"66":1,"68":14,"77":4,"78":6,"103":2,"106":14,"107":6,"109":5,"110":7,"113":2,"119":3,"121":7,"122":2,"123":10,"124":2,"126":9,"127":7,"128":6,"129":2,"133":3,"134":6,"140":7,"141":11,"142":3,"143":2,"145":7,"146":7,"147":6,"148":6,"150":2,"151":2,"152":9,"153":8,"155":5,"179":6,"184":10,"186":4,"187":7,"188":3,"194":4,"195":2,"208":5,"214":4,"215":4,"216":4,"217":2,"218":7,"220":16,"227":4,"228":2,"235":1,"236":1,"242":7,"244":3,"249":2,"256":2}}],["strain",{"2":{"254":5}}],["strokewidth=2",{"2":{"232":1,"236":2,"248":2,"256":2,"259":4}}],["strokecolor=",{"2":{"232":1,"236":2,"248":2}}],["strongly",{"2":{"173":1}}],["struct",{"2":{"121":1,"134":3,"138":1,"140":1,"188":1,"207":1}}],["structs",{"2":{"56":3}}],["structured",{"2":{"66":3,"142":1,"225":1}}],["structures",{"2":{"55":2,"83":1,"155":1}}],["structure",{"2":{"3":5,"7":2,"8":3,"27":2,"32":4,"34":1,"44":1,"55":3,"67":1,"109":5,"110":13,"134":1,"140":2,"141":2,"143":1,"217":1}}],["strength",{"2":{"53":1}}],["stride=window",{"2":{"65":3}}],["stride=1",{"2":{"63":2}}],["stride",{"2":{"63":5,"65":6}}],["stridearray",{"2":{"39":1}}],["stridearrayscore",{"2":{"39":1}}],["strings",{"2":{"103":1}}],["string=",{"2":{"60":2}}],["string",{"2":{"1":2,"3":1,"48":1,"60":1}}],["surprise",{"2":{"155":1}}],["surpassing",{"2":{"24":2}}],["sure",{"2":{"142":1,"256":1}}],["super",{"2":{"254":1}}],["supertype",{"2":{"141":1}}],["suppose",{"2":{"250":1}}],["supposed",{"2":{"83":1,"102":1,"166":1}}],["supporting",{"2":{"87":1}}],["support",{"0":{"4":1,"72":1,"73":1,"80":1,"88":1,"163":1},"2":{"3":4,"4":4,"7":2,"16":2,"18":2,"31":1,"53":2,"55":1,"58":1,"65":3,"77":2,"78":2,"80":5,"83":3,"85":1,"86":1,"87":1,"89":1,"93":2,"97":1,"117":1,"130":2,"136":1,"142":1}}],["supports",{"2":{"3":1,"59":1,"105":1,"136":1,"163":1,"171":1,"172":1,"240":1}}],["supported",{"0":{"22":1},"2":{"2":1,"3":2,"20":1,"27":2,"28":1,"37":1,"52":6,"57":1,"65":3,"70":1,"77":1,"82":2,"88":2,"89":1,"97":1,"130":2,"160":1,"168":2}}],["supplied",{"2":{"42":1,"62":1}}],["supply",{"2":{"42":1}}],["suggests",{"2":{"68":1}}],["subtracts",{"2":{"171":1}}],["subtract",{"2":{"171":1}}],["subtyping",{"2":{"141":1}}],["subtype",{"2":{"140":1,"141":2}}],["subarray",{"2":{"54":1}}],["suboptimal",{"2":{"52":1,"153":1}}],["sumit",{"2":{"53":1}}],["sum",{"2":{"48":3,"49":2,"59":5,"110":2,"133":2,"134":2,"145":1,"146":1,"147":2,"148":1,"150":1,"151":1,"186":1,"194":1,"216":1,"227":1,"242":3,"254":1}}],["summary",{"2":{"5":8,"134":1,"249":1}}],["success",{"2":{"258":1}}],["successor",{"2":{"112":1}}],["successfully",{"2":{"1":1}}],["such",{"2":{"4":2,"6":1,"8":1,"59":1,"63":2,"65":6,"83":1,"106":1,"111":1,"169":1,"185":1,"225":1}}],["seaborn",{"2":{"250":3}}],["seamlessly",{"2":{"52":1}}],["seq",{"2":{"183":1}}],["sequentially",{"2":{"62":2,"66":5}}],["sequences",{"2":{"183":1}}],["sequence=true",{"2":{"66":2}}],["sequence",{"2":{"62":1,"66":9,"183":7,"184":3}}],["separately",{"2":{"172":1}}],["separation",{"2":{"83":1}}],["separating",{"2":{"83":1}}],["several",{"2":{"161":1,"175":1}}],["sensealg=reversediffadjoint",{"2":{"217":1}}],["sensealg=gaussadjoint",{"2":{"217":1}}],["sensealg=interpolatingadjoint",{"2":{"215":1,"217":1}}],["sensealg",{"2":{"215":1,"217":1,"220":13}}],["sensitivities",{"2":{"214":1,"217":1}}],["sensible",{"2":{"83":1}}],["sensibly",{"2":{"54":1}}],["sensical",{"2":{"59":1}}],["send",{"2":{"84":1}}],["sendbuf",{"2":{"44":6}}],["sendrecvbuf",{"2":{"44":6}}],["seyed",{"2":{"53":1}}],["segmentation",{"2":{"53":2}}],["sec",{"2":{"249":1}}],["section",{"2":{"39":1,"62":2,"76":1,"111":1,"146":1,"162":1,"172":1,"254":1}}],["seconds",{"2":{"249":2}}],["second",{"2":{"17":1,"62":1,"171":1,"177":1,"254":2}}],["server",{"2":{"180":1,"189":1,"196":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["serves",{"2":{"37":1}}],["serious",{"2":{"238":1}}],["seriously",{"2":{"77":1}}],["serializes",{"2":{"113":1}}],["serialization",{"2":{"113":1,"188":1}}],["series",{"2":{"72":2}}],["sergey",{"2":{"19":1}}],["semvar",{"2":{"103":1}}],["semver",{"2":{"30":1}}],["semantic",{"2":{"145":2}}],["semantically",{"2":{"62":1}}],["semantics",{"2":{"5":2,"155":1}}],["semi",{"2":{"24":2}}],["self",{"2":{"17":1}}],["selu",{"2":{"17":1}}],["selecting",{"2":{"154":1}}],["selection",{"0":{"164":1},"2":{"2":1,"3":1,"131":1,"132":1,"164":1}}],["selectdim",{"2":{"68":1}}],["select",{"2":{"2":4}}],["selected",{"0":{"203":1},"2":{"2":3,"3":1,"102":1}}],["selects",{"2":{"2":1}}],["seems",{"2":{"258":2}}],["seeding",{"2":{"77":1}}],["seed",{"2":{"62":1,"68":4,"77":1,"78":1,"83":1,"129":1,"140":1,"142":1,"170":1,"174":2,"179":1,"215":1,"232":1,"244":3,"248":1}}],["see",{"2":{"3":1,"7":3,"8":3,"11":1,"14":1,"15":2,"17":2,"19":4,"31":1,"33":2,"34":1,"37":1,"39":2,"49":1,"52":1,"53":1,"55":2,"57":1,"58":2,"59":2,"60":1,"62":4,"64":3,"66":5,"68":2,"69":4,"70":1,"76":1,"83":2,"92":1,"95":3,"97":1,"102":2,"104":2,"106":1,"107":1,"108":1,"109":3,"110":2,"112":1,"116":1,"141":1,"155":2,"159":1,"162":1,"165":1,"166":1,"168":1,"171":2,"172":1,"184":1,"196":1,"219":1,"250":2,"254":2}}],["session",{"2":{"1":1,"60":1}}],["setprogress",{"2":{"247":1}}],["sets",{"2":{"48":1,"166":2}}],["setindexing",{"2":{"16":1,"18":1}}],["setups",{"2":{"148":1}}],["setup",{"2":{"8":1,"32":1,"34":1,"37":1,"39":1,"59":6,"62":1,"68":4,"77":1,"78":1,"106":1,"107":1,"109":2,"110":2,"113":1,"122":1,"124":1,"126":1,"129":1,"133":1,"134":1,"140":2,"141":1,"142":1,"145":1,"146":1,"147":1,"148":1,"152":1,"153":1,"155":1,"179":1,"187":1,"195":1,"208":1,"215":1,"228":1,"235":1,"244":1,"249":1,"256":1}}],["setfield",{"2":{"7":1,"95":1,"223":1}}],["setting",{"0":{"257":1},"2":{"4":2,"63":2,"66":4,"67":3,"110":1,"144":1,"157":1,"166":2,"167":1}}],["set",{"2":{"1":2,"2":1,"4":6,"17":4,"19":4,"20":1,"24":1,"31":2,"34":1,"37":2,"39":1,"42":2,"48":1,"49":1,"53":2,"55":1,"58":1,"59":3,"60":5,"63":2,"64":1,"66":18,"70":2,"73":3,"90":1,"110":2,"116":1,"131":1,"144":1,"145":4,"157":2,"161":2,"162":1,"163":3,"164":3,"166":2,"179":3,"247":1}}],["os",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["oops",{"2":{"155":1}}],["ok",{"2":{"147":1}}],["own",{"2":{"117":1,"142":1,"171":1}}],["oh",{"2":{"110":1}}],["our",{"2":{"77":1,"83":1,"88":1,"110":3,"136":1,"145":1,"149":1,"155":2,"170":1,"171":1,"173":1,"174":1,"176":1,"179":2,"183":1,"184":1,"204":1,"208":3,"235":1,"236":1,"238":1,"248":1,"249":3,"250":4}}],["outperforms",{"2":{"89":1}}],["outpad",{"2":{"63":3,"105":1}}],["outpad=0",{"2":{"63":1}}],["outputs",{"2":{"11":1,"33":1,"39":1,"58":1,"62":2,"66":6,"68":2,"82":1,"107":1,"146":1,"147":1,"174":1,"176":1,"256":1}}],["outputsize",{"2":{"11":3,"95":1}}],["output",{"2":{"11":1,"13":2,"16":1,"17":3,"18":1,"24":6,"33":1,"39":2,"53":6,"58":1,"59":2,"62":7,"63":8,"65":33,"66":13,"67":7,"68":4,"69":1,"70":4,"78":1,"105":1,"106":2,"109":4,"110":14,"140":1,"143":2,"153":13,"155":1,"174":1,"184":1,"214":1,"241":1,"249":2}}],["outside",{"2":{"55":1,"102":2}}],["out",{"2":{"17":2,"24":2,"30":1,"55":1,"59":10,"62":1,"63":8,"64":2,"65":3,"66":15,"67":19,"78":6,"90":2,"108":1,"110":1,"138":1,"140":8,"143":1,"157":1,"184":3,"185":2,"217":1,"247":1}}],["o=σ",{"2":{"66":1}}],["oi=",{"2":{"63":1,"65":3}}],["o",{"2":{"63":4,"65":6}}],["odefunction",{"2":{"214":2,"218":1}}],["odeproblem",{"2":{"206":1,"208":1,"209":1,"214":2,"218":1,"255":1,"256":1,"259":1}}],["odesolution",{"2":{"214":1}}],["odes",{"0":{"211":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1},"2":{"83":1,"120":1,"211":2,"255":1,"256":1}}],["ode",{"0":{"214":1,"215":1,"219":1,"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"58":1,"120":1,"204":3,"206":3,"207":1,"208":2,"209":2,"214":2,"215":1,"255":3,"256":6,"257":1,"259":2}}],["odd",{"2":{"24":1}}],["old",{"2":{"52":1,"92":1,"96":2,"130":2}}],["older",{"0":{"36":1},"1":{"37":1},"2":{"102":1,"116":1}}],["observe",{"2":{"249":1}}],["observations",{"2":{"63":1}}],["obtain",{"2":{"113":1}}],["obtained",{"2":{"20":1}}],["obviously",{"2":{"108":1}}],["obj",{"2":{"52":2,"141":2,"142":1}}],["objects",{"2":{"53":1,"99":3,"106":1,"132":1,"155":1,"207":1}}],["objective",{"2":{"52":8,"257":1}}],["object",{"2":{"2":3,"3":4,"16":1,"27":2,"52":6,"53":2,"113":1,"131":3}}],["occurs",{"2":{"110":1}}],["occurred",{"2":{"110":3}}],["occurrences",{"2":{"10":3}}],["octavian",{"0":{"167":1},"2":{"18":2,"159":1,"167":1}}],["other",{"0":{"25":1,"35":1},"1":{"36":1,"37":1,"38":1,"39":1},"2":{"5":1,"8":1,"16":1,"24":1,"25":1,"39":1,"44":2,"49":1,"53":1,"55":2,"62":1,"63":1,"68":2,"70":1,"77":1,"78":1,"89":1,"93":1,"102":1,"107":1,"138":1,"140":1,"141":1,"171":1,"175":1,"188":1,"204":1}}],["otherwisewhere",{"2":{"53":1}}],["otherwise",{"2":{"1":1,"2":2,"3":1,"60":1,"67":1}}],["opening",{"2":{"144":1}}],["open",{"2":{"87":1,"106":1,"111":1,"117":1,"153":1,"202":1,"203":2}}],["operating",{"2":{"66":1}}],["operation",{"2":{"16":2,"18":1,"20":4,"44":4,"65":3,"70":1,"175":1}}],["operations",{"0":{"14":1,"55":1},"2":{"2":1,"16":3,"18":1,"28":1,"39":1,"54":2,"55":1,"93":1,"144":2,"153":2,"167":1,"172":1}}],["operates",{"2":{"106":1}}],["operate",{"2":{"66":1,"134":1}}],["operators",{"2":{"68":1}}],["operator",{"2":{"55":1}}],["optprob",{"2":{"258":2}}],["optf",{"2":{"258":2}}],["opt",{"2":{"48":6,"113":5,"179":2,"208":10,"236":1}}],["optimal",{"2":{"208":1}}],["optim",{"2":{"59":2}}],["optimiser",{"0":{"186":1}}],["optimisers",{"2":{"37":2,"45":1,"52":2,"59":3,"77":10,"78":1,"106":1,"113":1,"134":2,"137":1,"179":2,"181":1,"182":1,"191":1,"204":1,"208":2,"212":1,"223":1,"231":1,"234":1,"239":1,"244":1}}],["optimizer",{"0":{"234":1},"2":{"45":4,"52":4,"113":3,"236":2}}],["optimizers",{"0":{"45":1},"2":{"258":1}}],["optimized",{"2":{"20":1,"159":1,"167":1,"208":1}}],["optimizationproblem",{"2":{"208":2,"258":1}}],["optimizationfunction",{"2":{"208":1,"258":1}}],["optimizationoptimjl",{"2":{"205":1,"253":1}}],["optimizationoptimisers",{"2":{"205":1}}],["optimizations",{"2":{"102":1,"106":1}}],["optimization",{"0":{"204":1},"1":{"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"24":1,"77":1,"78":3,"106":1,"204":4,"205":1,"208":8,"253":1,"258":4}}],["options",{"2":{"57":1,"164":1}}],["option",{"2":{"20":1,"57":2,"70":2,"87":2}}],["optional",{"0":{"159":1},"2":{"16":1,"18":1,"24":1,"31":3,"53":1,"58":1,"59":2,"66":1,"95":1,"167":1,"169":1}}],["optionally",{"2":{"7":1,"24":1,"69":1,"140":1,"169":1}}],["op",{"2":{"4":2,"5":1,"19":1,"44":6,"48":1,"54":3,"57":1,"100":1}}],["orbit₂",{"2":{"254":2}}],["orbit₁",{"2":{"254":2}}],["orbits",{"2":{"254":1}}],["orbit2",{"2":{"254":4}}],["orbit2tensor",{"2":{"254":2}}],["orbit1",{"2":{"254":4}}],["orbit",{"2":{"254":17}}],["orange",{"2":{"232":1,"236":1}}],["ordinarydiffeqloworderrk",{"2":{"253":1}}],["ordinarydiffeqcore",{"2":{"220":26}}],["ordinarydiffeq",{"2":{"214":1,"255":1}}],["ordinarydiffeqtsit5",{"2":{"205":1,"212":1,"220":13}}],["ordering",{"2":{"66":6}}],["order",{"0":{"29":1},"2":{"2":1,"31":1,"63":1,"86":1,"145":1,"147":1,"177":1,"240":3,"242":1,"254":2}}],["orcjit",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["originates",{"2":{"33":1}}],["originally",{"2":{"252":1}}],["original",{"2":{"24":1,"31":1,"34":1,"37":1,"106":1,"109":1,"173":2}}],["org",{"2":{"24":1,"68":1,"74":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"254":2,"260":1}}],["orthogonal",{"2":{"22":4,"24":6}}],["or",{"2":{"2":3,"3":2,"4":2,"8":1,"16":2,"17":3,"18":1,"19":2,"20":2,"24":2,"31":1,"33":3,"37":1,"39":2,"44":3,"52":1,"53":5,"54":2,"59":3,"60":1,"62":7,"63":6,"65":9,"66":3,"67":7,"70":4,"72":1,"76":1,"77":1,"83":2,"84":1,"87":1,"107":1,"110":2,"116":2,"117":1,"134":1,"136":1,"140":2,"141":1,"144":1,"147":1,"155":1,"158":2,"163":1,"169":1,"171":4,"172":1,"183":1,"202":1,"203":2,"254":1}}],["overloaded",{"2":{"102":1}}],["overloading",{"2":{"55":1}}],["overview",{"0":{"86":1}}],["overcome",{"2":{"37":1}}],["overridden",{"2":{"18":1}}],["overrides",{"2":{"49":1}}],["override",{"2":{"11":1,"37":1}}],["overfitting",{"2":{"17":2}}],["over",{"0":{"32":1,"83":1},"2":{"2":1,"5":2,"32":1,"33":1,"58":1,"66":1,"68":1,"69":3,"77":1,"125":1,"181":1,"207":1,"236":1,"238":1,"241":1,"257":1,"259":1}}],["onlinestats",{"2":{"239":1,"244":4}}],["online",{"2":{"203":1}}],["only",{"2":{"2":1,"8":1,"33":2,"49":3,"52":1,"53":3,"57":2,"59":1,"63":1,"64":1,"65":3,"66":9,"67":1,"70":1,"106":1,"113":1,"128":1,"141":1,"142":1,"144":2,"147":1,"148":1,"153":2,"176":2,"188":1,"213":1,"242":3}}],["once",{"2":{"11":1,"57":1,"66":2,"110":1,"140":1,"172":1,"173":1,"175":1}}],["onwards",{"2":{"7":1,"102":1}}],["on",{"0":{"118":1,"156":1,"222":1,"238":1},"1":{"119":1,"120":1,"121":1,"122":1,"123":1,"124":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"2":2,"3":6,"5":1,"6":2,"7":1,"8":2,"13":2,"14":3,"15":1,"16":3,"18":3,"19":4,"24":11,"29":2,"30":1,"31":1,"32":2,"39":1,"44":2,"48":1,"53":5,"55":3,"58":2,"59":2,"60":1,"62":2,"63":2,"64":2,"65":6,"66":2,"67":1,"71":1,"74":1,"76":1,"77":1,"83":1,"84":2,"87":2,"89":1,"93":1,"95":2,"103":1,"104":2,"105":1,"106":2,"111":1,"113":1,"117":1,"119":2,"123":1,"134":4,"138":2,"140":2,"143":1,"144":4,"145":4,"146":1,"147":2,"153":3,"155":2,"156":1,"157":1,"159":1,"160":2,"161":2,"162":1,"165":1,"166":1,"168":1,"170":1,"171":1,"172":1,"174":3,"177":2,"180":1,"189":1,"196":2,"197":1,"202":1,"204":1,"207":1,"208":2,"210":1,"217":2,"221":1,"225":1,"229":1,"230":1,"237":1,"243":2,"246":1,"249":1,"251":1,"260":1}}],["one2two",{"2":{"254":3}}],["onecold",{"2":{"194":2,"216":2,"227":2}}],["onehotbatch",{"2":{"192":1,"195":1,"213":1,"224":2}}],["onehotarrays",{"2":{"191":1,"212":1,"223":1}}],["onearray",{"2":{"22":3}}],["oneapidevice",{"2":{"4":2}}],["oneapi",{"2":{"2":1,"3":2,"22":1,"72":2,"77":1,"78":1,"80":2,"130":2,"164":1}}],["onesc64",{"2":{"25":1}}],["onesc32",{"2":{"25":1}}],["onesc16",{"2":{"25":1}}],["ones64",{"2":{"25":1}}],["ones32",{"2":{"25":1}}],["ones16",{"2":{"25":1}}],["ones",{"2":{"8":1,"25":7,"59":1,"121":1,"171":2,"177":1,"220":1,"243":2,"248":1,"249":1}}],["one",{"2":{"1":1,"33":1,"53":1,"56":1,"62":2,"66":1,"69":2,"70":2,"113":1,"144":1,"158":2,"169":1,"171":1,"202":1,"203":1,"249":1,"254":4}}],["official",{"2":{"154":1,"180":1,"189":1,"197":1,"203":1,"204":1,"210":1,"221":1,"229":1,"237":1,"246":1,"247":2,"251":1,"260":1}}],["off",{"2":{"149":1}}],["offending",{"2":{"110":9}}],["ofcourse",{"2":{"109":1}}],["often",{"2":{"3":1,"67":1,"88":2,"120":1,"143":1,"149":1,"152":1,"155":1,"158":2,"171":2,"204":1,"249":1}}],["of",{"0":{"126":1,"128":1,"129":1,"148":1},"2":{"1":1,"3":7,"4":3,"6":1,"7":2,"8":17,"9":3,"10":6,"11":4,"14":2,"15":2,"16":1,"17":4,"18":1,"19":8,"24":31,"25":36,"27":6,"28":6,"30":2,"31":6,"32":11,"33":2,"34":9,"37":7,"39":3,"42":2,"43":1,"47":1,"48":1,"49":7,"52":11,"53":13,"54":1,"55":10,"56":4,"57":10,"58":8,"59":16,"62":32,"63":38,"64":3,"65":42,"66":37,"67":32,"68":23,"69":30,"70":13,"71":1,"74":3,"76":1,"78":2,"83":5,"85":2,"89":1,"90":3,"95":2,"96":1,"97":2,"98":2,"100":2,"102":6,"104":2,"105":1,"106":6,"107":1,"109":1,"110":6,"113":4,"116":2,"120":2,"124":1,"126":2,"127":1,"128":3,"134":1,"138":3,"140":3,"141":4,"142":1,"143":3,"145":1,"146":6,"147":1,"148":1,"149":6,"153":1,"155":1,"158":4,"159":1,"161":1,"162":1,"163":1,"166":4,"167":1,"168":1,"171":9,"172":1,"173":1,"175":4,"177":1,"179":3,"181":1,"183":1,"184":3,"185":2,"186":1,"188":1,"196":1,"202":2,"203":2,"204":1,"211":1,"214":5,"218":1,"219":1,"220":1,"225":2,"236":2,"238":3,"240":1,"243":1,"247":2,"248":2,"249":6,"250":3,"254":5,"255":2,"256":3}}],["bfgs",{"2":{"204":1,"208":1,"258":2}}],["bs",{"2":{"192":1,"213":1}}],["bh2",{"2":{"254":1}}],["bh",{"2":{"254":1}}],["bhn",{"2":{"66":2}}],["bhz",{"2":{"66":2}}],["bhr",{"2":{"66":2}}],["bright",{"2":{"250":3}}],["break",{"2":{"244":1}}],["breaking",{"0":{"92":1,"95":1,"99":1,"102":1,"103":1,"104":1},"2":{"97":1}}],["branched",{"2":{"62":1}}],["branchlayer",{"2":{"62":5}}],["broken=true",{"2":{"48":2}}],["broken=false",{"2":{"48":2}}],["broken",{"2":{"30":1,"48":5,"49":4,"50":1}}],["broadcastfunction",{"2":{"106":1}}],["broadcastable",{"2":{"69":1}}],["broadcast",{"2":{"44":1,"64":1,"68":1,"106":1,"249":1}}],["broadcasted",{"2":{"15":1,"16":2,"44":2,"53":1}}],["broadcasting",{"2":{"13":1,"20":1,"171":1}}],["b=f",{"2":{"134":1}}],["b=layer",{"2":{"134":1}}],["b=",{"2":{"59":1}}],["b=zeros",{"2":{"59":1}}],["bc",{"2":{"242":7,"243":21,"244":120}}],["bce",{"2":{"53":5}}],["bcast",{"2":{"44":2}}],["bigger",{"2":{"171":1}}],["bigfloat",{"2":{"171":3}}],["bit",{"2":{"147":1,"208":1}}],["bibtex",{"2":{"74":2}}],["bilinear",{"2":{"67":3,"70":6,"104":1}}],["bidirectional",{"2":{"66":1}}],["bidirectionalrnn",{"2":{"66":1}}],["biz",{"2":{"66":1}}],["bir",{"2":{"66":1}}],["bin",{"2":{"53":7,"66":1}}],["binarycrossentropy",{"2":{"186":2}}],["binarycrossentropyloss",{"2":{"53":7,"186":1}}],["binaryfocalloss",{"2":{"53":3}}],["binary",{"2":{"53":3,"183":1}}],["bias=l",{"2":{"140":1}}],["bias=false",{"2":{"66":6,"67":3}}],["bias=true",{"2":{"63":4,"66":2,"67":6}}],["bias=nothing",{"2":{"63":1,"66":3,"67":3}}],["bias=ps",{"2":{"59":1}}],["bias=zeros32",{"2":{"63":1,"67":1,"69":4,"140":1,"256":3}}],["bias=zero",{"2":{"32":1}}],["biases",{"2":{"24":1,"249":1}}],["bias",{"0":{"15":1},"2":{"15":12,"16":3,"18":3,"19":15,"31":1,"32":4,"34":4,"59":1,"63":12,"66":25,"67":22,"69":20,"77":15,"105":4,"106":12,"110":3,"122":2,"126":4,"127":2,"128":2,"129":1,"140":6,"142":4,"145":3,"147":4,"148":4,"158":5,"166":1,"179":1,"220":30,"235":2,"249":1,"256":3,"258":3}}],["bn=batchnorm",{"2":{"32":1}}],["black",{"2":{"232":1,"236":2,"248":2}}],["blas",{"2":{"8":1,"18":1}}],["blue",{"2":{"206":1,"209":2,"232":1,"248":1}}],["block",{"2":{"59":3,"62":1,"65":3,"184":1}}],["blocks",{"2":{"52":1,"66":1,"184":1}}],["blog",{"2":{"39":1}}],["blisblas",{"2":{"18":1}}],["b",{"2":{"16":4,"17":2,"18":3,"28":3,"59":6,"65":6,"67":4,"109":2,"131":1,"134":9,"142":2,"146":8,"158":2,"179":8,"244":8,"254":4}}],["bulk",{"2":{"249":1}}],["builds",{"2":{"109":1,"220":3,"244":4}}],["buildkite",{"2":{"109":1,"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"220":3,"221":1,"229":1,"237":1,"244":4,"246":1,"251":1,"260":1}}],["build",{"2":{"77":1,"163":3,"170":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["building",{"0":{"249":1},"2":{"52":1}}],["built",{"0":{"61":1},"1":{"62":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"70":1},"2":{"81":1,"138":2,"173":1,"196":1}}],["buffer",{"2":{"16":1,"18":1,"44":3}}],["bugs",{"2":{"84":1,"98":1,"142":1}}],["bug",{"2":{"11":1,"59":1,"153":1,"202":1}}],["but",{"2":{"3":2,"7":2,"8":1,"13":1,"14":1,"15":1,"19":1,"24":1,"27":2,"37":1,"39":3,"54":3,"55":5,"57":2,"59":4,"68":1,"70":2,"82":1,"83":2,"85":1,"88":3,"89":2,"95":1,"106":1,"109":3,"110":2,"112":1,"130":1,"140":2,"141":1,"142":1,"145":3,"146":1,"147":1,"152":1,"153":1,"155":2,"160":1,"167":1,"179":1,"184":2,"186":1,"207":1,"217":1,"219":1,"233":1,"243":1,"249":1,"256":1}}],["box",{"2":{"248":1}}],["boxing",{"2":{"58":1,"218":1,"220":1}}],["borrow",{"2":{"247":1}}],["borders",{"2":{"63":2,"65":3}}],["book",{"2":{"177":1}}],["boom",{"2":{"147":1}}],["bool=true",{"2":{"66":1}}],["bool=false",{"2":{"2":2,"37":2,"39":1,"48":1,"66":4,"215":1,"217":1}}],["boolean",{"2":{"8":1,"39":1,"50":1}}],["bool",{"2":{"2":1,"3":5,"8":3,"33":1,"39":1,"53":2,"54":1,"55":1,"220":26}}],["boilerplate",{"2":{"106":1,"107":1,"185":1}}],["boundaries",{"2":{"250":1}}],["boundary",{"2":{"242":1}}],["bound",{"2":{"63":6,"66":18,"67":9}}],["bounds",{"2":{"63":2,"66":6,"67":3}}],["bottom",{"2":{"59":1,"106":1}}],["both",{"2":{"2":1,"33":2,"53":2,"77":1,"81":1,"105":1,"110":3,"134":1,"163":1,"166":1,"203":1,"249":1}}],["body",{"2":{"59":2,"220":3,"254":9}}],["bypasses",{"2":{"59":2}}],["bypass",{"2":{"11":1,"141":1,"164":1}}],["by",{"0":{"127":1},"2":{"2":1,"7":1,"8":2,"11":1,"13":2,"16":1,"18":1,"19":1,"24":5,"39":1,"52":3,"53":2,"54":1,"56":1,"59":2,"62":2,"63":5,"65":3,"66":5,"67":5,"68":1,"69":1,"70":1,"76":1,"77":1,"83":2,"95":1,"102":1,"106":2,"110":1,"113":3,"127":3,"128":2,"131":1,"132":1,"138":1,"144":3,"145":2,"148":1,"149":1,"155":1,"156":1,"157":1,"163":1,"167":1,"171":2,"172":2,"173":1,"175":1,"181":1,"193":1,"203":1,"207":1,"220":1,"235":2,"247":1,"249":1,"250":1}}],["bayes",{"2":{"249":3}}],["bayesian",{"0":{"247":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"248":1,"250":1}}],["battery",{"2":{"83":1}}],["batches",{"2":{"192":1,"213":1}}],["batched",{"0":{"14":1,"28":1,"146":1},"2":{"11":1,"14":6,"28":1,"144":1,"146":8,"150":1,"151":1,"158":2}}],["batchsize=32",{"2":{"244":2}}],["batchsize=256",{"2":{"224":1}}],["batchsize=min",{"2":{"224":2}}],["batchsize=8",{"2":{"207":2}}],["batchsize=128",{"2":{"183":2}}],["batchsize=12",{"2":{"160":2}}],["batchsize=13",{"2":{"5":1}}],["batchsize",{"2":{"192":3,"213":3,"224":4}}],["batching",{"2":{"153":3}}],["batchlastindex",{"2":{"54":1,"66":2}}],["batchnorm",{"2":{"7":1,"19":1,"62":4,"69":8,"109":4,"110":4,"126":1,"129":1,"133":2,"136":1,"140":1,"143":1,"145":2,"146":1}}],["batch",{"2":{"5":1,"19":6,"28":3,"39":1,"63":3,"65":3,"66":13,"68":1,"69":6,"70":3,"92":1,"146":1,"153":6,"183":1,"207":1,"208":9,"244":8}}],["basis",{"2":{"63":2,"67":1}}],["basically",{"2":{"141":1}}],["basic",{"2":{"52":2,"170":1}}],["baseline",{"2":{"53":1}}],["base",{"2":{"48":3,"54":3,"64":1,"68":1,"102":1,"122":1,"142":1,"144":3,"147":1,"148":2,"204":1,"207":2,"215":1,"220":25,"256":2}}],["based",{"2":{"2":1,"3":2,"18":1,"19":1,"55":1,"62":1,"84":1,"104":2,"105":1,"145":1,"149":1,"170":1,"204":1}}],["bad",{"2":{"33":1,"155":1,"250":1}}],["ba",{"2":{"19":1}}],["backtracking",{"2":{"258":1}}],["background",{"2":{"140":1,"155":1,"255":1}}],["backward",{"2":{"33":2,"54":1,"66":11,"110":3,"111":1,"173":1}}],["back",{"2":{"2":1,"3":1,"13":1,"142":1}}],["backendtpu",{"2":{"73":1}}],["backendgpu",{"2":{"73":1}}],["backends=",{"2":{"49":2}}],["backends",{"0":{"41":1},"2":{"1":1,"2":4,"3":5,"5":1,"16":3,"18":4,"20":1,"27":6,"28":2,"49":7,"52":5,"82":2,"88":3,"100":1,"130":3,"153":1,"160":1,"162":1,"172":1,"177":1,"217":1}}],["backend",{"0":{"38":1,"131":1,"132":1,"164":1},"1":{"39":1},"2":{"1":17,"2":3,"3":1,"12":1,"27":6,"28":3,"41":5,"42":20,"43":4,"44":11,"45":1,"46":1,"49":1,"52":4,"73":4,"93":1,"113":14,"114":2,"116":1,"131":5,"144":1,"153":6,"164":4,"175":1,"204":1}}],["bernoulli",{"2":{"249":1}}],["became",{"2":{"155":1}}],["because",{"2":{"142":1,"258":1}}],["become",{"2":{"140":1,"181":1}}],["benchmarking",{"2":{"196":1}}],["benefits",{"2":{"106":1}}],["bengio",{"2":{"24":2,"68":1}}],["behind",{"2":{"95":1}}],["behaving",{"2":{"169":1}}],["behaviour",{"2":{"95":1}}],["behavior",{"2":{"3":2,"7":2,"8":1,"11":2,"24":1,"57":3,"66":1,"145":2,"155":1,"184":1}}],["behaves",{"2":{"62":1,"96":1,"160":1}}],["behave",{"2":{"54":1}}],["better",{"2":{"83":1,"84":1}}],["between",{"0":{"35":1},"1":{"36":1,"37":1,"38":1,"39":1},"2":{"24":1,"34":1,"37":1,"53":1,"67":1,"83":1,"106":1,"116":1,"134":3,"250":3}}],["beginner",{"0":{"199":1}}],["begin",{"2":{"63":2,"65":3,"206":1,"209":1,"243":1,"245":1,"255":1,"256":1,"259":1}}],["beyond",{"2":{"62":1}}],["been",{"2":{"37":2,"83":1,"88":1,"92":2,"95":6,"99":3,"102":10,"103":5,"105":1,"115":1,"148":1,"170":1,"252":1}}],["below",{"2":{"32":1,"57":2,"62":1,"74":1,"214":1,"248":1,"249":2,"250":1}}],["beware",{"2":{"25":1,"142":1}}],["being",{"2":{"15":1,"17":2,"19":2,"55":1,"69":4,"87":2,"93":1,"100":1,"117":1,"144":1,"145":2,"155":1,"196":1,"208":1}}],["best",{"2":{"13":2,"16":1,"18":1,"37":1,"59":2,"87":2,"88":1}}],["before",{"2":{"8":2,"24":2,"42":1,"45":1,"46":1,"62":1,"78":1,"107":1,"113":1,"140":1,"174":1,"188":1}}],["be",{"2":{"1":2,"2":1,"3":10,"4":5,"6":1,"7":4,"8":2,"10":1,"13":2,"14":1,"15":3,"16":1,"17":4,"18":1,"19":20,"20":1,"24":8,"28":5,"31":2,"33":3,"34":5,"37":5,"39":5,"42":7,"44":2,"46":2,"48":3,"49":3,"50":4,"52":9,"53":5,"55":3,"58":2,"59":16,"60":2,"62":13,"63":16,"64":4,"65":9,"66":17,"67":8,"68":8,"69":4,"70":3,"71":1,"74":1,"77":1,"81":1,"82":3,"83":2,"84":1,"90":1,"98":1,"99":1,"102":1,"103":1,"106":3,"108":1,"111":1,"113":2,"114":1,"121":1,"125":1,"127":1,"130":2,"132":1,"133":1,"134":6,"136":1,"138":1,"140":1,"142":2,"143":1,"144":2,"145":5,"146":2,"148":1,"150":1,"155":1,"159":1,"164":1,"166":3,"167":2,"169":3,"171":1,"173":1,"174":1,"175":1,"176":1,"181":1,"184":1,"196":2,"202":1,"203":1,"208":1,"218":1,"226":1,"233":1,"238":2,"243":1,"248":1,"249":1,"254":4,"257":1}}],["ixy",{"2":{"254":2}}],["ixx",{"2":{"254":3}}],["iyy",{"2":{"254":3}}],["i∈",{"2":{"179":1}}],["i9rhw",{"2":{"145":2}}],["ibkvy",{"2":{"122":3}}],["iid",{"2":{"249":1}}],["iii",{"2":{"86":2,"88":1}}],["ii",{"2":{"86":2,"88":1}}],["ii+pi+p",{"2":{"63":1,"65":3}}],["i=σ",{"2":{"66":1}}],["ih",{"2":{"66":6,"105":1}}],["i+n",{"2":{"63":1,"65":3}}],["ignore",{"2":{"225":1,"256":1}}],["ignores",{"2":{"37":1}}],["ignored",{"2":{"2":1,"24":1,"67":3}}],["iclr",{"2":{"24":1}}],["ieee",{"2":{"24":2,"53":5}}],["ioffe",{"2":{"19":1}}],["imgs",{"2":{"192":5,"213":5,"224":4}}],["im",{"0":{"173":1}}],["immutability",{"2":{"174":1}}],["immutable",{"2":{"13":1,"16":1,"18":1,"55":1,"82":1,"83":1,"173":1}}],["immediately",{"2":{"106":1}}],["imbalanced",{"2":{"53":1}}],["images",{"2":{"69":3,"70":2,"192":1,"213":1}}],["image",{"2":{"53":1,"63":2}}],["imagenet",{"2":{"24":2,"112":1}}],["imply",{"2":{"143":1}}],["implements",{"2":{"47":1}}],["implemented",{"2":{"37":1,"83":1,"134":1,"144":1}}],["implementations",{"2":{"13":1,"14":1,"18":1,"20":1,"52":1,"55":1,"116":1,"167":1}}],["implementation",{"0":{"121":1,"135":1,"218":1},"1":{"136":1},"2":{"11":4,"13":3,"16":4,"18":6,"31":1,"52":1,"55":5,"58":2,"62":1,"66":1,"95":1,"116":1,"149":1,"214":1}}],["implement",{"0":{"225":1},"2":{"7":1,"31":1,"134":1,"140":2,"247":1}}],["implementing",{"0":{"134":1},"2":{"7":2,"83":1,"120":1,"211":1,"247":1}}],["improving",{"2":{"89":1}}],["imports",{"0":{"182":1,"191":1,"205":1,"212":1,"223":1,"231":1,"239":1,"253":1}}],["importing",{"2":{"140":1,"170":1,"247":1}}],["important",{"0":{"135":1},"1":{"136":1},"2":{"113":2,"130":1,"134":1,"140":1,"145":1,"146":1}}],["importantly",{"2":{"107":1,"171":1,"250":1}}],["imported",{"2":{"59":1}}],["import",{"2":{"37":1,"39":1,"79":1,"247":1}}],["i",{"2":{"5":10,"11":1,"24":1,"31":1,"37":1,"39":1,"52":2,"53":3,"57":1,"58":1,"59":1,"62":9,"63":6,"65":21,"67":4,"68":8,"70":2,"78":1,"86":4,"88":1,"95":2,"113":1,"121":3,"143":1,"147":1,"153":6,"174":4,"177":1,"179":4,"207":2,"244":3,"245":6,"248":12,"249":7,"250":10}}],["ith",{"2":{"62":1}}],["its",{"2":{"8":1,"65":3,"67":1,"68":1,"69":1,"71":1,"83":1,"250":1}}],["itself",{"2":{"3":1,"68":1,"109":1}}],["iter",{"2":{"78":4,"107":12,"208":2,"244":7,"257":1}}],["iteratively",{"2":{"62":1}}],["iterations",{"2":{"179":12,"249":2}}],["iteration",{"0":{"5":1},"2":{"5":2,"78":12,"107":4,"120":1,"160":1,"174":8,"207":1,"208":26,"244":101,"250":1,"257":1,"259":1}}],["iterate",{"2":{"5":1,"30":1}}],["iterates",{"2":{"5":1}}],["iterators",{"2":{"5":1,"184":1,"185":1,"243":1,"244":2,"245":1}}],["iterator",{"2":{"5":3,"107":1}}],["item",{"2":{"5":1}}],["it",{"2":{"2":1,"3":2,"4":5,"7":3,"8":6,"11":2,"13":3,"15":2,"16":2,"17":1,"24":3,"30":1,"31":1,"32":1,"34":1,"37":3,"47":2,"50":2,"52":1,"53":5,"54":1,"55":3,"57":3,"58":1,"59":5,"62":2,"63":4,"65":3,"66":8,"68":1,"69":7,"70":3,"74":1,"76":1,"78":1,"81":1,"83":4,"85":1,"89":3,"95":1,"103":1,"105":1,"106":5,"108":2,"109":1,"110":3,"113":2,"114":1,"119":1,"131":1,"138":2,"140":2,"141":5,"142":4,"144":3,"145":6,"146":2,"152":1,"153":2,"155":1,"156":2,"157":1,"160":1,"164":1,"167":1,"170":1,"171":2,"172":1,"173":2,"174":2,"177":1,"179":1,"184":4,"186":2,"202":1,"203":1,"204":1,"207":2,"208":2,"214":1,"225":2,"249":2,"254":1,"256":2}}],["inlined",{"2":{"244":4}}],["in2",{"2":{"67":6}}],["in12",{"2":{"67":3}}],["in1",{"2":{"67":8}}],["injection",{"2":{"62":5}}],["independent",{"2":{"146":1}}],["index",{"2":{"62":1,"67":1,"68":2,"153":1,"250":1}}],["indexing",{"0":{"156":1},"2":{"20":1,"62":1,"68":1,"156":1}}],["indexed",{"2":{"2":2,"4":2,"68":1,"84":1}}],["individual",{"2":{"68":1,"208":1}}],["individually",{"2":{"62":1,"70":2,"146":1}}],["indices",{"2":{"67":2}}],["inplace",{"2":{"52":2}}],["input",{"0":{"118":1,"124":1},"1":{"119":1,"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"8":2,"13":2,"15":1,"16":1,"17":3,"18":2,"19":9,"24":4,"27":2,"28":1,"33":2,"39":3,"53":4,"58":1,"59":1,"62":30,"63":7,"64":2,"65":6,"66":20,"67":12,"68":7,"69":14,"70":6,"77":1,"106":1,"109":6,"110":14,"113":1,"114":1,"116":1,"120":1,"140":1,"143":2,"146":3,"148":1,"153":8,"155":2,"165":1,"166":1,"169":1,"184":1,"193":1,"214":1,"241":1,"256":1}}],["inputsize",{"2":{"66":1,"95":1}}],["inputs",{"0":{"146":1},"2":{"8":2,"11":1,"16":3,"18":1,"28":1,"32":1,"33":1,"52":1,"53":4,"55":1,"58":1,"62":9,"63":2,"64":3,"65":9,"66":11,"67":1,"68":6,"69":5,"70":2,"82":1,"103":1,"106":1,"116":1,"144":1,"146":2,"147":1,"148":1,"178":1,"214":2,"235":1,"249":1,"256":1}}],["inner",{"2":{"31":2,"33":1,"127":1}}],["incase",{"2":{"256":1}}],["including",{"2":{"98":1,"106":1}}],["included",{"2":{"24":1,"83":1}}],["include",{"2":{"7":1,"8":1,"24":1}}],["inconvenient",{"2":{"119":1}}],["inconsistent",{"2":{"95":1}}],["incorrectly",{"2":{"155":1}}],["incorrect",{"0":{"109":1},"2":{"28":2,"59":1,"108":1,"111":1}}],["increase",{"2":{"63":1,"105":1}}],["initilly",{"2":{"64":1}}],["initializing",{"0":{"168":1},"1":{"169":1},"2":{"59":1,"113":1}}],["initialize",{"0":{"215":1,"226":1},"2":{"42":5,"59":3,"77":1,"78":1,"113":3,"114":1,"116":1,"153":1,"179":1,"232":1,"249":1}}],["initialized",{"2":{"24":4,"42":2,"69":8}}],["initializers",{"2":{"24":1}}],["initializer",{"2":{"24":1,"59":3,"66":10,"67":7}}],["initializations",{"2":{"104":2}}],["initialization",{"0":{"42":1},"2":{"21":1,"24":2,"39":1,"42":1,"63":4,"104":1,"134":1,"168":1,"184":1,"225":1,"256":1}}],["initial",{"2":{"9":1,"10":1,"59":2,"62":1,"66":11,"83":1,"95":1,"134":1,"184":1,"255":1,"256":1,"258":2}}],["initialstates",{"2":{"7":3,"8":1,"10":1,"134":2,"140":2,"184":1}}],["initialparameters",{"2":{"7":3,"8":1,"9":1,"134":2,"140":2,"184":1,"225":3}}],["init",{"2":{"24":7,"54":6,"59":6,"63":10,"66":26,"67":21,"69":16,"104":2,"105":1,"114":1,"134":4,"140":10,"169":8,"184":2,"185":2,"256":6}}],["ingredient",{"2":{"19":1,"69":1}}],["investigate",{"2":{"219":1}}],["investigated",{"2":{"117":1}}],["inversability",{"2":{"63":1}}],["inverse",{"2":{"17":1}}],["inv",{"2":{"63":2,"66":6,"67":3,"254":1}}],["involving",{"2":{"58":1}}],["invokes",{"2":{"32":1}}],["invoked",{"2":{"2":1,"184":1}}],["invariant",{"2":{"53":1}}],["invp",{"2":{"17":4}}],["informed",{"2":{"242":3}}],["informs",{"2":{"141":1}}],["information",{"2":{"8":1,"11":1,"17":1,"29":1,"33":1,"49":1,"57":1,"82":1,"83":1,"116":1,"157":1,"203":1,"204":1}}],["info",{"2":{"109":8,"110":20,"130":1,"132":1,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"247":2,"251":2,"260":2}}],["inf",{"2":{"53":1,"145":2,"146":2,"147":2,"148":2,"152":4}}],["inference",{"0":{"136":1},"2":{"33":1,"64":3,"69":6,"74":1,"136":1,"145":2,"153":2,"249":2}}],["inferred",{"2":{"19":1}}],["infinity",{"2":{"17":1}}],["int=50000",{"2":{"244":1}}],["int=32",{"2":{"241":1,"244":1}}],["int=0",{"2":{"44":5,"244":1}}],["intro",{"2":{"170":1}}],["introduction",{"2":{"96":1}}],["introductory",{"2":{"84":1}}],["introducing",{"2":{"37":1}}],["introduces",{"2":{"170":1}}],["introduced",{"2":{"24":1}}],["introduce",{"2":{"11":1,"58":1,"110":2}}],["int64",{"2":{"53":3,"77":16,"106":6,"122":4,"171":3,"173":2,"207":2,"220":99,"236":4}}],["int",{"2":{"19":1,"63":2,"66":2,"68":1,"69":1,"70":1,"140":4,"179":1,"214":2,"224":3,"248":1}}],["into",{"0":{"113":1},"2":{"8":1,"16":1,"24":2,"33":1,"39":1,"44":1,"53":1,"54":1,"56":4,"59":2,"62":2,"63":2,"65":3,"66":4,"68":1,"70":1,"77":1,"95":1,"106":1,"138":1,"140":1,"153":2,"171":2,"184":1,"192":1,"213":1}}],["intentionally",{"2":{"133":1}}],["interpolatingadjoint",{"2":{"220":13}}],["intermediate",{"0":{"200":1},"2":{"244":4}}],["interactiveutils",{"2":{"180":2,"189":2,"197":2,"210":2,"212":1,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["interactive",{"2":{"161":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["interested",{"2":{"146":1,"250":1}}],["interoperability",{"0":{"35":1},"1":{"36":1,"37":1,"38":1,"39":1}}],["inter",{"2":{"28":1}}],["interval",{"2":{"24":2}}],["international",{"2":{"19":1,"24":6,"53":3}}],["internals",{"2":{"225":1,"249":1}}],["internally",{"2":{"37":1,"58":1,"95":1,"144":1,"153":1,"220":1}}],["internal",{"2":{"19":2,"20":3,"31":1,"39":1,"52":1,"58":2,"62":1,"68":2,"81":1,"83":1,"102":1,"113":1}}],["interfacem",{"2":{"39":1}}],["interface",{"0":{"138":1,"139":1,"142":1,"143":1},"1":{"139":1,"140":2,"141":2,"142":1,"143":1},"2":{"7":1,"39":2,"46":2,"138":2,"140":1,"204":1,"214":1,"249":1}}],["integral",{"2":{"53":1}}],["integrating",{"0":{"113":1}}],["integration",{"0":{"45":1,"46":1},"2":{"114":1}}],["integrated",{"2":{"8":1,"157":1}}],["integers",{"2":{"42":1,"63":11,"65":15,"67":5}}],["integer",{"2":{"2":3,"4":3,"24":11,"57":3,"63":12,"65":9,"67":1,"68":1,"69":5}}],["intelligence",{"2":{"24":2}}],["intel",{"2":{"3":1}}],["inside",{"0":{"153":1},"2":{"32":1,"57":1,"59":1,"127":1,"130":1,"134":3,"144":3,"153":3}}],["instructions",{"2":{"76":1}}],["institute",{"2":{"74":1}}],["installation",{"0":{"76":1},"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["install",{"0":{"71":1},"2":{"71":1,"72":1,"73":1,"76":1,"77":1,"79":1}}],["installed",{"2":{"42":4,"46":1,"113":1,"172":1}}],["instability",{"2":{"58":1,"157":1,"166":1}}],["instabilities",{"0":{"157":1},"2":{"37":1,"106":1,"157":2}}],["instance",{"2":{"19":3,"66":1,"69":3,"249":1}}],["instancenorm",{"2":{"19":2,"69":7,"104":1,"105":1}}],["instead",{"2":{"3":2,"4":2,"8":1,"31":2,"37":1,"41":2,"48":1,"52":1,"57":1,"59":1,"63":3,"98":1,"99":1,"100":1,"102":1,"103":4,"104":2,"113":1,"119":1,"120":1,"128":1,"134":2,"138":1,"141":1,"142":1,"147":1,"153":2,"155":1,"158":1,"160":1,"167":1,"171":1,"184":1,"211":1,"244":4,"247":1}}],["inspiration",{"2":{"5":1}}],["in",{"0":{"61":1,"104":1,"119":1},"1":{"62":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"70":1},"2":{"2":2,"3":1,"4":2,"5":2,"7":4,"8":5,"10":3,"13":3,"15":2,"17":3,"19":2,"24":15,"27":2,"30":1,"31":2,"32":3,"33":3,"34":6,"37":5,"39":1,"42":1,"44":2,"45":1,"47":1,"48":1,"52":7,"53":3,"54":4,"55":7,"57":1,"58":4,"59":18,"62":20,"63":22,"64":2,"65":3,"66":23,"67":13,"68":6,"69":10,"71":3,"74":2,"76":1,"78":7,"79":1,"81":2,"82":1,"83":3,"84":2,"85":2,"87":1,"89":1,"90":1,"92":1,"95":1,"104":1,"105":1,"107":4,"108":1,"109":1,"110":11,"111":3,"113":4,"116":2,"117":1,"120":1,"121":1,"125":2,"126":1,"127":1,"130":2,"131":1,"133":2,"134":1,"137":2,"140":10,"143":1,"144":3,"145":3,"146":8,"147":1,"148":1,"149":2,"152":1,"153":5,"155":2,"156":2,"157":1,"160":5,"161":2,"167":1,"168":1,"169":1,"170":2,"171":2,"173":1,"174":3,"175":1,"179":2,"181":1,"183":3,"184":6,"185":3,"187":3,"190":1,"194":1,"195":2,"202":1,"204":2,"207":1,"208":4,"216":1,"217":2,"219":1,"225":1,"227":1,"228":4,"230":1,"236":1,"238":1,"243":1,"244":1,"245":2,"247":1,"248":6,"249":2,"250":13,"255":1,"256":3}}],["idx",{"2":{"227":2,"228":12}}],["ideal",{"2":{"196":1}}],["ideally",{"2":{"20":1,"203":1}}],["identity",{"2":{"16":1,"19":5,"24":13,"62":1,"77":3,"106":1,"122":2,"220":9,"236":1}}],["id",{"2":{"2":7,"4":3}}],["if",{"2":{"1":2,"2":12,"3":9,"4":3,"6":1,"7":1,"8":5,"11":2,"13":4,"15":2,"16":2,"17":7,"18":2,"19":6,"28":2,"31":3,"32":1,"33":5,"34":2,"37":2,"39":1,"42":2,"48":1,"49":2,"50":3,"52":2,"53":8,"54":4,"55":3,"57":1,"58":3,"59":6,"60":1,"62":17,"63":8,"64":10,"66":28,"67":12,"68":5,"69":14,"70":3,"71":1,"74":2,"76":1,"77":1,"78":2,"83":1,"87":2,"88":1,"95":2,"102":1,"106":3,"107":1,"109":1,"110":1,"111":1,"116":1,"117":1,"126":1,"127":1,"130":1,"131":4,"132":1,"136":1,"137":1,"138":1,"140":3,"141":3,"142":3,"144":3,"145":2,"148":1,"153":1,"159":1,"163":3,"167":1,"168":2,"169":2,"172":2,"174":1,"175":1,"179":1,"180":3,"189":3,"197":3,"202":1,"203":2,"208":2,"210":3,"221":3,"229":3,"236":1,"237":3,"244":1,"246":3,"251":3,"254":3,"260":3}}],["isdefined",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"229":3,"237":3,"246":3,"251":3,"260":3}}],["isnan",{"2":{"244":1}}],["isn",{"2":{"95":1,"137":1,"254":1}}],["issuing",{"2":{"57":1}}],["issue",{"2":{"57":1,"87":1,"106":1,"111":1,"117":1,"144":1,"145":2,"148":1,"159":1,"202":1,"203":2}}],["issues",{"2":{"52":1,"59":1,"66":1,"83":1,"84":1,"88":2,"89":1,"144":1,"153":1}}],["isbitstype",{"2":{"55":1}}],["istraining",{"2":{"54":4}}],["iszero",{"2":{"32":1}}],["isa",{"2":{"24":1,"32":1,"59":1}}],["isleaf",{"2":{"3":5,"10":1}}],["is",{"2":{"0":1,"1":4,"2":19,"3":18,"4":10,"5":3,"6":1,"7":6,"8":12,"10":2,"11":4,"13":2,"14":1,"15":3,"16":4,"17":6,"18":2,"19":7,"20":4,"21":1,"24":9,"27":4,"31":5,"32":4,"33":6,"37":3,"39":7,"42":2,"46":2,"47":2,"50":2,"52":4,"53":36,"54":5,"55":9,"57":5,"58":3,"59":9,"60":2,"62":23,"63":3,"64":7,"65":3,"66":38,"67":9,"68":10,"69":18,"70":3,"71":1,"76":1,"83":6,"85":1,"87":2,"89":2,"90":1,"93":1,"95":1,"96":1,"98":1,"102":3,"104":1,"105":1,"106":4,"108":2,"109":4,"110":5,"111":1,"112":1,"113":4,"114":1,"115":1,"116":2,"117":2,"119":1,"127":2,"130":1,"131":9,"132":1,"134":6,"136":3,"138":4,"140":3,"141":6,"142":2,"144":3,"145":12,"146":2,"149":1,"150":1,"152":1,"153":3,"155":2,"156":2,"157":1,"158":2,"160":3,"161":1,"163":2,"164":1,"169":3,"170":1,"171":4,"172":3,"173":4,"175":3,"179":1,"183":1,"184":1,"185":1,"186":2,"190":1,"196":2,"204":2,"208":4,"214":1,"217":2,"220":1,"225":1,"235":1,"236":1,"238":1,"244":4,"247":3,"248":1,"249":8,"250":2,"252":1,"254":1,"256":2}}],["fθ",{"2":{"250":1}}],["fw",{"2":{"179":1}}],["fwiw",{"2":{"141":1}}],["fd",{"2":{"145":8,"146":8,"147":8,"148":8}}],["f2",{"2":{"140":2}}],["f=σ",{"2":{"66":1}}],["f=identity",{"2":{"19":1}}],["ft",{"2":{"58":3}}],["f64",{"2":{"56":1,"256":1}}],["f32",{"2":{"56":1}}],["f16",{"2":{"56":1}}],["f1",{"2":{"53":1,"140":2}}],["fn=neuralode",{"2":{"215":1}}],["fn",{"2":{"52":2,"53":2,"59":8,"215":1}}],["fetch",{"2":{"142":1,"232":1}}],["feel",{"2":{"140":1}}],["feedforward",{"2":{"24":2,"249":1}}],["few",{"2":{"81":1,"171":1,"175":1}}],["fed",{"2":{"66":4}}],["features",{"0":{"30":1,"93":1,"96":1,"100":1,"105":1},"1":{"31":1,"32":1,"33":1,"34":1},"2":{"30":3,"47":1,"82":1,"90":1,"102":1,"117":2,"144":1,"192":1,"213":1}}],["feature",{"2":{"2":1,"24":2,"58":1,"63":1,"65":6,"69":3,"89":1,"106":1,"111":1,"144":2,"153":2,"171":1}}],["func",{"2":{"208":3}}],["functor",{"2":{"55":2,"56":3,"134":2}}],["functors",{"2":{"3":1,"5":1,"7":2,"10":3,"32":2,"55":2,"56":3,"95":1,"100":1,"103":1,"106":1,"126":1,"134":1,"247":1}}],["function3",{"2":{"148":4}}],["function2",{"2":{"147":4}}],["function1",{"2":{"145":5}}],["functions",{"0":{"20":1,"24":1,"25":1,"43":1,"53":1,"194":1,"216":1,"227":1,"242":1,"254":1},"2":{"7":1,"13":1,"22":1,"25":1,"37":1,"39":1,"52":1,"53":2,"54":1,"56":1,"62":1,"66":4,"81":1,"82":2,"83":1,"85":1,"95":1,"102":2,"106":1,"107":1,"116":1,"130":2,"131":1,"134":1,"140":3,"144":1,"148":1,"166":1,"167":1,"171":1,"173":1,"177":2,"225":1,"235":1,"254":1}}],["function",{"0":{"119":1,"145":1,"147":1,"148":1,"235":1},"1":{"146":1},"2":{"3":1,"4":4,"8":4,"10":2,"11":3,"13":4,"14":1,"15":2,"16":1,"17":2,"18":1,"19":7,"24":2,"27":4,"28":3,"31":1,"32":4,"39":1,"41":2,"42":1,"48":1,"49":1,"52":15,"53":2,"55":1,"57":3,"58":1,"59":11,"62":2,"63":4,"66":3,"67":4,"68":5,"70":1,"78":3,"83":1,"95":1,"102":2,"104":3,"106":9,"107":1,"110":2,"113":1,"121":1,"123":1,"126":1,"127":3,"128":2,"131":3,"134":2,"137":1,"140":3,"141":1,"142":1,"144":5,"145":1,"146":6,"148":2,"150":1,"151":1,"152":1,"153":2,"169":1,"172":2,"174":1,"176":1,"178":1,"179":1,"183":1,"185":1,"186":2,"192":1,"194":2,"195":1,"205":1,"208":4,"213":1,"214":4,"215":1,"216":2,"217":2,"218":2,"224":2,"225":1,"226":1,"227":2,"232":2,"235":3,"236":2,"238":1,"239":1,"241":2,"242":14,"244":1,"248":1,"249":5,"250":3,"254":16,"256":2,"257":7}}],["functionalities",{"2":{"33":1,"40":1}}],["functionality",{"0":{"33":1,"102":1,"103":1,"115":1},"2":{"6":1,"52":1,"66":1,"79":1,"95":2,"102":1,"138":2,"172":1}}],["functional",{"2":{"2":5,"3":5,"42":1,"112":1,"132":2,"166":1,"172":1,"180":2,"189":2,"197":2,"203":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["full",{"0":{"152":1},"2":{"146":1,"152":16}}],["fully",{"0":{"18":1},"2":{"24":1,"53":1,"55":1,"67":2,"88":1,"112":1}}],["future",{"2":{"144":1}}],["further",{"2":{"52":1}}],["fusion",{"2":{"16":1,"18":1}}],["fuse",{"2":{"16":1}}],["fuses",{"2":{"16":1}}],["fused",{"2":{"16":2,"18":2,"53":1,"158":3,"166":1}}],["fp32",{"2":{"13":1}}],["f",{"2":{"13":4,"16":2,"18":2,"27":5,"28":7,"32":3,"48":2,"49":6,"52":2,"55":4,"59":1,"62":4,"68":6,"134":1,"169":1,"176":3,"177":1,"178":1,"179":1,"208":1}}],["flexibility",{"2":{"142":1}}],["flexible",{"2":{"7":1}}],["flow",{"2":{"106":2}}],["float16",{"2":{"25":8,"56":1,"57":6}}],["floating",{"0":{"56":1},"2":{"13":1,"56":3}}],["float32",{"2":{"5":3,"24":6,"25":8,"37":1,"39":1,"53":6,"56":2,"57":5,"62":2,"68":2,"77":57,"78":1,"106":28,"107":1,"109":9,"110":27,"122":6,"123":3,"124":3,"126":10,"129":6,"131":3,"132":1,"133":2,"140":2,"141":2,"142":3,"145":10,"146":2,"147":11,"148":11,"152":2,"155":5,"168":5,"171":8,"172":1,"176":5,"177":4,"178":1,"179":8,"183":1,"192":1,"195":1,"207":4,"213":1,"220":77,"232":3,"235":4,"243":5,"244":1,"248":8,"256":7}}],["float64",{"2":{"5":4,"25":8,"53":1,"56":1,"57":4,"59":4,"153":7,"155":3,"171":5,"174":1,"207":4,"249":13,"255":1,"256":1,"257":1,"258":2}}],["flipkernel=true",{"2":{"105":1}}],["flat",{"2":{"217":1}}],["flattening",{"2":{"102":1}}],["flatten",{"2":{"68":1}}],["flattens",{"2":{"68":1}}],["flattenlayer",{"2":{"39":1,"68":3,"193":3,"215":1,"220":9,"226":1}}],["flattened",{"2":{"37":1,"68":3,"247":1}}],["flaky",{"2":{"115":1}}],["flag",{"2":{"39":1}}],["fluxlinear",{"2":{"134":5}}],["fluxlayer",{"2":{"37":2}}],["fluxmpifluxmodel",{"2":{"115":1}}],["fluxmpi",{"0":{"114":1},"1":{"115":1,"116":1},"2":{"112":1,"114":3,"115":1,"116":2}}],["flux",{"0":{"37":1,"83":1,"133":1,"137":1},"1":{"134":1,"135":1,"136":1,"137":1},"2":{"37":14,"66":3,"81":1,"83":4,"115":1,"133":3,"134":5,"136":4,"137":1,"140":2,"170":1,"247":1,"252":1}}],["fmaps",{"2":{"8":1}}],["fmap",{"2":{"7":3,"8":2,"10":1,"32":1,"55":7,"249":1}}],["framerate=10",{"2":{"245":1}}],["framework",{"2":{"81":1,"83":1,"138":1,"170":1,"173":1,"190":1}}],["frameworks",{"0":{"36":1},"1":{"37":1},"2":{"63":1,"66":1,"81":3,"138":1,"171":1,"256":1}}],["frequently",{"2":{"202":1}}],["frees",{"2":{"160":1}}],["freeze",{"2":{"31":5,"125":1,"126":6,"127":6,"128":5,"129":1}}],["freezing",{"0":{"31":1,"125":1,"126":1,"127":1,"128":1,"129":1},"1":{"126":1,"127":1,"128":1,"129":1},"2":{"125":1,"127":1,"128":2}}],["free",{"2":{"5":1,"24":1,"52":1,"254":1}}],["freeable",{"2":{"5":1}}],["friendly",{"2":{"81":2}}],["frozen",{"2":{"31":8,"126":9,"129":4}}],["frozenlayer",{"2":{"31":8}}],["fromfluxadaptor",{"2":{"37":5,"137":1}}],["from",{"0":{"36":1,"114":1,"133":1},"1":{"37":1,"115":1,"116":1,"134":1,"135":1,"136":1,"137":1},"2":{"4":1,"5":3,"7":2,"11":1,"13":1,"17":1,"20":1,"24":6,"25":12,"32":1,"37":1,"39":1,"52":5,"53":2,"54":1,"59":1,"62":1,"63":2,"64":1,"66":10,"67":1,"69":2,"77":1,"78":1,"83":1,"102":2,"103":1,"104":2,"130":1,"140":1,"141":1,"142":1,"145":2,"146":2,"147":1,"148":1,"155":1,"158":1,"160":3,"171":1,"179":2,"184":1,"190":1,"204":1,"208":1,"211":1,"220":3,"230":1,"232":1,"234":1,"238":1,"247":1,"249":3,"250":4,"252":2,"254":3,"256":1}}],["familiar",{"2":{"181":1,"225":1}}],["far",{"2":{"140":1}}],["farley",{"2":{"68":1}}],["fake",{"2":{"110":1}}],["favor",{"2":{"63":1}}],["facusapienza",{"2":{"145":1}}],["fact",{"2":{"116":1}}],["factor",{"2":{"17":2,"19":8,"24":2,"53":2,"70":2}}],["facilitates",{"2":{"58":1}}],["fausto",{"2":{"53":1}}],["fashionmnist",{"0":{"222":1},"1":{"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"224":1,"228":2}}],["fashion",{"2":{"42":1}}],["fastest",{"2":{"87":2,"150":1}}],["faster",{"0":{"158":1},"2":{"3":1,"13":2,"14":3,"87":1,"144":1,"158":2,"159":1}}],["fast",{"2":{"13":4,"15":2,"19":1,"20":1,"30":1,"39":1,"69":1,"102":1,"149":1,"158":2,"160":1,"177":1,"256":3}}],["failed",{"2":{"109":1}}],["failing",{"2":{"49":1}}],["failures",{"2":{"37":1}}],["fail",{"2":{"37":1,"49":4,"88":1}}],["fails",{"2":{"33":1,"48":1,"87":2}}],["fancy",{"2":{"106":1}}],["fan",{"2":{"24":6,"63":2}}],["fallback",{"2":{"7":1,"11":3,"16":2,"18":2,"20":1,"95":1,"137":1}}],["fall",{"2":{"3":1}}],["falls",{"2":{"2":1,"13":1}}],["false",{"2":{"2":2,"10":1,"17":1,"20":1,"33":1,"37":1,"39":2,"50":1,"53":5,"54":1,"55":1,"58":1,"62":2,"63":2,"66":10,"67":3,"104":1,"144":1,"156":1,"162":1,"163":1,"212":1,"214":3,"218":2,"220":13,"223":1,"239":1,"257":1}}],["fitting",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1}}],["fit",{"2":{"208":1,"230":1,"244":4,"250":1}}],["figure",{"2":{"206":1,"209":1,"232":1,"236":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["figured",{"2":{"110":1}}],["fig",{"2":{"206":3,"209":3,"232":3,"236":3,"245":6,"248":3,"250":6,"255":3,"256":3,"259":6}}],["fixing",{"2":{"98":1,"148":1,"168":1}}],["fix",{"2":{"88":1,"109":1,"110":2,"145":2,"149":1,"155":2,"202":1}}],["fix1",{"2":{"64":1,"68":1,"144":3,"147":1,"148":2,"215":1,"220":9,"256":2}}],["fixed=",{"2":{"59":1}}],["fixed",{"2":{"17":1,"58":1,"59":4,"103":2,"109":4,"173":1}}],["finetune",{"2":{"208":1}}],["fine",{"2":{"141":1}}],["finite",{"2":{"145":4,"146":2,"149":1}}],["finitediff",{"2":{"49":1,"144":1,"145":2,"146":2}}],["finish",{"2":{"108":1}}],["findmax",{"2":{"250":1}}],["find",{"2":{"59":1,"111":1,"153":1,"179":1,"202":1,"250":1}}],["final",{"2":{"59":2,"228":3}}],["finally",{"0":{"196":1},"2":{"42":1,"59":1,"113":1,"140":1,"141":1,"171":1,"179":1,"184":1,"220":1,"236":1,"259":1}}],["fill",{"2":{"53":1,"171":1}}],["filter",{"2":{"24":1}}],["file",{"2":{"1":1,"131":3}}],["fields",{"2":{"52":1,"55":1,"62":10,"68":2,"142":1}}],["fieldnames",{"2":{"7":1,"184":1}}],["field",{"2":{"7":3,"8":1,"37":2,"54":1,"62":1,"184":1}}],["first",{"2":{"2":1,"8":1,"32":1,"34":1,"37":1,"39":1,"59":6,"62":2,"65":3,"68":2,"77":1,"89":1,"106":2,"110":4,"114":1,"131":1,"133":1,"134":1,"140":2,"142":2,"144":1,"145":1,"153":2,"155":2,"161":1,"173":1,"174":2,"176":1,"179":1,"184":3,"194":1,"196":1,"214":1,"216":1,"227":1,"236":1,"240":1,"247":1,"248":2,"250":3,"255":1,"256":2,"257":1,"259":1}}],["footnotes",{"0":{"89":1}}],["foldl",{"2":{"54":3}}],["follow",{"2":{"107":1}}],["follows",{"2":{"3":1,"5":1,"62":1}}],["following",{"2":{"2":1,"5":1,"20":1,"28":1,"32":1,"55":1,"56":1,"57":2,"71":2,"72":2,"73":2,"107":1,"126":1,"130":1,"133":1,"138":1,"144":1,"149":1,"154":1,"155":1,"159":1,"161":2,"163":1,"184":1,"242":1,"250":1}}],["focuses",{"2":{"53":1}}],["focalloss",{"2":{"53":2}}],["focal",{"2":{"53":4}}],["four",{"2":{"171":1}}],["fourth",{"2":{"53":1}}],["found",{"2":{"2":1,"74":1,"203":1}}],["forum",{"2":{"144":1}}],["forget",{"2":{"108":1}}],["formulas",{"2":{"254":1}}],["format",{"2":{"171":1}}],["formats",{"2":{"62":4,"68":1}}],["form",{"2":{"59":1,"67":1,"153":1,"167":1,"173":2}}],["forms",{"2":{"53":1}}],["forwarded",{"2":{"48":1}}],["forward",{"2":{"33":1,"54":1,"59":5,"66":1,"67":2,"78":1,"86":2,"110":1,"111":1,"145":1,"147":1,"148":1,"175":1,"176":2,"177":2,"249":1,"250":5}}],["forwarddiff",{"2":{"20":1,"49":1,"55":1,"86":1,"144":3,"145":2,"147":4,"148":2,"152":1,"175":2,"176":4,"177":1}}],["forbidden",{"2":{"24":1}}],["forces",{"2":{"83":1}}],["force",{"2":{"2":6,"37":4,"48":1}}],["for",{"0":{"38":1,"124":1,"146":1,"159":1,"161":1,"170":1,"257":1},"1":{"39":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"0":1,"1":1,"2":3,"3":14,"4":9,"5":3,"6":1,"7":11,"8":7,"11":1,"12":1,"13":1,"14":1,"16":5,"17":8,"18":5,"19":15,"20":3,"21":1,"24":11,"27":2,"28":3,"29":1,"31":5,"32":3,"33":8,"34":2,"37":3,"39":5,"41":2,"42":8,"43":2,"47":2,"48":1,"49":5,"52":5,"53":9,"54":5,"55":15,"56":1,"57":4,"58":2,"59":16,"60":3,"62":11,"63":10,"65":15,"66":24,"67":12,"68":6,"69":18,"70":8,"76":1,"77":2,"78":7,"79":1,"80":5,"81":1,"82":1,"83":9,"84":1,"85":2,"87":11,"88":2,"89":1,"92":2,"93":2,"95":2,"96":1,"97":1,"98":1,"99":1,"100":1,"102":1,"104":2,"105":1,"106":5,"107":4,"108":1,"109":1,"110":1,"112":1,"113":1,"116":1,"117":2,"119":1,"121":2,"125":1,"127":2,"130":1,"133":1,"136":1,"138":2,"140":6,"141":3,"142":3,"143":1,"144":6,"146":1,"147":2,"148":1,"149":1,"150":1,"152":2,"153":4,"154":2,"155":3,"157":1,"159":1,"160":3,"162":2,"164":1,"165":1,"166":5,"167":1,"168":2,"169":1,"170":1,"171":3,"172":1,"173":2,"174":2,"175":5,"176":2,"177":2,"179":3,"183":3,"184":4,"185":2,"186":1,"187":3,"190":1,"194":1,"195":2,"196":1,"203":1,"204":2,"208":3,"213":1,"214":2,"216":1,"217":5,"220":3,"227":1,"228":3,"233":1,"236":2,"238":1,"243":2,"244":1,"245":2,"248":5,"249":2,"250":5,"255":1}}],["tmp",{"2":{"254":4}}],["tmatch",{"2":{"8":3}}],["t∈",{"2":{"243":1}}],["ttest",{"2":{"217":1,"228":2}}],["ttraining",{"2":{"217":1,"228":2}}],["ttime",{"2":{"195":2,"217":3,"228":3}}],["td",{"0":{"124":1}}],["tdchain",{"0":{"122":1},"2":{"121":4,"123":1,"124":1}}],["tl",{"2":{"108":1}}],["tloss",{"2":{"107":1}}],["tpu",{"2":{"73":1,"106":1}}],["tsteps",{"2":{"255":5,"256":4,"259":6}}],["tstate",{"2":{"179":6,"217":7,"236":9}}],["tsit5",{"2":{"206":1,"208":1,"209":1,"220":13}}],["tspan=",{"2":{"214":2,"218":1}}],["tspan",{"2":{"206":5,"209":2,"214":5,"218":3,"220":7,"255":4,"256":1,"259":1}}],["tsung",{"2":{"53":2}}],["ts",{"2":{"52":16,"245":6,"248":1,"249":4}}],["t=rand",{"2":{"49":1}}],["t=float32",{"2":{"24":8}}],["tail",{"2":{"249":1}}],["tab>`",{"2":{"176":1}}],["table",{"2":{"67":1,"171":1}}],["tall",{"2":{"175":1}}],["tasklocalrng",{"2":{"77":1,"129":1,"170":1,"179":1}}],["tasks",{"2":{"53":3}}],["taking",{"2":{"53":1,"54":1,"68":1}}],["takeaway",{"2":{"142":1}}],["takes",{"2":{"8":1,"53":2,"59":1,"62":4,"78":1,"106":1,"208":1,"235":1,"250":1,"256":2}}],["taken",{"2":{"5":1,"24":1,"60":1,"63":2,"67":1,"104":1,"106":1,"148":1}}],["take",{"2":{"1":1,"25":1,"32":1,"52":1,"53":2,"60":1,"63":1,"77":1,"88":1,"120":1,"145":1,"169":1,"241":1,"243":1,"249":1}}],["targets",{"2":{"53":1,"192":1,"213":1}}],["target",{"2":{"48":10,"53":2,"194":3,"216":3,"227":3,"242":6,"243":12,"244":10}}],["tanh",{"2":{"34":2,"66":1,"77":11,"145":1,"146":1,"147":3,"148":3,"152":3,"208":2,"215":4,"220":36,"241":3,"249":3}}],["turing",{"2":{"247":6}}],["tutorials",{"0":{"198":1,"199":1,"200":1,"201":1,"203":1},"1":{"199":1,"200":1,"201":1,"202":1,"203":1},"2":{"59":1,"84":1,"203":1,"238":1,"244":4}}],["tutorial",{"2":{"58":1,"78":1,"170":3,"175":1,"181":2,"190":2,"203":1,"204":6,"230":1,"238":1,"247":2}}],["tu",{"2":{"27":1}}],["tuple=true",{"2":{"217":1}}],["tuples",{"2":{"7":2}}],["tuple",{"2":{"3":2,"7":2,"19":2,"20":1,"24":3,"31":4,"32":1,"39":1,"52":1,"53":1,"55":1,"62":7,"63":11,"65":15,"66":22,"67":5,"69":1,"70":3,"77":12,"82":1,"121":1,"147":1,"148":1,"153":1,"207":2,"215":2,"220":36,"249":1}}],["two",{"2":{"24":3,"55":2,"62":4,"67":1,"69":1,"70":3,"131":1,"140":1,"141":1,"142":1,"149":1,"169":1,"249":1,"254":4,"256":1}}],["temporal",{"2":{"243":1}}],["te",{"2":{"195":4,"196":1,"217":2}}],["technology",{"2":{"74":1}}],["terminate",{"2":{"208":1}}],["terminology",{"2":{"208":1}}],["terms",{"2":{"89":1}}],["term",{"2":{"66":1,"145":1,"152":1,"249":2}}],["terrible",{"2":{"52":1}}],["tested",{"2":{"82":1,"83":1,"88":2,"175":1,"202":1}}],["test",{"0":{"50":1},"2":{"48":7,"49":10,"50":8,"64":3,"69":2,"83":1,"89":1,"106":1,"145":2,"153":1,"192":5,"195":3,"196":20,"213":5,"217":3,"224":5,"228":13,"255":1}}],["tests",{"2":{"48":2,"49":2,"88":1,"117":2,"149":1}}],["testing",{"0":{"48":1},"2":{"47":3,"82":1}}],["testmode`",{"2":{"145":2}}],["testmode",{"2":{"10":1,"64":3,"69":3,"106":3,"136":1,"187":1,"194":1,"216":1,"227":1,"244":1}}],["tends",{"2":{"17":1,"145":1,"146":1}}],["tensors",{"2":{"70":1}}],["tensorflow",{"2":{"66":1}}],["tensor",{"2":{"16":3,"24":1,"69":1,"171":1,"254":1}}],["tile",{"2":{"171":2}}],["tiles",{"2":{"171":1}}],["tightly",{"2":{"167":1}}],["tier",{"2":{"86":8,"88":3}}],["tied",{"0":{"34":1},"2":{"83":1}}],["title",{"2":{"74":2,"245":1,"250":1}}],["tips",{"2":{"154":2}}],["tip",{"2":{"8":1,"18":1,"37":1,"39":1,"48":1,"59":1,"66":1,"112":1,"140":1,"144":2,"203":1,"254":1}}],["timewrapper",{"2":{"207":7,"208":1}}],["timelastindex",{"2":{"54":1,"66":2}}],["timestep",{"2":{"255":1}}],["timespace",{"2":{"255":1}}],["times",{"2":{"47":1,"62":3,"171":1,"174":1,"204":1}}],["time",{"0":{"121":1},"2":{"3":1,"37":1,"66":2,"87":2,"106":1,"120":3,"123":5,"195":3,"196":20,"207":1,"217":3,"219":1,"228":2,"245":1,"255":2,"256":3,"257":1,"259":2}}],["typing",{"2":{"76":1}}],["typical",{"2":{"59":1}}],["typically",{"2":{"7":1,"66":1,"83":1,"147":1,"160":1,"186":1,"243":1,"256":1}}],["typed",{"2":{"176":1}}],["typeof",{"2":{"58":1,"77":8,"106":3,"110":1,"122":2,"140":4,"220":80,"236":2,"256":2}}],["types",{"0":{"7":1,"22":1,"118":1},"1":{"119":1,"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"3":1,"8":2,"20":2,"31":1,"55":6,"97":2,"100":1,"142":1,"143":1,"166":1,"168":2}}],["type",{"0":{"57":1,"155":1,"157":1,"220":1},"2":{"3":13,"4":8,"7":6,"8":5,"16":1,"20":1,"22":2,"24":7,"25":1,"37":1,"39":1,"42":5,"53":1,"55":6,"57":12,"58":3,"59":4,"62":2,"69":1,"106":1,"109":5,"110":13,"113":3,"119":1,"126":1,"127":1,"141":1,"142":3,"143":1,"155":2,"157":2,"166":2,"169":1,"184":2,"187":2,"219":1,"220":1,"224":1}}],["tr",{"2":{"149":2,"152":11,"195":4,"217":2}}],["trying",{"2":{"140":1,"208":1}}],["try",{"2":{"87":1,"106":1,"109":2,"110":1,"122":1,"142":2,"152":1,"171":1,"179":1,"188":1,"203":1}}],["treat",{"2":{"49":1,"141":1}}],["treated",{"2":{"15":1,"59":1}}],["truth",{"2":{"147":1,"179":1}}],["truncation",{"2":{"102":2}}],["truncated",{"2":{"22":4,"24":2}}],["truly",{"2":{"83":1}}],["true",{"0":{"255":1},"2":{"2":2,"3":3,"10":1,"17":6,"19":4,"24":2,"32":1,"33":2,"34":1,"37":2,"39":1,"49":2,"53":40,"54":3,"55":1,"58":1,"59":4,"62":3,"63":2,"64":4,"66":11,"70":1,"77":8,"82":1,"104":1,"106":3,"109":1,"110":2,"122":2,"126":2,"129":2,"131":1,"132":1,"145":3,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"163":2,"167":1,"179":2,"186":4,"194":1,"206":2,"208":2,"216":1,"218":1,"220":58,"227":1,"232":1,"236":3,"242":3,"244":2,"247":2,"249":1,"255":1,"256":2}}],["trivial",{"2":{"83":1,"138":1,"173":1,"208":1,"220":26}}],["trilinear",{"2":{"70":6}}],["tries",{"2":{"16":1}}],["trigger",{"2":{"3":4,"4":2,"113":1,"130":1,"131":2}}],["traditional",{"2":{"67":1,"249":1}}],["transitioned",{"2":{"130":1}}],["transposed",{"2":{"63":2}}],["transpose",{"2":{"63":3,"171":1}}],["transformation",{"2":{"69":1}}],["transformations",{"2":{"37":1}}],["transforms",{"2":{"65":3}}],["transform",{"2":{"37":1}}],["transformed",{"2":{"15":1}}],["transferred",{"2":{"3":1,"134":1}}],["transferring",{"2":{"0":1,"5":1,"160":1}}],["transfer",{"0":{"2":1,"160":1},"2":{"2":1,"134":1,"160":2,"188":1}}],["trace",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":7,"150":3,"151":2,"152":10,"254":4}}],["tracing",{"2":{"55":1,"106":1}}],["tracking",{"0":{"110":1},"2":{"105":1,"111":2}}],["track",{"2":{"33":1,"57":1,"62":2,"69":12,"109":1,"110":5,"155":1}}],["tracker",{"2":{"16":1,"18":1,"20":1,"49":1,"52":1,"55":1,"86":1,"122":1,"244":12,"247":1}}],["tracked",{"2":{"109":1}}],["trackedarray",{"2":{"8":1,"122":2}}],["trackedreals",{"2":{"8":1}}],["train=1024",{"2":{"224":1}}],["trainstate",{"0":{"107":1},"2":{"52":11,"77":3,"78":1,"102":1,"106":2,"107":3,"179":1,"187":1,"195":1,"204":1,"217":1,"228":1,"236":4,"241":1,"244":1}}],["trained2",{"2":{"187":2}}],["trained",{"2":{"39":1,"187":2,"188":8,"208":2,"209":2,"244":5,"245":1,"259":3}}],["train",{"0":{"219":1},"2":{"39":1,"52":6,"59":1,"66":24,"77":8,"78":7,"106":1,"107":10,"179":4,"183":4,"187":12,"190":1,"192":6,"195":15,"196":3,"204":2,"208":3,"213":6,"217":8,"224":10,"228":24,"236":1,"241":1,"243":1,"244":9}}],["trainmode",{"2":{"10":1}}],["trainable",{"2":{"7":1,"31":1,"52":2,"59":9,"63":2,"66":4,"67":3,"69":1,"134":9}}],["training=val",{"2":{"64":3}}],["training",{"0":{"52":1,"112":1,"136":1,"181":1,"187":1,"195":1,"196":1,"204":1,"206":1,"208":1,"217":1,"222":1,"228":1,"236":1,"238":1,"244":1,"252":1,"257":1,"258":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"4":3,"10":2,"17":9,"19":8,"24":2,"33":1,"41":2,"42":4,"52":14,"53":2,"54":3,"64":6,"69":9,"74":1,"77":8,"78":3,"83":2,"102":1,"103":2,"106":1,"107":5,"109":1,"110":3,"112":2,"126":2,"128":1,"129":2,"136":2,"179":5,"181":1,"187":2,"190":1,"195":4,"196":20,"204":1,"217":5,"219":1,"228":2,"235":1,"236":5,"241":1,"244":2,"257":2,"258":1}}],["t",{"2":{"2":1,"3":5,"4":6,"8":1,"13":2,"16":1,"18":1,"24":20,"25":1,"32":1,"34":1,"37":1,"47":1,"49":1,"52":1,"54":1,"59":6,"66":1,"68":1,"77":1,"78":1,"83":1,"88":2,"89":1,"95":2,"98":1,"102":1,"108":1,"111":1,"116":1,"117":1,"119":1,"120":1,"121":4,"123":2,"134":1,"136":1,"137":1,"140":4,"142":3,"143":1,"144":2,"145":1,"146":1,"147":10,"149":1,"153":1,"171":2,"173":2,"177":1,"179":1,"183":1,"184":4,"185":2,"188":1,"192":1,"195":3,"202":1,"204":1,"206":6,"207":11,"208":8,"209":9,"213":1,"214":5,"218":3,"225":1,"226":1,"228":2,"236":1,"243":6,"244":12,"254":16,"255":1,"256":1,"257":2}}],["thousands",{"2":{"171":1}}],["though",{"2":{"140":1,"159":1,"175":1}}],["those",{"2":{"7":1,"8":1,"49":1,"70":1,"79":1,"110":1,"117":1,"141":1,"171":1,"214":1,"250":1}}],["thunk",{"2":{"106":1}}],["thumb",{"2":{"70":1}}],["thus",{"2":{"19":1,"62":1,"70":1}}],["threads",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"229":3,"237":3,"246":3,"251":3,"260":3}}],["three",{"2":{"68":1,"171":1}}],["through",{"2":{"5":1,"8":1,"62":4,"69":4,"76":1,"84":2,"120":1,"134":1,"142":1,"154":1,"173":1,"184":3,"207":1,"250":1}}],["throws",{"2":{"50":1}}],["throw",{"2":{"2":1,"3":1,"33":1,"52":1,"57":1,"95":1,"244":1}}],["thrown",{"2":{"2":2,"31":1,"33":2}}],["than",{"2":{"24":2,"175":1,"217":1}}],["that",{"2":{"2":3,"3":8,"5":1,"7":4,"8":5,"10":1,"11":1,"13":2,"16":1,"17":1,"18":2,"20":1,"24":3,"30":1,"31":1,"33":1,"34":2,"39":2,"44":2,"47":1,"52":1,"53":1,"54":1,"55":3,"58":1,"59":5,"62":5,"63":2,"64":2,"65":6,"67":1,"68":4,"69":6,"77":1,"79":1,"83":2,"89":3,"90":2,"95":2,"98":2,"106":1,"107":2,"108":2,"109":3,"110":1,"111":2,"113":5,"114":1,"116":1,"133":1,"134":2,"137":1,"140":3,"141":4,"142":1,"143":2,"144":3,"145":2,"146":3,"147":2,"149":2,"152":3,"153":2,"155":3,"156":1,"161":1,"163":1,"167":1,"171":1,"173":3,"174":1,"176":1,"177":1,"184":3,"188":2,"202":1,"204":2,"208":4,"217":1,"220":1,"233":1,"235":1,"240":1,"241":1,"243":1,"249":1,"250":3,"256":3}}],["third",{"2":{"171":2}}],["thirteenth",{"2":{"24":2}}],["thing",{"2":{"68":2,"153":1,"155":1}}],["things",{"2":{"59":1,"140":1,"142":1,"171":1}}],["think",{"2":{"59":1,"140":1}}],["this",{"2":{"2":3,"3":8,"4":10,"5":1,"7":5,"8":6,"10":1,"11":6,"13":3,"14":1,"16":3,"18":3,"19":2,"20":4,"21":1,"24":5,"27":2,"28":2,"30":2,"32":2,"33":5,"37":6,"39":3,"41":2,"42":3,"45":1,"46":1,"47":2,"48":1,"49":2,"52":6,"53":4,"54":1,"55":12,"57":7,"58":6,"59":11,"62":3,"63":5,"65":3,"66":6,"67":5,"68":1,"69":5,"70":3,"74":2,"79":1,"82":1,"83":2,"85":1,"87":2,"89":1,"90":2,"95":6,"96":1,"97":1,"98":3,"103":1,"104":1,"105":1,"106":5,"107":3,"108":2,"109":3,"111":2,"113":3,"115":2,"117":2,"125":2,"127":1,"130":1,"131":2,"134":2,"136":3,"138":1,"140":8,"142":6,"143":1,"144":6,"145":9,"146":5,"147":1,"148":1,"149":3,"150":1,"152":2,"153":5,"155":7,"156":1,"157":1,"159":1,"160":4,"162":1,"163":2,"164":3,"166":1,"167":1,"170":1,"171":5,"173":4,"175":1,"179":1,"180":1,"181":2,"183":1,"185":2,"189":1,"190":1,"193":1,"196":1,"197":1,"204":3,"208":5,"210":1,"217":1,"218":1,"220":2,"221":1,"225":1,"229":1,"230":1,"233":1,"235":1,"237":1,"238":2,"246":1,"247":1,"249":4,"251":1,"252":1,"254":3,"256":2,"258":1,"260":1}}],["theoretical",{"2":{"254":1}}],["theoretically",{"2":{"175":1}}],["theorem",{"2":{"68":1}}],["thesis",{"2":{"74":1}}],["these",{"2":{"7":5,"8":1,"25":1,"30":1,"37":3,"39":3,"40":1,"42":1,"52":1,"69":1,"77":1,"81":2,"83":1,"88":7,"95":2,"102":3,"117":1,"120":1,"140":2,"141":2,"144":1,"148":1,"159":1,"163":1,"166":1,"167":3,"170":1,"173":1,"177":1,"202":3,"203":1,"211":1,"249":1}}],["they",{"2":{"52":1,"59":2,"110":1,"126":1,"134":1,"171":1,"175":2}}],["there",{"2":{"31":1,"39":1,"52":1,"77":1,"87":1,"89":1,"97":1,"110":1,"111":1,"113":1,"144":1,"147":2,"155":1,"159":1,"169":1,"171":1,"196":1}}],["them",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"30":1,"32":1,"53":1,"59":1,"62":1,"67":1,"70":1,"77":1,"81":1,"82":1,"88":1,"95":1,"102":1,"153":1,"155":1,"166":1,"171":2,"174":1,"177":1,"203":2,"241":1,"249":1,"256":2}}],["their",{"2":{"7":2,"63":1,"84":1,"158":1,"171":1,"173":2}}],["then",{"2":{"1":2,"2":6,"3":1,"8":1,"17":2,"19":1,"28":2,"31":3,"33":2,"48":1,"49":2,"50":1,"53":6,"54":1,"55":2,"58":2,"60":1,"62":5,"63":4,"64":4,"66":9,"67":7,"68":1,"74":1,"76":1,"83":1,"106":1,"131":3,"140":1,"141":3,"149":2,"161":1}}],["the",{"0":{"107":1,"122":1,"123":1,"124":1,"128":1,"148":2,"150":1,"151":1,"152":1,"170":1,"185":1,"187":1,"188":1,"193":1,"195":1,"196":1,"207":1,"208":1,"209":1,"214":1,"215":1,"219":1,"226":1,"241":1,"242":1,"243":1,"245":1,"249":1,"255":1,"257":1,"258":1,"259":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"1":6,"2":12,"3":31,"4":14,"5":13,"6":3,"7":19,"8":22,"9":4,"10":7,"11":6,"13":10,"14":2,"15":3,"16":11,"17":16,"18":6,"19":21,"20":5,"24":60,"25":25,"27":20,"28":13,"29":1,"30":1,"31":14,"32":22,"33":15,"34":14,"37":12,"39":9,"40":1,"42":10,"43":4,"44":11,"45":5,"46":3,"48":6,"49":9,"50":4,"52":35,"53":55,"54":4,"55":13,"56":6,"57":19,"58":18,"59":49,"60":6,"62":46,"63":38,"64":6,"65":45,"66":69,"67":19,"68":33,"69":58,"70":14,"71":7,"72":2,"73":2,"76":3,"77":6,"78":8,"79":1,"81":1,"82":5,"83":6,"84":4,"85":3,"87":7,"88":2,"89":1,"90":2,"92":2,"95":2,"96":2,"97":2,"98":4,"99":1,"102":11,"104":9,"105":4,"106":23,"107":7,"108":3,"109":6,"110":19,"112":1,"113":22,"114":3,"115":1,"116":6,"119":2,"120":2,"121":1,"123":3,"124":2,"126":1,"127":7,"128":3,"130":3,"131":5,"133":4,"134":6,"136":4,"137":1,"138":6,"140":20,"141":10,"142":4,"143":5,"144":9,"145":5,"146":16,"147":5,"148":13,"149":11,"150":2,"152":7,"153":10,"154":1,"155":9,"157":1,"158":3,"159":2,"160":8,"161":4,"162":1,"163":1,"164":1,"165":1,"166":10,"167":4,"169":11,"170":2,"171":12,"172":7,"173":2,"174":5,"175":6,"176":1,"177":4,"178":2,"179":6,"181":2,"183":6,"184":24,"185":3,"186":2,"187":6,"188":5,"190":2,"192":2,"193":3,"195":2,"196":5,"202":1,"203":6,"204":5,"207":8,"208":11,"213":2,"214":13,"215":1,"217":3,"218":2,"219":3,"220":5,"225":6,"228":1,"232":4,"235":3,"236":2,"238":2,"240":3,"241":2,"242":4,"243":3,"247":5,"248":3,"249":16,"250":22,"252":1,"254":7,"255":2,"256":10,"257":4,"258":2,"259":2}}],["top",{"2":{"106":1,"138":1}}],["topic",{"2":{"66":1}}],["toolchain",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["tools",{"2":{"33":2,"59":1,"108":1,"170":1,"175":1}}],["too",{"2":{"81":1,"175":1,"250":1}}],["towards",{"2":{"55":1,"173":1,"204":1}}],["toarray",{"2":{"39":1}}],["tosimplechainsadaptor",{"2":{"39":4,"83":1,"193":2}}],["together",{"2":{"32":1}}],["total",{"2":{"9":1,"10":1,"34":1,"43":2,"59":4,"62":4,"69":3,"77":1,"78":1,"109":1,"110":1,"121":1,"193":2,"194":6,"216":6,"227":6,"233":1,"244":3,"256":1}}],["to",{"0":{"37":1,"39":1,"50":1,"71":1,"84":1,"90":1,"113":1,"133":1,"154":1,"252":1},"1":{"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":1,"103":1,"104":1,"105":1,"134":1,"135":1,"136":1,"137":1,"155":1,"156":1,"157":1,"158":1,"159":1,"160":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"1":3,"2":7,"3":10,"4":5,"5":5,"6":2,"7":4,"8":10,"10":2,"11":2,"13":7,"14":2,"15":4,"16":5,"17":13,"18":3,"19":16,"20":2,"24":14,"27":8,"28":5,"30":1,"31":7,"32":2,"33":10,"34":5,"37":14,"39":17,"42":9,"44":6,"46":1,"47":2,"48":3,"49":10,"52":8,"53":14,"54":3,"55":9,"56":4,"57":4,"58":4,"59":21,"60":1,"62":18,"63":19,"64":13,"65":9,"66":59,"67":7,"68":9,"69":22,"70":8,"71":2,"72":1,"73":1,"74":1,"76":3,"77":2,"78":2,"81":2,"82":1,"83":9,"84":2,"88":3,"89":1,"90":5,"92":1,"95":5,"96":1,"97":1,"98":5,"99":2,"102":6,"103":3,"104":6,"105":3,"106":14,"107":8,"108":4,"109":1,"110":7,"112":1,"113":5,"114":3,"116":4,"119":2,"120":2,"121":1,"125":1,"126":1,"127":1,"128":1,"130":3,"131":2,"133":3,"134":8,"136":1,"137":3,"138":5,"140":7,"141":5,"142":4,"143":2,"144":7,"145":10,"146":6,"147":1,"148":4,"149":6,"152":1,"153":9,"155":7,"156":1,"157":2,"158":1,"160":3,"161":4,"162":2,"163":3,"164":4,"166":4,"167":3,"168":1,"169":3,"170":4,"171":7,"172":1,"173":3,"175":1,"176":1,"177":3,"179":2,"181":2,"183":3,"184":10,"186":1,"188":1,"190":2,"192":1,"193":3,"202":1,"203":6,"204":4,"207":6,"208":7,"211":2,"213":1,"214":6,"215":1,"217":1,"218":1,"220":11,"225":1,"226":1,"235":1,"238":2,"242":2,"243":2,"247":1,"248":3,"249":12,"250":5,"254":5,"256":6,"257":2,"258":2}}],["toml",{"2":{"1":1,"47":1}}],["l3",{"2":{"259":2}}],["lb",{"2":{"256":1,"259":1}}],["lbfgs",{"2":{"204":1,"208":3}}],["lr",{"2":{"244":2}}],["ld",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["llvm",{"2":{"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["ll",{"2":{"170":1,"248":1,"249":1,"256":2}}],["lpnorm",{"2":{"105":1}}],["lppool",{"2":{"65":1,"105":1}}],["lp",{"2":{"65":4,"249":1,"250":1}}],["l=",{"2":{"53":1}}],["ls",{"2":{"53":4}}],["lstmcell",{"2":{"66":6,"104":1,"105":1,"184":3,"185":1}}],["lstm",{"0":{"181":1},"1":{"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1},"2":{"7":1,"66":1,"184":17,"185":4}}],["l2",{"2":{"34":5,"141":4,"256":2,"259":2}}],["l2=dense",{"2":{"34":1}}],["l1",{"2":{"34":5,"141":4,"256":2,"259":2}}],["l1=dense",{"2":{"34":1}}],["lecture",{"2":{"211":1}}],["lecun",{"2":{"53":1}}],["leveraging",{"2":{"153":1}}],["levels=10",{"2":{"245":1}}],["level",{"2":{"24":2,"85":1,"107":1,"153":1,"166":2}}],["less",{"2":{"68":1,"81":1,"88":1,"175":1}}],["lets",{"2":{"142":1,"195":1,"217":1,"228":1,"247":1}}],["let",{"2":{"59":1,"106":2,"107":1,"108":1,"109":2,"110":7,"114":1,"126":1,"134":2,"140":1,"141":2,"142":2,"144":3,"145":2,"147":1,"148":1,"152":3,"153":1,"155":1,"170":2,"171":1,"174":1,"176":1,"177":1,"178":1,"179":4,"184":1,"186":1,"188":1,"208":2,"217":1,"219":1,"232":1,"233":1,"236":1,"249":1,"255":1,"256":1,"257":1,"259":2}}],["len",{"2":{"49":2,"183":1,"243":11}}],["length=25",{"2":{"250":4}}],["length=bc",{"2":{"243":3}}],["length=grid",{"2":{"243":1}}],["length=datasize",{"2":{"206":1,"255":1}}],["length=50",{"2":{"183":1}}],["length",{"2":{"24":7,"25":24,"34":2,"62":2,"63":2,"65":6,"140":8,"141":2,"183":5,"186":1,"194":1,"207":2,"216":1,"227":1,"245":8,"249":3,"254":2,"259":1}}],["leibler",{"2":{"53":1}}],["lei",{"2":{"19":1}}],["lempitsky",{"2":{"19":1,"69":1}}],["leakyrelu",{"2":{"104":1}}],["learn",{"2":{"171":1}}],["learned",{"2":{"83":1}}],["learnable",{"2":{"67":1,"69":4}}],["learning",{"2":{"2":1,"17":1,"19":1,"21":1,"24":3,"53":1,"149":1,"155":1,"158":1,"168":1,"169":1,"171":1,"179":1,"204":1,"254":1}}],["leads",{"2":{"142":1}}],["leading",{"2":{"70":1}}],["lead",{"2":{"13":1,"33":1,"37":1,"55":1,"59":2,"130":1,"153":1,"208":1}}],["leaf",{"2":{"3":1,"8":1,"55":2,"77":12}}],["leaves",{"2":{"3":1,"55":4}}],["l",{"2":{"8":5,"9":2,"10":2,"31":13,"32":5,"37":2,"39":1,"62":2,"64":1,"121":5,"126":2,"127":2,"134":4,"140":16,"179":1,"184":2,"208":4,"214":4,"232":2,"236":2,"255":2,"257":3}}],["lag",{"2":{"244":1}}],["latter",{"2":{"158":2}}],["later",{"2":{"248":1}}],["latest",{"2":{"71":1,"203":1}}],["latentsize",{"2":{"66":3}}],["lazy",{"2":{"134":1}}],["language",{"2":{"55":2}}],["label=l",{"2":{"206":2,"209":4}}],["labels",{"2":{"53":2,"183":3,"192":2,"213":2,"224":4}}],["label",{"2":{"53":18}}],["larger",{"0":{"202":1},"2":{"83":1,"171":1,"202":1}}],["large",{"2":{"37":1,"83":3,"175":1}}],["layerfreezing",{"2":{"128":1}}],["layer2",{"2":{"62":1}}],["layer1",{"2":{"62":1}}],["layernorm",{"2":{"19":1,"69":4,"92":1}}],["layer",{"0":{"11":1,"32":1,"58":1,"59":1,"127":1,"139":1,"140":1,"141":1,"214":1,"215":1,"218":1,"225":1},"1":{"140":1,"141":1},"2":{"7":13,"8":9,"9":4,"10":4,"11":4,"19":3,"31":7,"32":6,"33":13,"37":10,"39":11,"57":5,"58":7,"59":11,"62":67,"63":5,"64":3,"65":12,"66":2,"67":5,"68":8,"69":24,"70":3,"77":47,"83":2,"102":2,"103":1,"104":1,"106":24,"108":2,"109":30,"110":69,"119":2,"121":3,"123":5,"126":9,"127":9,"128":5,"129":9,"134":5,"140":8,"141":12,"144":1,"145":4,"146":1,"147":4,"148":6,"153":2,"155":1,"166":1,"179":1,"184":1,"193":18,"214":2,"220":211,"225":1,"226":1,"233":2,"235":4,"236":4,"249":1,"256":12,"258":4}}],["layers",{"0":{"16":1,"18":1,"61":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"78":1,"126":1,"134":1,"137":1},"1":{"62":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"70":1},"2":{"6":1,"7":9,"8":2,"24":8,"32":3,"33":2,"37":2,"57":1,"58":2,"62":39,"63":1,"68":12,"78":1,"82":4,"83":4,"93":1,"102":2,"105":1,"109":9,"110":19,"121":4,"126":1,"133":1,"134":3,"138":2,"140":1,"141":3,"144":1,"155":1,"159":1,"165":1,"166":1,"184":1,"185":1,"214":1,"219":1,"220":6,"225":1,"249":2}}],["last",{"2":{"2":1,"15":1,"24":1,"58":1,"59":1,"66":1,"68":2,"70":2,"113":1,"121":1,"146":1,"214":1,"248":2}}],["luxlinear",{"2":{"134":7}}],["luxlib",{"0":{"12":1,"91":1},"1":{"13":1,"14":1,"15":1,"16":1,"17":1,"18":1,"19":1,"20":1,"92":1,"93":1},"2":{"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"20":1,"48":2,"60":2,"92":1,"145":4,"158":8,"161":1,"166":2,"167":2,"256":2}}],["luxflux",{"2":{"133":1,"134":2}}],["luxdeviceutils",{"2":{"98":3}}],["luxdl",{"2":{"66":1,"71":1,"79":1}}],["luxops",{"0":{"54":1},"2":{"54":8,"184":1,"185":1}}],["luxtestutils",{"0":{"47":1},"1":{"48":1,"49":1,"50":1},"2":{"48":3,"49":2,"50":1}}],["luxcorearrayinterfacereversediffext",{"2":{"122":2}}],["luxcorearrayinterfacetrackerext",{"2":{"122":2}}],["luxcore",{"0":{"6":1,"94":1},"1":{"7":1,"8":1,"9":1,"10":1,"11":1,"95":1,"96":1},"2":{"6":2,"7":3,"8":7,"9":2,"10":5,"11":1,"60":2,"95":2,"122":5,"138":3,"140":14,"141":8,"142":1,"161":1,"166":3}}],["luxcudadevice",{"2":{"99":1}}],["luxcuda",{"2":{"3":1,"72":2,"77":1,"78":1,"80":1,"130":1,"153":1,"168":1,"172":3,"182":1,"205":1,"212":1,"223":1,"231":1,"239":1}}],["lux",{"0":{"35":1,"37":1,"38":1,"39":1,"71":1,"81":1,"83":1,"90":1,"101":1,"106":1,"108":1,"133":1,"138":1,"161":1,"170":1,"204":1},"1":{"36":1,"37":1,"38":1,"39":2,"82":1,"83":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":2,"103":2,"104":2,"105":2,"107":1,"109":1,"110":1,"111":1,"134":1,"135":1,"136":1,"137":1,"139":1,"140":1,"141":1,"142":1,"143":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"4":4,"6":4,"7":3,"8":4,"11":1,"12":1,"27":2,"28":1,"31":13,"32":4,"33":8,"34":3,"37":11,"39":15,"40":1,"41":2,"42":3,"43":2,"44":4,"45":1,"46":1,"47":1,"48":2,"52":9,"53":16,"54":9,"55":9,"56":4,"57":2,"58":5,"59":19,"60":1,"62":11,"63":3,"64":7,"65":9,"66":9,"67":4,"68":12,"69":8,"70":2,"71":5,"72":4,"73":3,"74":1,"76":4,"77":8,"78":4,"79":1,"80":1,"81":1,"83":9,"84":1,"85":2,"88":1,"89":2,"90":2,"95":3,"98":1,"99":2,"102":6,"103":8,"106":7,"107":1,"108":3,"109":6,"110":4,"111":1,"112":1,"116":1,"119":1,"121":3,"122":1,"123":4,"124":1,"126":6,"127":2,"128":2,"129":3,"130":3,"131":1,"133":3,"134":13,"136":2,"137":2,"138":7,"140":3,"141":2,"142":1,"144":4,"145":1,"146":1,"147":1,"148":1,"152":1,"153":8,"155":6,"157":2,"161":3,"166":3,"169":1,"170":3,"172":1,"173":1,"174":2,"175":1,"177":1,"179":2,"181":2,"182":1,"184":7,"185":1,"187":2,"190":1,"191":1,"193":4,"194":1,"195":1,"196":2,"202":1,"203":3,"204":2,"205":1,"208":1,"212":1,"214":5,"215":1,"216":1,"218":4,"220":96,"223":1,"225":4,"226":2,"227":1,"228":1,"231":1,"235":2,"236":4,"238":2,"239":1,"240":1,"241":1,"244":6,"247":4,"249":7,"253":1,"256":2}}],["lt",{"2":{"4":2,"5":1,"24":3,"53":1,"59":2,"79":1,"95":2,"131":1,"141":2,"144":12,"206":1,"209":1}}],["literate",{"2":{"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["literature",{"2":{"149":1}}],["little",{"2":{"106":1,"110":1}}],["lib64",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["libllvm",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["libopenlibm",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["libm",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["libraries",{"2":{"144":1,"169":1,"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1,"247":2,"249":1}}],["library",{"2":{"74":1,"133":1,"173":1,"180":1,"188":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["lib",{"2":{"109":1,"180":1,"189":1,"197":1,"210":1,"220":3,"221":1,"229":1,"237":1,"244":4,"246":1}}],["linux",{"2":{"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["linked",{"2":{"203":1}}],["link",{"2":{"170":1}}],["linking",{"2":{"62":1}}],["lin",{"2":{"53":2}}],["liner",{"2":{"249":1}}],["linewidth=2",{"2":{"245":1,"255":1,"256":2,"259":3}}],["linewidth=3",{"2":{"232":1,"236":1,"250":3}}],["linewidth=4",{"2":{"206":2,"209":4,"259":1}}],["linesearch=linesearches",{"2":{"258":1}}],["linesearches",{"2":{"253":1}}],["linestyle=",{"2":{"206":2,"209":2}}],["lines",{"2":{"206":2,"209":4,"232":1,"236":1,"255":1,"256":2,"259":4}}],["line",{"2":{"49":1}}],["linearalgebra",{"2":{"144":1,"247":1}}],["linear",{"0":{"67":1,"179":1},"2":{"24":1,"59":4,"68":1,"70":1,"140":12,"141":23,"153":1,"179":1}}],["lighter",{"2":{"138":1}}],["light",{"2":{"21":1}}],["lightweight",{"2":{"0":1}}],["like",{"2":{"15":1,"16":2,"18":1,"20":1,"24":1,"28":1,"47":1,"52":1,"53":3,"55":1,"59":3,"63":2,"66":3,"83":2,"96":1,"133":1,"134":2,"138":1,"140":2,"142":1,"160":1,"169":1,"171":3,"177":1,"204":1,"214":1,"217":1,"248":1,"255":1,"256":1}}],["limits=extrema",{"2":{"245":1}}],["limiter",{"2":{"220":26}}],["limitations",{"2":{"37":1,"52":1,"83":1}}],["limit",{"2":{"13":1,"17":1,"54":1,"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["lists",{"2":{"34":1,"85":1}}],["listed",{"2":{"30":1,"57":1}}],["list",{"2":{"3":1,"34":5,"42":1,"48":1,"49":3,"62":4,"68":1,"78":1,"90":2,"161":1,"168":1,"171":1,"203":1,"228":4}}],["lost",{"2":{"170":1}}],["lossfn",{"2":{"179":3,"186":2,"187":2}}],["lossfunctions",{"2":{"53":1}}],["losses",{"2":{"53":1,"257":2,"259":3}}],["loss",{"0":{"53":1,"145":1,"147":1,"148":1,"186":1,"235":1,"242":1},"1":{"146":1},"2":{"52":5,"53":35,"59":5,"77":2,"78":14,"106":5,"107":4,"144":1,"145":10,"146":9,"147":4,"148":6,"179":19,"186":4,"187":456,"194":1,"195":2,"208":30,"227":1,"228":1,"235":4,"236":10,"238":1,"240":1,"242":23,"244":436,"257":6,"258":1,"259":2}}],["lower",{"2":{"217":1}}],["low",{"2":{"107":1}}],["love",{"2":{"88":1}}],["lotka",{"2":{"206":2}}],["lot",{"2":{"83":1,"174":1}}],["logs",{"2":{"113":1}}],["logging",{"2":{"113":1,"247":1}}],["logarithm",{"2":{"53":1}}],["logarithmic",{"2":{"53":1}}],["log⁡",{"2":{"53":1}}],["log",{"2":{"53":2,"54":2,"249":1,"250":1}}],["logitcrossentropy",{"2":{"216":1,"217":1}}],["logitbinarycrossentropy",{"2":{"186":1}}],["logitbce",{"2":{"53":5}}],["logit",{"2":{"53":1}}],["logits=val",{"2":{"53":2,"194":1,"216":1,"227":1}}],["logits",{"2":{"53":4}}],["longer",{"2":{"102":2,"115":3}}],["long",{"2":{"37":1,"66":1,"142":1}}],["location",{"2":{"33":3,"109":4,"110":10}}],["local",{"2":{"4":1,"43":2,"59":1,"113":1,"114":1,"180":2,"189":2,"196":1,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["localpreferences",{"2":{"1":1,"131":2}}],["loosely",{"2":{"170":1}}],["looks",{"2":{"160":1}}],["lookup",{"2":{"67":1}}],["look",{"2":{"31":1,"70":1,"177":1,"184":1,"211":1}}],["loops",{"2":{"20":1,"55":1}}],["loopedarrayop",{"2":{"20":1}}],["loopvectorization",{"2":{"13":1,"14":2,"18":1,"20":1,"159":1,"167":1}}],["loop",{"0":{"167":1,"195":1},"2":{"13":1,"20":1,"159":1,"167":1,"236":1}}],["looping",{"2":{"5":1}}],["lo",{"2":{"24":2}}],["loadmnist",{"2":{"192":2,"195":1,"213":2,"217":1}}],["loader",{"2":{"107":1,"187":4}}],["loaded",{"2":{"2":2,"3":7,"4":2,"10":1,"11":1,"18":1,"46":1,"95":2,"113":1,"131":2,"137":1,"167":1,"172":1}}],["load",{"2":{"13":1,"14":1,"18":1,"46":1,"47":1,"102":1,"107":1,"130":1,"159":1,"188":1,"192":1,"213":1,"224":4,"228":1}}],["loading",{"0":{"160":1,"192":1,"213":1,"224":1},"2":{"7":1,"14":1,"18":1,"37":1,"39":1,"80":1,"160":1,"188":1}}],["=ϕwhere",{"2":{"255":1,"256":1}}],["=χu",{"2":{"255":1,"256":1}}],["=∫θp",{"2":{"250":1}}],["=wx+b",{"2":{"179":1}}],["=x",{"2":{"176":1}}],["=12xtx",{"2":{"176":1}}],["=1v∑i=1vvitjvinote",{"2":{"149":1}}],["=1v∑i=1vvitaviwe",{"2":{"149":1}}],["=e",{"2":{"149":1}}],["=i",{"2":{"149":1}}],["=vcat",{"2":{"66":1}}],["=val",{"2":{"39":1,"53":1}}],["=true",{"2":{"33":1}}],["=static",{"2":{"33":1}}],["=>",{"2":{"31":2,"32":3,"34":8,"37":2,"39":5,"59":8,"62":6,"63":2,"66":4,"67":4,"69":12,"77":6,"78":8,"106":6,"107":3,"109":17,"110":14,"121":4,"133":4,"145":2,"146":2,"147":4,"148":4,"152":4,"155":1,"179":1,"184":2,"185":2,"193":15,"215":5,"226":1,"233":4,"236":2,"241":4,"249":3,"256":6}}],["=0",{"2":{"24":2}}],["=nothing",{"2":{"2":1,"53":2,"66":1,"69":1}}],["==3",{"2":{"254":1}}],["===",{"2":{"34":4}}],["==",{"2":{"1":1,"3":1,"17":1,"24":4,"28":1,"54":2,"58":1,"59":1,"63":10,"65":15,"67":3,"70":2,"78":2,"107":2,"113":1,"121":1,"127":1,"128":2,"142":2,"143":1,"179":2,"186":1,"194":1,"208":1,"216":1,"227":1,"228":2,"236":2,"244":2,"249":1,"254":5}}],["=",{"2":{"1":2,"2":2,"3":1,"5":8,"15":1,"17":3,"19":6,"24":14,"28":1,"31":2,"32":5,"34":8,"37":4,"39":5,"41":3,"48":3,"49":2,"53":59,"55":1,"58":2,"59":46,"62":40,"63":7,"64":6,"65":9,"66":6,"67":14,"68":25,"69":15,"70":3,"72":4,"73":3,"74":13,"77":52,"78":26,"95":2,"106":61,"107":10,"109":25,"110":47,"113":7,"114":1,"121":10,"122":3,"123":3,"124":2,"126":29,"127":1,"129":23,"131":4,"132":7,"133":7,"134":12,"140":8,"141":9,"142":5,"145":21,"146":12,"147":20,"148":21,"150":2,"151":2,"152":12,"153":19,"155":5,"160":6,"168":9,"169":6,"170":1,"171":8,"172":1,"173":3,"174":3,"176":3,"177":4,"178":1,"179":17,"180":8,"183":6,"184":6,"185":5,"186":5,"187":13,"189":8,"192":7,"193":21,"194":5,"195":12,"196":1,"197":8,"205":2,"206":12,"207":3,"208":19,"209":6,"210":8,"213":7,"214":6,"215":5,"216":5,"217":10,"218":3,"220":136,"221":8,"224":4,"225":2,"226":3,"227":5,"228":21,"229":8,"232":8,"233":3,"234":1,"235":12,"236":13,"237":8,"239":2,"242":13,"243":19,"244":20,"245":11,"246":8,"248":21,"249":19,"250":19,"251":7,"254":55,"255":20,"256":38,"257":4,"258":14,"259":13,"260":7}}],["jointly",{"2":{"184":1}}],["journal",{"2":{"17":1}}],["jvi",{"2":{"149":1}}],["jvp",{"0":{"27":1},"2":{"27":1,"151":6,"152":10,"175":4,"177":4}}],["j∈rd×d",{"2":{"149":1}}],["jet`",{"2":{"48":1}}],["jet",{"0":{"48":1},"2":{"48":14}}],["jax",{"2":{"170":1}}],["jacobian`",{"2":{"145":1}}],["jacobian",{"0":{"145":1,"146":1,"148":1,"150":1,"151":1,"152":1,"177":1,"178":1},"1":{"146":1},"2":{"27":11,"28":6,"144":3,"145":3,"146":9,"148":5,"149":4,"150":1,"151":1,"152":17,"175":6,"177":2,"178":1}}],["jamie",{"2":{"19":1}}],["j",{"2":{"24":1,"28":2,"145":2,"146":3,"148":2,"149":1,"152":8}}],["jimmy",{"2":{"19":1}}],["just",{"2":{"3":1,"59":1,"62":1,"64":3,"69":3,"77":1,"116":1,"131":1,"138":2,"169":1,"171":1,"177":1,"217":1}}],["julia∂t",{"2":{"147":1}}],["julia∂x",{"2":{"145":1,"146":1,"148":1}}],["juliazygote",{"2":{"110":1}}],["juliazeros",{"2":{"171":2}}],["juliazerosc64",{"2":{"25":1}}],["juliazerosc32",{"2":{"25":1}}],["juliazerosc16",{"2":{"25":1}}],["juliazeros64",{"2":{"25":1}}],["juliazeros32",{"2":{"25":1}}],["juliazeros16",{"2":{"25":1}}],["juliaxt",{"2":{"123":1}}],["juliax",{"2":{"106":1,"131":1,"155":2,"171":11,"173":2,"179":1}}],["juliaxlogy",{"2":{"54":1}}],["juliaxlogx",{"2":{"54":1}}],["juliaxla",{"2":{"2":1}}],["juliaw",{"2":{"171":1,"179":1}}],["juliaweights",{"2":{"168":1,"169":3}}],["juliaweightnorm",{"2":{"69":1}}],["juliawrappedfunction",{"2":{"68":1}}],["juliann",{"2":{"153":1}}],["julian",{"2":{"78":1,"179":1}}],["julianooplayer",{"2":{"68":1}}],["juliancclbackend",{"2":{"41":1}}],["juliaembedding",{"2":{"67":1}}],["juliaeachslice",{"2":{"54":1}}],["juliavjp",{"2":{"178":1,"236":1}}],["juliavariationalhiddendropout",{"2":{"64":1}}],["juliavector",{"2":{"27":1}}],["juliay",{"2":{"62":2}}],["juliapkg>",{"2":{"161":1}}],["juliaprob",{"2":{"255":1,"256":1,"259":1}}],["juliaprintln",{"2":{"152":1,"176":1}}],["juliapred",{"2":{"106":2}}],["juliaps",{"2":{"110":1,"113":1,"179":1,"187":1}}],["juliapixelshuffle",{"2":{"70":1}}],["juliaparallel",{"2":{"62":1}}],["juliaparameterlength",{"2":{"9":1}}],["juliapairwisefusion",{"2":{"62":1}}],["juliapoissonloss",{"2":{"53":1}}],["juliamodel",{"2":{"106":2,"107":1,"109":2,"110":4,"124":1,"141":1,"146":1,"152":1,"179":1,"220":2,"233":1}}],["juliameanpool",{"2":{"65":1}}],["juliamaxout",{"2":{"68":1}}],["juliamaxpool",{"2":{"65":1}}],["juliamatch",{"2":{"57":1}}],["juliamaeloss",{"2":{"53":1}}],["juliamultigate",{"2":{"54":1}}],["juliamsleloss",{"2":{"53":1}}],["juliamseloss",{"2":{"53":1}}],["juliampibackend",{"2":{"41":1}}],["juliakldivergenceloss",{"2":{"53":1}}],["juliakaiming",{"2":{"24":2}}],["juliahuberloss",{"2":{"53":1}}],["juliahingeloss",{"2":{"53":1}}],["juliaupsample",{"2":{"70":1}}],["juliaupdate",{"2":{"10":1}}],["juliausing",{"2":{"48":1,"72":8,"73":4,"77":1,"78":1,"106":1,"109":1,"110":1,"121":1,"126":1,"129":1,"133":2,"134":2,"140":1,"142":1,"144":1,"153":1,"155":1,"156":1,"161":1,"168":2,"170":1,"172":1,"175":1,"179":1,"180":1,"182":1,"189":1,"191":1,"197":1,"205":1,"210":1,"212":1,"221":1,"223":1,"229":1,"231":1,"237":1,"239":1,"246":1,"251":1,"253":1,"260":1}}],["juliaunfreeze",{"2":{"31":2}}],["juliajvp",{"2":{"177":1}}],["juliajet",{"2":{"48":1}}],["juliajacobian",{"2":{"27":1}}],["juliajulia>",{"2":{"5":1,"24":2,"31":1,"32":1,"34":1,"37":1,"39":1,"48":1,"49":1,"53":15,"59":6,"62":3,"68":4,"69":3,"71":2}}],["juliarng",{"2":{"78":1,"109":1,"122":1,"134":2,"140":1,"170":1,"174":1,"232":1}}],["juliarnncell",{"2":{"66":1}}],["juliarandom",{"2":{"174":1}}],["juliarandc64",{"2":{"25":1}}],["juliarandc32",{"2":{"25":1}}],["juliarandc16",{"2":{"25":1}}],["juliarand64",{"2":{"25":1}}],["juliarandnc64",{"2":{"25":1}}],["juliarandnc32",{"2":{"25":1}}],["juliarandnc16",{"2":{"25":1}}],["juliarandn64",{"2":{"25":1}}],["juliarandn32",{"2":{"25":1}}],["juliarandn16",{"2":{"25":1}}],["juliarand32",{"2":{"25":1}}],["juliarand16",{"2":{"25":1}}],["juliareversesequence",{"2":{"68":1}}],["juliarecurrence",{"2":{"66":1}}],["juliarecursive",{"2":{"55":6}}],["juliareshapelayer",{"2":{"68":1}}],["juliares",{"2":{"62":2}}],["juliareset",{"2":{"3":1}}],["juliarepeatedlayer",{"2":{"62":1}}],["juliareplicate",{"2":{"8":1}}],["juliareduce",{"2":{"44":1}}],["juliaopt",{"2":{"113":1,"234":1}}],["juliaonesc64",{"2":{"25":1}}],["juliaonesc32",{"2":{"25":1}}],["juliaonesc16",{"2":{"25":1}}],["juliaones64",{"2":{"25":1}}],["juliaones32",{"2":{"25":1}}],["juliaones16",{"2":{"25":1}}],["juliaorthogonal",{"2":{"24":1}}],["juliaoutputsize",{"2":{"11":1}}],["julialux",{"2":{"193":1}}],["julialuxops",{"2":{"54":1}}],["julialength",{"2":{"171":1}}],["julialang",{"2":{"109":1,"180":1,"189":1,"197":1,"210":1,"220":3,"221":1,"229":1,"237":1,"244":4,"246":1,"251":1,"260":1}}],["julialayer",{"2":{"32":1}}],["julialayernorm",{"2":{"19":1,"69":1}}],["julialstmcell",{"2":{"66":1}}],["julialppool",{"2":{"65":1}}],["julialossfn",{"2":{"179":1}}],["julialoss",{"2":{"106":1,"257":1}}],["julialocal",{"2":{"43":1}}],["julialoaded",{"2":{"3":1}}],["juliabegin",{"2":{"232":1,"236":1,"259":1}}],["juliabackend",{"2":{"113":1,"153":2}}],["juliabatchnorm",{"2":{"19":1,"69":1}}],["juliabatched",{"2":{"14":1,"28":1}}],["juliabranchlayer",{"2":{"62":1}}],["juliabilinear",{"2":{"67":1}}],["juliabidirectionalrnn",{"2":{"66":1}}],["juliabinaryfocalloss",{"2":{"53":1}}],["juliabinarycrossentropyloss",{"2":{"53":1}}],["juliabias",{"2":{"15":2}}],["juliabcast",{"2":{"44":1}}],["juliafig",{"2":{"250":2}}],["juliaf",{"2":{"176":1,"177":1}}],["juliafmap",{"2":{"106":1}}],["juliaflattenlayer",{"2":{"68":1}}],["juliafluxlayer",{"2":{"37":1}}],["juliaf64",{"2":{"56":1}}],["juliaf32",{"2":{"56":1}}],["juliaf16",{"2":{"56":1}}],["juliafor",{"2":{"174":1}}],["juliafoldl",{"2":{"54":1}}],["juliafocalloss",{"2":{"53":1}}],["juliafromfluxadaptor",{"2":{"37":1}}],["juliafrozenlayer",{"2":{"31":1}}],["juliafreeze",{"2":{"31":2}}],["juliafunction",{"2":{"106":2,"110":1,"123":1,"140":2,"145":1,"147":1,"148":1,"150":1,"151":1,"152":1,"179":1,"183":1,"184":2,"185":1,"187":1,"192":1,"195":1,"206":1,"208":1,"213":1,"214":2,"215":1,"217":1,"224":1,"225":2,"226":1,"228":1,"232":1,"236":1,"242":1,"244":1,"249":1,"254":4,"255":1,"256":1}}],["juliafunctional",{"2":{"3":1}}],["juliafused",{"2":{"16":1,"18":1}}],["juliafast",{"2":{"13":2}}],["juliats",{"2":{"245":1}}],["juliatstate",{"2":{"236":1}}],["juliatest",{"2":{"49":1}}],["juliatestmode",{"2":{"10":1}}],["juliatotal",{"2":{"43":1}}],["juliatosimplechainsadaptor",{"2":{"39":1}}],["juliatr",{"2":{"152":1,"196":1}}],["juliatry",{"2":{"109":1,"110":1}}],["juliatrain",{"2":{"196":1,"217":4,"219":1}}],["juliatrainstate",{"2":{"52":1}}],["juliatrainmode",{"2":{"10":1}}],["juliatruncated",{"2":{"24":1}}],["juliaimport",{"2":{"76":1}}],["juliaistraining",{"2":{"54":1}}],["juliaisleaf",{"2":{"3":1}}],["juliaidentity",{"2":{"24":1}}],["juliainput",{"2":{"153":1}}],["juliainternal",{"2":{"20":1}}],["juliainstancenorm",{"2":{"19":1,"69":1}}],["juliainitialized",{"2":{"42":1}}],["juliainitialize",{"2":{"42":1}}],["juliainitialstates",{"2":{"10":1}}],["juliainitialparameters",{"2":{"9":1}}],["juliadudt",{"2":{"209":1}}],["juliadataloader",{"2":{"160":2}}],["juliadata",{"2":{"113":1}}],["juliadicecoeffloss",{"2":{"53":1}}],["juliadistributedutils",{"2":{"113":1}}],["juliadistributeddatacontainer",{"2":{"46":1}}],["juliadistributedoptimizer",{"2":{"45":1}}],["juliadisplay",{"2":{"8":1}}],["juliadropout",{"2":{"17":1,"64":1}}],["juliadense",{"2":{"67":1}}],["juliadebuglayer",{"2":{"33":1}}],["juliadeviceiterator",{"2":{"5":1}}],["juliadefault",{"2":{"3":1}}],["juliacdev",{"2":{"131":1,"132":1}}],["juliachain",{"2":{"62":1}}],["juliacheck",{"2":{"8":1}}],["juliacrossentropyloss",{"2":{"53":1}}],["juliaconst",{"2":{"106":1,"186":1,"194":1,"216":1,"227":1,"235":1,"249":1,"256":2,"257":2}}],["juliaconvtranspose",{"2":{"63":1}}],["juliaconv",{"2":{"63":1}}],["juliacontains",{"2":{"8":1}}],["juliacompute",{"2":{"52":1}}],["juliacpu",{"2":{"2":1}}],["juliaadtype",{"2":{"258":1}}],["juliaadaptor",{"2":{"193":1}}],["juliaadaptivemeanpool",{"2":{"65":1}}],["juliaadaptivemaxpool",{"2":{"65":1}}],["juliaadaptivelppool",{"2":{"65":1}}],["juliaadapt",{"2":{"37":1,"39":1}}],["juliaanalytical",{"2":{"243":1}}],["juliaallreduce",{"2":{"44":1}}],["juliaalphadropout",{"2":{"64":1}}],["juliaalpha",{"2":{"17":1}}],["juliaapply",{"2":{"8":1,"52":2}}],["juliaabstract",{"2":{"7":3}}],["julia>",{"2":{"5":4,"24":3,"32":6,"34":3,"37":6,"39":7,"48":1,"49":2,"53":45,"59":34,"62":3,"68":8,"71":2}}],["juliastruct",{"2":{"123":1,"141":1,"142":1,"184":1,"207":1,"214":1,"218":1,"241":1}}],["juliastatefulrecurrentcell",{"2":{"66":1}}],["juliastatefulluxlayer",{"2":{"58":1}}],["juliastatelength",{"2":{"10":1}}],["juliastateless",{"2":{"8":1}}],["juliaselectdim",{"2":{"68":1}}],["juliasetup",{"2":{"8":1}}],["juliaset",{"2":{"4":2,"60":1}}],["juliascale",{"2":{"67":1}}],["juliaskipconnection",{"2":{"62":1}}],["juliasquaredhingeloss",{"2":{"53":1}}],["juliasize",{"2":{"171":1}}],["juliasiamesecontrastiveloss",{"2":{"53":1}}],["juliasingle",{"2":{"52":2}}],["juliasimplechainslayer",{"2":{"39":1}}],["juliasynchronize",{"2":{"44":1}}],["juliashare",{"2":{"34":1}}],["juliasparse",{"2":{"24":1}}],["juliasupported",{"2":{"3":1}}],["juliagdev",{"2":{"131":1,"153":1}}],["juliagrucell",{"2":{"66":1}}],["juliagroupnorm",{"2":{"19":1,"69":1}}],["juliaglobalmeanpool",{"2":{"65":1}}],["juliaglobalmaxpool",{"2":{"65":1}}],["juliagloballppool",{"2":{"65":1}}],["juliaglorot",{"2":{"24":2}}],["juliagenericlossfunction",{"2":{"53":1}}],["juliagetproperty",{"2":{"54":1}}],["juliaget",{"2":{"3":2,"42":1}}],["juliagpu",{"2":{"1":1,"2":1}}],["julia",{"0":{"170":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"1":1,"16":1,"20":1,"33":1,"37":1,"48":1,"49":1,"50":1,"59":3,"60":1,"71":2,"74":1,"76":2,"77":3,"79":1,"81":1,"83":1,"84":1,"85":1,"102":2,"106":1,"122":3,"127":2,"128":2,"130":1,"131":1,"140":1,"144":1,"145":3,"147":1,"148":1,"152":1,"153":2,"154":1,"155":1,"167":2,"168":4,"170":2,"171":7,"174":1,"175":1,"180":12,"188":2,"189":12,"197":9,"210":12,"220":2,"221":12,"229":12,"237":12,"242":1,"246":12,"247":1,"248":1,"249":4,"250":2,"251":9,"254":1,"260":9}}],["jld2",{"2":{"182":1,"188":3}}],["jll",{"2":{"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["jl`",{"2":{"18":1,"145":2}}],["jl",{"0":{"45":1,"46":1,"48":1,"71":1,"91":1,"94":1,"97":1,"98":1,"101":1,"106":1,"114":1,"161":1,"204":1},"1":{"92":1,"93":1,"95":1,"96":1,"99":1,"100":1,"102":1,"103":1,"104":1,"105":1,"107":1,"115":1,"116":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"0":1,"2":3,"3":8,"6":5,"7":2,"8":1,"10":2,"11":1,"12":1,"13":3,"14":1,"18":2,"20":2,"22":2,"27":1,"28":1,"37":1,"39":4,"42":8,"45":1,"46":2,"47":1,"48":1,"49":6,"52":7,"53":2,"54":1,"60":1,"66":1,"71":4,"76":2,"78":3,"79":1,"80":5,"83":2,"86":8,"87":7,"89":1,"95":3,"98":5,"100":1,"102":1,"106":3,"107":1,"109":2,"112":1,"113":1,"114":1,"115":1,"116":1,"122":3,"130":1,"138":5,"140":1,"145":2,"153":3,"157":2,"158":2,"159":2,"160":3,"161":3,"163":1,"167":2,"168":1,"170":1,"172":1,"173":1,"175":4,"177":1,"179":1,"180":1,"181":2,"189":1,"190":4,"193":1,"196":1,"197":1,"203":3,"204":3,"208":3,"210":1,"211":1,"214":1,"220":6,"221":1,"229":1,"234":1,"237":1,"238":3,"244":4,"246":1,"247":2,"251":1,"252":1,"255":1,"256":1,"260":1}}],["p=res",{"2":{"259":1}}],["p=θ",{"2":{"257":1}}],["p=params",{"2":{"256":1}}],["p=2",{"2":{"65":3}}],["p^",{"2":{"255":1,"256":1}}],["p^2",{"2":{"255":1}}],["pdes",{"2":{"238":1}}],["pde",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"243":8,"244":2,"245":3}}],["pcie",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["physics",{"2":{"242":6,"244":108,"256":1}}],["philosophies",{"2":{"134":1}}],["phase",{"2":{"69":4}}],["pptional",{"2":{"77":1,"78":1}}],["pkg>",{"2":{"161":2}}],["pkg",{"2":{"71":4,"72":9,"73":2,"76":2,"77":1,"79":2,"161":1,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["pytorch",{"2":{"63":3,"67":1,"83":1,"104":3,"105":1,"170":1}}],["p2",{"2":{"53":6}}],["p1",{"2":{"53":4}}],["pipe",{"2":{"207":1}}],["pipeline",{"2":{"124":1}}],["pipelines",{"2":{"52":1}}],["pitfalls",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1}}],["pinn2dpde",{"2":{"244":8}}],["pinn",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"238":1,"241":4,"244":5,"245":1}}],["pinns",{"2":{"147":1,"238":1}}],["pin",{"2":{"108":2}}],["pixel",{"2":{"70":2}}],["pixelshuffle",{"2":{"70":1}}],["pixels",{"2":{"65":3,"70":2}}],["pi",{"2":{"53":1}}],["pmlr",{"2":{"19":1}}],["p",{"2":{"17":7,"37":1,"59":2,"64":12,"206":2,"208":4,"209":2,"214":6,"218":2,"250":2,"254":3,"255":6,"256":2,"258":1}}],["peel",{"2":{"184":1,"185":1}}],["peel`",{"2":{"184":1}}],["people",{"2":{"140":1}}],["pesky",{"2":{"110":1}}],["pended",{"2":{"102":1}}],["penalties",{"2":{"59":1}}],["penultimate",{"2":{"15":1,"16":1,"66":2}}],["perspective",{"2":{"254":1}}],["perceptron",{"2":{"230":1}}],["per",{"2":{"69":4,"146":1,"249":2}}],["periodic",{"2":{"63":1}}],["period",{"2":{"30":1}}],["perfect",{"2":{"24":1}}],["performs",{"2":{"131":1,"208":1}}],["performing",{"2":{"65":3}}],["performed",{"2":{"39":2,"63":1,"249":1}}],["performance",{"0":{"154":1,"159":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"13":1,"24":2,"33":1,"52":1,"59":3,"87":2,"88":2,"89":1,"130":2,"153":1,"154":2,"155":2,"159":1}}],["perform",{"2":{"8":1,"44":2,"52":2,"63":3,"134":1,"249":1,"254":1}}],["plot",{"2":{"236":1,"248":3,"250":9,"255":1,"256":1,"259":1}}],["plotting",{"0":{"209":1}}],["platform",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["plan",{"2":{"144":1}}],["places",{"2":{"59":2}}],["place",{"2":{"13":1,"15":2,"55":5,"116":1,"130":1}}],["plugin",{"2":{"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["plus",{"2":{"31":1,"34":1,"59":3,"62":6,"69":7,"77":1,"78":1,"109":2,"110":2,"121":1,"193":2,"233":1,"256":1}}],["please",{"2":{"4":2,"74":2,"76":1,"78":1,"84":1,"89":1,"111":1,"117":1,"144":1,"184":1,"202":1,"203":3,"204":1,"238":1}}],["push",{"2":{"257":1}}],["pullback",{"2":{"110":2}}],["pull",{"2":{"84":1,"95":1}}],["publisher",{"2":{"74":1}}],["public",{"2":{"3":1,"53":1}}],["purpose",{"2":{"175":1}}],["purposes",{"2":{"57":1,"142":1,"208":1,"213":1}}],["pure",{"2":{"8":1,"81":1,"82":1,"83":1,"173":1}}],["pseudorandom",{"2":{"174":1}}],["pseudo",{"2":{"64":3,"170":1}}],["ps",{"2":{"7":2,"8":5,"31":2,"32":18,"34":17,"37":4,"39":2,"44":2,"53":2,"57":1,"58":5,"59":22,"62":4,"68":9,"77":3,"78":6,"106":16,"107":6,"109":5,"110":12,"113":2,"119":3,"121":2,"122":2,"123":6,"124":2,"126":9,"127":7,"128":6,"129":2,"133":5,"134":7,"140":7,"141":7,"142":10,"144":3,"145":9,"146":9,"147":8,"148":11,"150":2,"151":2,"152":9,"153":7,"155":5,"179":7,"184":4,"186":2,"187":3,"188":3,"194":2,"195":2,"208":4,"209":1,"214":2,"215":5,"216":2,"217":2,"218":3,"220":15,"225":2,"227":2,"228":2,"235":1,"236":1,"242":4,"244":3,"249":10,"250":1,"256":2}}],["power",{"2":{"250":1}}],["powerful",{"2":{"171":1}}],["pooling",{"0":{"65":1},"2":{"65":21,"105":1}}],["poisson",{"2":{"53":1}}],["poissonloss",{"2":{"53":2}}],["points",{"2":{"134":1,"179":1,"232":1,"248":4,"249":1,"250":1}}],["point",{"0":{"56":1},"2":{"4":2,"13":1,"56":3,"108":2,"113":1,"125":1,"130":1,"171":1,"255":1,"256":1}}],["populated",{"2":{"184":2}}],["populate",{"2":{"33":1}}],["polynomial",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"230":1,"232":1}}],["polyester",{"2":{"20":1}}],["polyalgorithm",{"2":{"18":1}}],["position=",{"2":{"206":1,"209":1,"256":1,"259":1}}],["position",{"2":{"68":2,"254":1}}],["possibly",{"2":{"63":2,"65":3,"87":1}}],["possible",{"2":{"1":1,"2":1,"7":1,"11":1,"13":3,"15":1,"16":2,"18":1,"37":2,"42":2,"88":1,"145":1}}],["posterior",{"2":{"250":2}}],["posted",{"2":{"144":1}}],["post",{"2":{"7":1,"39":1}}],["potentially",{"2":{"7":1,"11":1,"39":1}}],["painful",{"2":{"108":1}}],["pairwisefusion",{"2":{"62":3}}],["pairs",{"2":{"53":1,"220":13}}],["pair",{"2":{"53":1,"63":2,"66":1}}],["past",{"2":{"83":1}}],["passes",{"2":{"62":2,"111":1,"173":1}}],["passed",{"2":{"10":1,"19":1,"34":2,"48":1,"49":1,"59":1,"62":6,"66":5,"68":6,"82":1,"104":1,"141":1}}],["passing",{"2":{"8":1,"59":2,"62":1,"69":1,"113":2,"168":2}}],["pass",{"2":{"8":1,"27":2,"49":1,"50":1,"54":2,"56":1,"59":3,"62":1,"63":1,"67":2,"69":1,"78":1,"83":1,"102":1,"110":3,"113":1,"140":1,"168":1,"184":3}}],["pal2023efficient",{"2":{"74":1}}],["pal2023lux",{"2":{"74":1}}],["pal",{"2":{"74":2}}],["pattern",{"2":{"53":1,"106":1,"160":1,"248":1}}],["path",{"2":{"10":1,"62":1,"103":2,"126":2,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"254":4,"260":1}}],["page",{"2":{"29":1,"30":1,"31":1,"58":1,"90":1,"106":1,"108":1,"153":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["pad",{"2":{"63":6,"65":3}}],["pad=0",{"2":{"63":2,"65":3}}],["padding",{"2":{"24":1,"63":9,"65":12}}],["pads",{"2":{"24":1}}],["parse",{"2":{"146":1}}],["parallelism",{"2":{"160":1}}],["parallel=true",{"2":{"160":2}}],["parallel",{"0":{"112":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1},"2":{"32":1,"62":12,"68":1}}],["params=nothing",{"2":{"254":2}}],["params",{"2":{"31":14,"32":6,"52":2,"69":3,"126":3,"129":1,"254":4,"255":3,"256":6,"257":2,"258":1,"259":1}}],["parameterized",{"2":{"250":1}}],["parameterization",{"2":{"74":1,"247":1}}],["parameter",{"0":{"31":1,"142":1},"2":{"24":3,"31":2,"34":2,"53":1,"59":6,"63":4,"68":1,"69":2,"77":1,"82":1,"83":4,"110":1,"113":1,"128":2,"134":1,"140":5,"141":2,"142":5,"208":2,"217":1,"247":1,"249":2}}],["parameterlength",{"2":{"7":1,"9":1,"140":4,"141":1,"226":1,"249":3}}],["parameters",{"0":{"9":1,"34":1,"125":1,"128":1,"148":1},"1":{"126":1,"127":1,"128":1,"129":1},"2":{"3":1,"7":5,"8":1,"9":2,"24":1,"31":13,"32":7,"33":1,"34":23,"37":4,"39":1,"45":1,"52":9,"53":1,"56":2,"57":4,"58":5,"59":17,"62":24,"63":2,"66":9,"67":4,"68":2,"69":31,"77":5,"78":7,"81":1,"83":2,"95":2,"105":1,"107":1,"109":6,"110":7,"113":1,"121":5,"125":2,"128":3,"134":6,"138":1,"140":5,"141":3,"142":7,"144":1,"148":1,"175":1,"179":6,"184":2,"187":2,"188":1,"193":12,"195":2,"208":5,"214":1,"217":2,"225":1,"228":4,"233":3,"235":1,"236":3,"244":1,"249":79,"250":2,"256":6}}],["parts",{"2":{"208":1}}],["party",{"0":{"203":1}}],["particle",{"2":{"255":2,"256":1}}],["particles",{"2":{"254":1}}],["participate",{"0":{"119":1}}],["particular",{"0":{"126":1},"2":{"108":1,"126":1}}],["partition",{"2":{"46":1}}],["part",{"0":{"128":1,"129":1},"2":{"3":1,"54":1,"90":1,"145":1,"202":1,"204":1,"214":1}}],["packages",{"0":{"35":1,"79":1,"205":1},"1":{"36":1,"37":1,"38":1,"39":1},"2":{"3":3,"27":4,"28":2,"39":1,"52":2,"60":1,"77":1,"78":1,"79":3,"80":1,"85":2,"88":3,"89":2,"95":1,"98":1,"122":3,"130":1,"145":2,"154":1,"159":1,"166":1,"167":3,"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1,"247":1}}],["package",{"0":{"182":1,"191":1,"212":1,"223":1,"231":1,"239":1,"253":1},"2":{"0":1,"3":1,"4":3,"21":1,"22":1,"45":1,"47":3,"72":1,"73":1,"76":1,"79":1,"85":1,"86":1,"89":1,"98":3,"102":1,"131":2,"164":1,"169":1,"172":1,"203":1,"238":1}}],["practitioners",{"2":{"208":1}}],["prngs",{"2":{"174":1}}],["prng",{"2":{"170":1,"174":2}}],["prs",{"2":{"144":1}}],["primarily",{"2":{"249":1}}],["primitives",{"0":{"44":1,"158":1},"2":{"158":1,"249":1}}],["principles",{"0":{"82":1}}],["printf",{"2":{"78":2,"106":1,"107":1,"179":2,"182":1,"187":2,"191":1,"195":1,"205":1,"208":1,"212":1,"217":1,"223":1,"228":2,"231":1,"236":1,"239":1,"244":1,"253":1,"257":1}}],["printouts",{"2":{"83":1}}],["printout",{"2":{"59":1}}],["print",{"2":{"57":1,"59":1}}],["printing",{"2":{"39":1,"68":1,"78":1,"83":1}}],["println",{"2":{"32":1,"109":2,"110":2,"140":2,"141":3,"142":2,"145":2,"146":2,"147":2,"148":2,"152":6,"173":2,"174":2,"176":2,"177":1,"178":1,"179":3,"180":2,"189":2,"197":2,"210":2,"221":2,"228":1,"229":2,"237":2,"246":2,"251":2,"260":2}}],["prints",{"2":{"4":2,"110":1}}],["printed",{"2":{"2":1,"8":1,"37":1,"59":1}}],["priority",{"2":{"88":2}}],["prior",{"2":{"63":1,"249":1}}],["pr",{"2":{"11":1,"202":1,"203":1}}],["precompile",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["precision",{"0":{"56":1},"2":{"16":2,"56":1,"155":3,"171":1}}],["press",{"2":{"161":1}}],["pressing",{"2":{"76":1}}],["presence",{"2":{"59":1}}],["presented",{"2":{"177":1}}],["presently",{"2":{"106":1}}],["present",{"2":{"37":1,"39":1,"48":1,"54":1,"63":2,"66":10,"67":2,"102":1,"109":1,"110":1,"131":2}}],["preserving",{"2":{"39":1,"70":1}}],["preservation",{"2":{"37":1}}],["preserved",{"2":{"39":1}}],["preserve",{"2":{"37":7,"39":1}}],["pretty",{"2":{"68":1,"78":1,"83":1,"114":1,"145":1}}],["preds",{"2":{"249":2}}],["pred=ŷ",{"2":{"186":1}}],["predictive",{"2":{"250":1}}],["predictions",{"2":{"236":1,"249":2,"250":2}}],["prediction",{"0":{"250":1},"2":{"53":2,"249":1,"250":1}}],["predictor",{"2":{"245":1}}],["predict",{"2":{"183":1,"250":6}}],["predicted",{"2":{"53":3,"194":2,"216":2,"227":2,"250":2}}],["pred",{"2":{"53":8,"106":4,"186":5,"208":2,"209":3,"236":2,"245":4,"257":4}}],["preprint",{"2":{"19":2,"69":1}}],["preprocessing",{"2":{"8":1}}],["prevents",{"2":{"102":1}}],["prevent",{"2":{"17":2,"42":1,"52":1,"53":1,"102":1,"140":1}}],["previously",{"2":{"7":1,"11":1,"83":1,"95":1,"104":1,"115":1}}],["previous",{"2":{"5":1,"7":1,"83":1,"146":1}}],["pre",{"2":{"7":1,"11":1,"63":1,"76":1,"77":1,"102":1}}],["prefer",{"2":{"158":1}}],["preferably",{"2":{"20":1}}],["preferred",{"2":{"5":1,"20":1}}],["preferencetools",{"2":{"161":2}}],["preference",{"2":{"1":2,"2":1,"57":1,"60":2,"144":1,"155":1,"157":1,"161":6,"164":2,"165":1,"166":3,"167":1}}],["preferences",{"0":{"1":1,"161":1},"1":{"162":1,"163":1,"164":1,"165":1,"166":1,"167":1},"2":{"57":1,"60":4,"102":1,"157":2,"161":4,"163":3,"166":2}}],["progress",{"2":{"247":3}}],["programming",{"2":{"170":1}}],["proto",{"2":{"195":4}}],["probabilistic",{"2":{"249":2}}],["probability",{"2":{"17":3,"53":1,"64":3}}],["prob",{"2":{"206":2,"208":9,"209":2,"214":4,"218":2,"255":1,"256":1,"257":1,"259":1}}],["problem",{"0":{"240":1},"2":{"109":1,"110":1,"145":1,"148":1,"173":1,"179":2,"208":1,"218":1,"220":1,"233":1,"240":1}}],["problematic",{"2":{"108":1,"109":1}}],["problems",{"0":{"109":1},"2":{"81":1,"106":1}}],["promotion",{"0":{"155":1},"2":{"155":1}}],["promotions",{"2":{"57":1,"155":1}}],["promote",{"2":{"16":1,"155":1}}],["project",{"2":{"47":1}}],["produce",{"2":{"54":1,"95":1}}],["products",{"2":{"27":2}}],["product",{"0":{"150":1,"151":1,"177":1,"178":1},"2":{"27":8,"144":2,"149":4,"150":1,"151":1,"175":5,"177":2,"178":1,"243":1,"245":1}}],["prod",{"2":{"24":1}}],["propagating",{"2":{"58":1}}],["propagated",{"2":{"62":1}}],["propagate",{"2":{"8":1}}],["proper",{"2":{"124":1,"170":1}}],["properly",{"2":{"33":1}}],["properties",{"2":{"28":1,"62":1}}],["proportion",{"2":{"24":3}}],["providing",{"2":{"21":1,"193":1}}],["provides",{"2":{"39":1,"49":1,"58":1,"77":1,"100":1,"138":1,"161":1,"168":1,"177":1,"204":1,"218":1}}],["provide",{"2":{"11":1,"24":1,"37":1,"52":2,"59":1,"79":1,"85":1,"149":1,"172":1}}],["provided",{"2":{"4":2,"5":1,"17":1,"30":1,"32":2,"54":1,"66":3,"113":1,"138":1,"172":1,"235":1,"250":1}}],["proceeding",{"2":{"140":1}}],["proceedings",{"2":{"19":1,"24":5,"53":2}}],["proceeds",{"2":{"66":8}}],["processor",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["processes",{"2":{"45":1,"46":1,"113":3,"160":1}}],["processing",{"2":{"17":1,"66":1}}],["process",{"2":{"4":1,"113":1,"160":1,"192":1,"213":1,"243":1}}],["ddot",{"2":{"254":2}}],["ddp",{"2":{"112":1}}],["dt^2",{"2":{"254":1}}],["dt2",{"2":{"254":3}}],["dt",{"2":{"254":21,"255":4,"256":2,"257":2,"259":2}}],["dset",{"2":{"224":4}}],["dnns",{"2":{"108":1}}],["dl",{"2":{"54":1,"256":1}}],["dynamic",{"2":{"47":1}}],["dynamics",{"2":{"24":1}}],["d2",{"2":{"34":5,"121":1,"122":3,"254":3}}],["d2=dense",{"2":{"34":1,"121":1,"124":1}}],["d3",{"2":{"34":9}}],["d3=chain",{"2":{"34":1}}],["d1",{"2":{"34":5,"121":1,"122":3}}],["d1=dense",{"2":{"34":1,"121":1,"124":1}}],["d1×",{"2":{"19":2,"69":2}}],["driver",{"2":{"180":3,"189":3,"210":3,"221":3,"229":3,"237":3,"246":3}}],["dr",{"2":{"108":1}}],["drawn",{"2":{"24":5,"250":1}}],["drop",{"2":{"53":1,"144":1}}],["dropped",{"2":{"17":2,"27":2}}],["dropout",{"0":{"17":1,"64":1},"2":{"17":14,"64":17,"83":1,"126":1,"129":1,"136":1,"143":1}}],["duration",{"2":{"249":2}}],["during",{"2":{"3":1,"69":4}}],["dudt",{"2":{"208":2,"209":1,"214":4,"218":2}}],["du",{"2":{"206":3}}],["due",{"2":{"146":1,"153":1,"220":1}}],["dummy",{"2":{"77":1}}],["dual",{"2":{"20":1}}],["dmitry",{"2":{"19":1,"69":1}}],["d",{"2":{"8":1,"59":11,"63":4,"65":6,"68":2,"70":4,"88":1,"126":2,"127":5,"128":6,"142":4,"183":4,"217":2,"254":2}}],["domain",{"2":{"243":1}}],["domainerror",{"2":{"33":1,"110":2}}],["doubt",{"2":{"140":1}}],["dot",{"2":{"109":1,"206":2,"209":2,"220":3,"244":4}}],["doi",{"2":{"74":2,"254":2}}],["doing",{"2":{"55":4,"107":1,"225":1,"249":1}}],["documented",{"2":{"102":1}}],["document",{"2":{"85":1}}],["documentations",{"2":{"154":1}}],["documentation",{"2":{"7":1,"8":1,"14":1,"49":1,"60":1,"70":1,"79":1,"84":1,"104":2,"140":1,"165":1,"166":1,"196":1,"203":1}}],["doctor",{"0":{"166":1},"2":{"60":4,"157":1,"166":3}}],["docstrings",{"2":{"140":1}}],["docstring",{"2":{"102":1}}],["docs",{"2":{"59":2,"92":1,"204":1,"244":4,"247":2}}],["downstream",{"2":{"83":1,"89":1,"203":1}}],["downright",{"2":{"83":1,"138":1}}],["down",{"0":{"110":1},"2":{"53":2,"57":1,"109":1,"110":1,"111":1,"141":1,"155":1,"171":2}}],["does",{"2":{"13":1,"19":2,"57":1,"68":1}}],["doesn",{"2":{"13":2,"32":1,"34":1,"54":1,"59":3,"66":1,"119":1,"136":1,"140":2,"142":1,"143":1,"145":1,"226":1}}],["do",{"2":{"6":1,"59":14,"62":1,"68":2,"76":1,"78":2,"97":1,"106":2,"107":1,"136":1,"140":1,"141":1,"142":1,"143":1,"144":1,"149":2,"153":1,"155":1,"160":3,"171":2,"173":1,"184":1,"185":1,"193":1,"208":2,"214":1,"225":1,"233":1,"245":1,"249":1,"250":1,"254":1,"256":1}}],["done",{"2":{"13":2,"32":1,"57":1,"77":1,"113":1,"131":1,"146":1,"167":1}}],["don",{"2":{"4":2,"16":1,"18":1,"25":1,"47":1,"59":2,"77":1,"83":1,"88":2,"89":1,"108":1,"111":1,"116":1,"117":1,"134":1,"140":2,"142":1,"144":1,"146":1,"171":2,"173":2,"177":1,"183":1,"184":1,"188":1,"192":1,"207":1,"213":1,"225":1}}],["digits=2",{"2":{"228":4}}],["digits",{"2":{"171":2}}],["direactly",{"2":{"102":1}}],["direct",{"2":{"102":1}}],["direction",{"2":{"69":2}}],["directly",{"2":{"3":1,"5":1,"6":1,"8":1,"17":1,"31":1,"41":2,"52":1,"55":2,"62":2,"68":1,"70":1,"99":1,"103":1,"113":1,"132":1,"144":1,"168":1,"211":1}}],["diagonal",{"2":{"67":1,"249":1}}],["divisible",{"2":{"63":2}}],["divides",{"2":{"70":1}}],["divide",{"2":{"63":2}}],["diverges",{"2":{"53":1}}],["divergence",{"2":{"53":2}}],["dilation",{"2":{"63":2,"65":3}}],["dilation=1",{"2":{"63":2,"65":3}}],["dice",{"2":{"53":2}}],["dicecoeffloss",{"2":{"53":4}}],["diffeqsol",{"2":{"214":3,"215":1,"220":9}}],["diffeqflux",{"2":{"211":1}}],["differ",{"2":{"175":1}}],["differern",{"2":{"146":1}}],["differences",{"0":{"116":1},"2":{"112":1,"134":1,"145":2}}],["difference",{"2":{"64":1,"95":1,"106":3,"141":1,"145":2,"146":2,"149":1,"219":1,"254":2}}],["differentiate",{"2":{"106":1,"152":1,"173":1}}],["differentiationinterface",{"2":{"144":1,"177":2}}],["differentiation",{"0":{"26":1,"85":1,"144":1,"162":1,"175":1},"1":{"27":1,"28":1,"29":1,"86":1,"87":1,"88":1,"89":1,"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1,"176":1,"177":1,"178":1},"2":{"29":1,"106":1,"142":1,"144":1,"147":1,"153":1,"162":2,"170":1,"177":1}}],["differential",{"2":{"74":1,"257":1}}],["differently",{"2":{"62":1}}],["different",{"0":{"38":1},"1":{"39":1},"2":{"18":1,"20":1,"32":1,"53":1,"60":1,"62":1,"104":1,"134":1,"146":1,"165":1,"174":1,"208":1}}],["diff",{"2":{"145":1,"147":1,"148":1,"175":1}}],["diffractor",{"2":{"86":1}}],["difficulty",{"2":{"24":2}}],["dim",{"2":{"68":6,"107":1,"179":7}}],["dimwhere",{"2":{"53":1}}],["dims=3",{"2":{"183":1}}],["dims=ndims",{"2":{"121":1}}],["dims=colon",{"2":{"69":1}}],["dims=2",{"2":{"66":1}}],["dims=",{"2":{"64":2}}],["dims=pad",{"2":{"63":1}}],["dims=1",{"2":{"19":1,"53":1,"243":5}}],["dims",{"2":{"17":2,"19":3,"24":11,"39":2,"53":2,"54":1,"63":1,"64":4,"66":31,"67":55,"68":7,"69":3,"92":1,"140":12,"169":4,"184":9,"185":7,"241":11,"244":2}}],["dimensionmismatch",{"2":{"109":2}}],["dimensionality",{"2":{"53":1}}],["dimensional",{"2":{"19":2,"67":3,"70":2,"177":1}}],["dimensions",{"2":{"19":2,"24":11,"28":2,"39":1,"63":3,"65":9,"66":2,"67":10,"68":5,"69":4,"70":8,"105":1,"179":1,"193":1}}],["dimension",{"0":{"109":1},"2":{"15":2,"16":1,"19":1,"39":1,"54":1,"63":4,"64":2,"65":6,"66":8,"67":1,"68":9,"69":14,"92":1,"108":1,"121":1,"124":1,"169":1}}],["disabling",{"0":{"167":1},"2":{"159":1}}],["disabled",{"2":{"8":1,"63":2,"67":3,"156":1}}],["disable",{"2":{"8":1,"33":1,"60":3,"102":2,"110":1,"144":1,"156":1,"162":1,"166":1,"167":2}}],["disallow",{"2":{"156":1}}],["disaster",{"2":{"155":1}}],["discrete",{"2":{"214":1,"217":1}}],["discouraged",{"2":{"145":2}}],["discourse",{"2":{"144":1,"145":1,"147":1}}],["discuss",{"2":{"204":1}}],["discussed",{"2":{"111":1}}],["discussions",{"2":{"84":1}}],["discussion",{"2":{"66":1,"147":1,"177":1}}],["disruptive",{"2":{"81":1}}],["dissimilar",{"2":{"53":1}}],["distinguish",{"2":{"134":2}}],["distinction",{"2":{"83":1}}],["distinct",{"2":{"66":1}}],["distance",{"2":{"53":1}}],["distributions",{"2":{"53":1,"249":2}}],["distribution",{"2":{"24":8,"25":12,"53":5,"63":2,"66":6,"67":3,"250":1}}],["distributeddatacontainer",{"2":{"46":1,"113":2,"114":1}}],["distributedoptimizer",{"2":{"45":2,"113":2,"114":1}}],["distributedutils",{"0":{"113":1},"2":{"4":2,"40":1,"41":2,"42":3,"43":2,"44":6,"45":1,"46":1,"112":1,"113":11,"114":5,"115":1,"116":3}}],["distributed",{"0":{"40":1,"112":1},"1":{"41":1,"42":1,"43":1,"44":1,"45":1,"46":1,"113":1,"114":1,"115":1,"116":1,"117":1},"2":{"4":3,"24":2,"41":4,"42":7,"83":1,"113":2}}],["disjoint",{"2":{"34":1}}],["displays",{"2":{"248":1,"250":1}}],["displayed",{"2":{"131":1}}],["display",{"2":{"8":1,"83":1,"141":1}}],["dispatch=",{"2":{"225":1}}],["dispatch=nothing",{"2":{"59":1}}],["dispatch₋₋₋",{"2":{"220":3}}],["dispatching",{"0":{"118":1},"1":{"119":1,"120":1,"121":1,"122":1,"123":1,"124":1}}],["dispatchdoctor",{"2":{"8":1,"60":1,"157":1}}],["dispatches",{"2":{"3":1,"8":1,"20":2,"54":1,"59":1,"138":1}}],["dispatch",{"0":{"119":1,"123":1,"166":1},"2":{"3":1,"47":1,"59":2,"60":4,"119":2,"123":1,"124":1,"143":1,"153":1,"157":1,"166":3}}],["date",{"2":{"203":2}}],["datapoints",{"2":{"232":1}}],["datasize",{"2":{"206":1,"255":1}}],["datasets",{"0":{"224":1},"2":{"183":1,"224":2,"228":1}}],["dataset",{"0":{"183":1,"232":1},"2":{"46":1,"160":2,"183":7,"192":3,"208":1,"213":3,"224":2,"232":2,"248":2}}],["datatype",{"2":{"8":1}}],["dataloaders",{"2":{"183":3,"187":2,"228":3}}],["dataloader",{"0":{"207":1},"2":{"5":8,"100":1,"107":4,"160":5,"183":5,"192":3,"194":2,"195":5,"207":5,"208":4,"212":1,"213":3,"216":2,"217":5,"224":2,"227":2,"228":9,"244":6}}],["data",{"0":{"2":1,"112":1,"160":1,"206":1,"243":1,"248":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1},"2":{"0":1,"2":1,"3":3,"5":1,"46":2,"52":6,"53":1,"59":10,"63":5,"65":9,"66":1,"68":1,"69":6,"70":2,"78":10,"83":1,"100":1,"106":1,"107":2,"113":3,"140":1,"153":1,"155":1,"160":5,"179":1,"183":8,"192":6,"206":3,"207":3,"208":3,"209":2,"213":6,"227":2,"228":16,"230":1,"232":4,"235":1,"236":5,"242":5,"243":10,"244":113,"248":6,"249":1,"250":7,"255":3,"256":2,"257":1,"259":2}}],["danger",{"2":{"2":1,"4":2,"28":1,"42":1}}],["derivative",{"2":{"147":1}}],["derivatives",{"2":{"145":1,"146":1,"177":1,"240":1,"256":1}}],["de",{"2":{"145":2}}],["demonstration",{"2":{"208":1,"213":1,"238":1}}],["demonstrative",{"2":{"142":1}}],["demonstrate",{"2":{"134":1,"145":1,"179":1,"190":1}}],["deal",{"2":{"147":1,"214":1}}],["deadlocks",{"2":{"117":1}}],["deactivate",{"2":{"66":3}}],["decision",{"2":{"161":1}}],["decides",{"2":{"136":1}}],["decouples",{"2":{"69":1}}],["declared",{"2":{"59":1}}],["decrease",{"2":{"13":1}}],["debuglayer",{"2":{"33":3,"109":4,"110":4}}],["debug",{"2":{"33":5,"103":1,"108":4,"109":6,"110":10,"155":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["debugging",{"0":{"33":1,"108":1},"1":{"109":1,"110":1,"111":1},"2":{"33":3,"57":1,"108":2}}],["delta=0",{"2":{"53":1}}],["delta",{"2":{"53":2}}],["delving",{"2":{"24":2}}],["deleted",{"2":{"1":1}}],["descent",{"2":{"179":2}}],["describe",{"2":{"153":1}}],["describes",{"2":{"39":1,"108":1,"255":1,"256":1}}],["described",{"2":{"24":3,"62":2}}],["despite",{"2":{"52":1}}],["destructure",{"2":{"37":3,"83":2,"137":1}}],["desirable",{"2":{"167":1,"249":1}}],["desired",{"2":{"1":1,"63":1,"105":1}}],["design",{"0":{"82":1},"2":{"5":1,"81":1,"134":1}}],["density",{"2":{"249":1}}],["denselayerparameters",{"2":{"142":3}}],["dense",{"2":{"18":2,"31":2,"32":18,"34":4,"37":2,"39":3,"53":2,"59":10,"62":12,"67":2,"68":1,"69":12,"77":17,"78":6,"93":1,"104":1,"106":9,"107":3,"109":17,"110":14,"121":6,"122":2,"124":2,"126":9,"127":3,"129":3,"133":5,"140":1,"142":2,"145":2,"146":2,"147":4,"148":4,"152":4,"153":2,"155":1,"158":1,"166":1,"179":2,"184":2,"185":1,"193":9,"208":3,"215":5,"220":45,"226":4,"233":4,"236":4,"241":4,"249":4,"256":6}}],["denom",{"2":{"255":3,"256":3}}],["denominator",{"2":{"19":4,"69":4}}],["denotes",{"2":{"3":4}}],["detour",{"2":{"175":1}}],["detection",{"2":{"53":2}}],["detected",{"2":{"33":1,"110":2,"244":1}}],["deterministic",{"2":{"24":1,"83":1}}],["determined",{"2":{"77":1}}],["determines",{"2":{"39":1}}],["determine",{"2":{"3":1,"17":2,"19":2,"55":1,"116":1,"249":1}}],["details",{"0":{"135":1},"1":{"136":1},"2":{"14":1,"17":2,"19":4,"33":1,"34":1,"37":1,"39":2,"49":1,"58":1,"59":2,"60":1,"69":1,"70":1,"92":1,"95":1,"97":1,"102":1,"104":2,"140":1,"162":1,"165":1,"166":1,"172":1}}],["detailed",{"2":{"7":1,"31":1,"109":1}}],["depot",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["depots",{"2":{"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["depwarn",{"2":{"102":1}}],["deprecated",{"2":{"92":1,"244":4}}],["deprecation",{"2":{"30":1,"98":2,"145":2}}],["deprecations",{"2":{"2":1,"102":1}}],["depthwise",{"2":{"63":2}}],["depths",{"2":{"32":1}}],["dependent",{"0":{"121":1},"2":{"120":1}}],["dependencies",{"0":{"159":1},"2":{"47":1,"95":1,"167":1,"249":1}}],["dependency",{"2":{"6":1,"21":1,"47":1,"138":1,"146":1}}],["depend",{"2":{"6":1,"19":1,"138":1,"140":1}}],["depending",{"2":{"6":1,"8":1,"24":1}}],["defiing",{"0":{"256":1}}],["definition",{"0":{"240":1}}],["definitions",{"2":{"141":1}}],["definitely",{"2":{"152":1}}],["defining",{"0":{"78":1,"186":1},"2":{"0":1,"3":2,"11":1,"59":1,"78":1,"106":1,"119":1,"138":1,"140":1,"141":1,"142":1,"184":1,"185":2,"193":1,"225":1}}],["define",{"0":{"193":1,"195":1,"207":1,"214":1,"216":1,"227":1,"241":1,"242":1,"254":1},"2":{"7":1,"20":1,"78":1,"110":2,"119":2,"120":1,"123":1,"134":2,"138":2,"140":3,"141":1,"142":2,"153":1,"179":1,"184":3,"185":1,"186":1,"207":1,"208":1,"214":1,"241":1,"242":1,"249":3,"254":2,"256":2,"257":2}}],["defines",{"2":{"6":1,"7":2,"55":1,"254":1,"255":1}}],["defined",{"2":{"3":1,"8":1,"11":1}}],["defer",{"2":{"177":1}}],["defaults",{"0":{"104":1},"2":{"58":1,"66":3,"104":4,"256":1}}],["default",{"2":{"3":2,"8":1,"19":10,"22":5,"24":8,"25":24,"32":1,"37":1,"39":3,"56":1,"57":1,"59":1,"66":1,"68":4,"69":1,"73":4,"77":1,"78":1,"83":1,"92":1,"95":2,"102":1,"104":2,"106":1,"107":1,"129":1,"133":2,"134":2,"136":2,"140":1,"142":5,"155":1,"156":1,"163":1,"166":1,"167":1,"168":4,"169":1,"170":1,"171":1,"173":1,"179":1,"180":1,"184":1,"189":1,"197":1,"207":1,"208":1,"210":1,"215":1,"220":1,"221":1,"225":1,"229":1,"237":1,"244":1,"246":1,"248":1,"251":1,"256":1,"260":1}}],["dev=gpu",{"2":{"215":1}}],["developer",{"2":{"106":1}}],["developed",{"2":{"83":1,"203":1}}],["deviate",{"2":{"256":1}}],["deviation",{"2":{"24":3,"69":1}}],["devicememory",{"2":{"5":3,"77":18,"131":1,"153":2,"168":2,"172":1,"220":18}}],["deviceiterator",{"2":{"5":3,"100":2,"107":2,"207":1}}],["device",{"0":{"160":1},"2":{"2":25,"3":32,"4":11,"5":3,"20":1,"42":1,"72":5,"73":4,"77":2,"78":1,"98":1,"99":3,"103":2,"106":5,"107":5,"116":4,"131":8,"132":2,"134":3,"153":1,"160":5,"180":1,"187":2,"189":1,"205":2,"207":1,"210":1,"215":1,"217":2,"220":1,"221":1,"228":1,"229":1,"235":2,"236":1,"237":1,"239":2,"246":1}}],["devices",{"2":{"0":1,"42":4,"134":1}}],["dev",{"2":{"2":1,"3":1,"4":2,"5":4,"20":2,"72":4,"73":3,"77":5,"78":4,"187":3,"215":2,"217":3,"228":4,"235":3,"236":2}}],["deepcopy",{"2":{"52":1}}],["deep",{"2":{"2":1,"19":1,"21":1,"24":6,"74":1,"83":1,"155":1,"158":1,"168":1,"169":1}}],["axes",{"2":{"225":3}}],["axislegend",{"2":{"206":1,"209":1,"232":1,"236":1,"255":1,"256":1,"259":1}}],["axis",{"2":{"206":1,"209":1,"220":44,"232":1,"236":1,"245":1,"248":1,"250":1,"255":1,"256":1,"259":2}}],["ax",{"2":{"206":4,"209":6,"232":4,"236":5,"245":4,"248":3,"255":4,"256":6,"259":11}}],["a100",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["ai",{"2":{"177":1}}],["aims",{"2":{"24":1}}],["a∈rd×d",{"2":{"149":1}}],["a=layer",{"2":{"134":1}}],["a×b×x",{"2":{"134":1}}],["aware",{"0":{"163":1},"2":{"116":2,"117":1,"163":4}}],["ah",{"2":{"110":1}}],["ahmadi",{"2":{"53":1}}],["ahmad",{"2":{"53":1}}],["author",{"2":{"74":2}}],["autojacvec=reversediffvjp",{"2":{"217":1}}],["autojacvec=zygotevjp",{"2":{"215":1,"217":1}}],["auto",{"2":{"136":1,"155":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["autoreactant",{"2":{"52":1}}],["autoreversediff",{"2":{"49":1,"52":2}}],["autoenzyme",{"2":{"49":1,"52":1,"107":2}}],["autofinitediff",{"2":{"49":1}}],["autoforwarddiff",{"2":{"27":1,"28":1,"49":1,"146":1,"151":1,"177":2}}],["autotracker",{"2":{"49":1,"52":1}}],["autozygote",{"2":{"27":1,"28":1,"49":1,"52":1,"77":2,"78":1,"150":1,"178":1,"179":1,"187":1,"195":2,"208":1,"217":1,"228":1,"236":2,"244":1,"258":1}}],["autodiff",{"2":{"17":2,"19":2,"81":1,"142":1,"144":1,"145":2,"175":2}}],["autoselection",{"2":{"2":1}}],["automatically",{"2":{"5":1,"7":2,"8":1,"17":3,"19":2,"66":2,"116":1,"136":1,"137":1,"144":1,"159":1,"183":1,"184":2,"185":1,"192":1,"213":1,"249":1}}],["automatic",{"0":{"26":1,"85":1,"131":1,"144":1,"162":1,"165":1,"175":1},"1":{"27":1,"28":1,"29":1,"86":1,"87":1,"88":1,"89":1,"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1,"176":1,"177":1,"178":1},"2":{"2":1,"3":1,"8":1,"29":1,"106":1,"131":2,"132":1,"142":1,"144":2,"153":1,"162":4,"164":1,"170":1,"177":1}}],["affected",{"2":{"149":1}}],["affects",{"2":{"63":1}}],["affine",{"2":{"69":1}}],["affine=false",{"2":{"69":5,"104":1}}],["affine=true",{"2":{"62":2,"69":20,"104":1,"109":1,"110":2}}],["afterwards",{"2":{"69":1}}],["after",{"2":{"17":2,"33":1,"59":1,"62":2,"64":1,"65":3,"69":4,"83":1,"108":1,"113":1,"116":1,"160":1,"179":14,"184":1}}],["abyss",{"2":{"170":1}}],["ability",{"2":{"138":1}}],["ab7d",{"2":{"122":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["able",{"2":{"95":1,"181":1}}],["above",{"2":{"54":1,"62":2,"76":1,"148":1,"169":1,"172":1,"250":1}}],["about",{"2":{"33":1,"57":1,"83":1,"140":1,"171":1,"174":1,"184":1,"243":1,"250":1}}],["abstol",{"2":{"220":13}}],["abstol=1",{"2":{"215":1}}],["abstraction",{"2":{"98":1}}],["abstractions",{"2":{"20":1}}],["abstractexplicitcontainerlayer",{"2":{"95":2,"96":1}}],["abstractexplicitlayer",{"2":{"95":1}}],["abstractrecurrentcell",{"2":{"66":2}}],["abstractrng=utils",{"2":{"24":8,"25":24}}],["abstractrng",{"2":{"7":2,"8":2,"9":1,"10":1,"17":4,"24":3,"134":2,"140":2,"225":1,"232":1}}],["abstracttimeseriesdatabatchordering=batchlastindex",{"2":{"66":2}}],["abstractadtype",{"2":{"27":2,"28":1,"52":1}}],["abstractarrays",{"2":{"49":1}}],["abstractarray",{"2":{"13":2,"16":2,"19":1,"20":1,"24":16,"25":60,"28":1,"54":1,"55":1,"64":3,"67":6,"68":6,"69":1,"123":1,"134":2,"144":3,"148":1,"184":1,"185":1,"208":1,"209":1,"242":3}}],["abstractmatrix",{"2":{"18":2,"68":1,"140":1,"141":1,"214":1}}],["abstractvector",{"2":{"16":1,"18":1,"68":1,"249":1,"254":2}}],["abstractluxdistributedbacked",{"2":{"45":1}}],["abstractluxdistributedbackend",{"2":{"42":3,"43":2,"44":7,"46":1}}],["abstractluxwrapperlayer",{"2":{"7":4,"32":1,"95":1,"96":2,"121":1,"141":1,"214":1,"218":1}}],["abstractluxcontainerlayer",{"2":{"7":5,"95":1,"141":3,"184":2,"241":1}}],["abstractluxlayers",{"2":{"62":1}}],["abstractluxlayer",{"2":{"7":5,"8":2,"31":4,"32":3,"33":1,"39":3,"58":1,"62":4,"69":1,"95":1,"119":2,"122":3,"123":2,"134":1,"140":2,"214":3,"218":2,"225":2}}],["abstract",{"0":{"7":1},"2":{"3":2,"6":1,"7":3,"67":4,"208":1}}],["abstractdevice",{"2":{"2":1,"3":9,"4":4,"5":1}}],["abstractgpudevice",{"2":{"1":1,"132":1}}],["absolute",{"2":{"53":1,"155":1}}],["absolutely",{"2":{"52":1}}],["abs2",{"2":{"49":1,"53":1,"59":3,"145":2,"146":2,"147":2,"148":1,"242":3,"249":1}}],["abs",{"2":{"24":1,"68":1,"245":2,"254":1}}],["amazing",{"2":{"203":1}}],["am",{"2":{"147":1,"256":1}}],["ambiguous",{"2":{"55":1,"95":1}}],["amp",{"0":{"27":1,"154":1,"170":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"27":2,"28":1,"52":1,"63":4,"66":1,"67":1,"68":1,"69":1,"81":2}}],["amd",{"2":{"3":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["amdgpudevice",{"2":{"4":2,"116":1,"132":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["amdgpu",{"2":{"2":2,"3":1,"22":2,"42":1,"72":2,"77":1,"78":1,"80":2,"88":1,"117":1,"130":2,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["average",{"2":{"250":3}}],["averages",{"2":{"44":2,"45":1}}],["avik",{"2":{"74":2}}],["avg",{"2":{"44":2}}],["avoids",{"2":{"218":1}}],["avoiding",{"2":{"146":1}}],["avoid",{"2":{"17":1,"66":1,"83":1,"116":1,"218":1,"220":1,"249":2}}],["available",{"2":{"13":2,"16":1,"18":1,"40":1,"46":1,"76":1,"79":1,"85":1,"106":1,"131":1,"132":1,"137":1,"159":1,"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["aka",{"2":{"16":1,"18":1}}],["academic",{"2":{"74":1}}],["achieved",{"2":{"63":1,"146":1}}],["achieve",{"2":{"24":1,"66":1}}],["acc",{"2":{"187":2,"195":8,"196":2,"217":4,"228":13}}],["accumulated",{"2":{"208":1}}],["accumulates",{"2":{"69":2}}],["accuracy",{"0":{"186":1},"2":{"13":1,"186":2,"187":102,"194":2,"195":4,"196":40,"216":2,"217":94,"219":18,"227":2,"228":212}}],["account",{"2":{"59":1}}],["according",{"2":{"24":1}}],["accordingly",{"2":{"19":2,"69":2}}],["accidental",{"2":{"57":1}}],["accelerate",{"2":{"167":1}}],["accelerating",{"2":{"19":1}}],["acceptance",{"2":{"249":1}}],["accept",{"2":{"142":1,"249":1}}],["accepts",{"2":{"70":1}}],["access",{"2":{"59":1,"62":2,"72":1,"73":1,"140":1,"141":1,"142":1}}],["accessors",{"2":{"34":1}}],["accessing",{"2":{"32":1,"37":1,"39":1}}],["acts",{"2":{"249":1}}],["action",{"2":{"60":1}}],["activates",{"2":{"166":1}}],["activated",{"2":{"66":4}}],["activate",{"2":{"49":1}}],["activation=tanh",{"2":{"66":1}}],["activation=identity",{"2":{"63":2,"67":4,"69":4}}],["activations",{"2":{"13":1,"16":1,"249":1}}],["activation",{"0":{"13":1,"15":1},"2":{"13":7,"15":12,"16":4,"18":3,"19":5,"59":1,"63":6,"66":3,"67":10,"69":12,"102":3,"104":3,"158":7,"166":1,"249":1,"256":3}}],["activity",{"2":{"49":2}}],["actively",{"2":{"93":1,"106":1}}],["active",{"2":{"37":2}}],["act=relu",{"2":{"59":2,"78":1}}],["actually",{"2":{"95":1,"138":1}}],["actual",{"2":{"33":1,"176":2,"236":1}}],["act",{"2":{"13":2,"19":2,"59":8,"78":5,"241":4}}],["across",{"2":{"0":1,"19":2,"44":2,"45":1,"46":1,"82":2,"113":3,"171":2,"250":1}}],["adjust",{"2":{"244":1}}],["adjoint",{"2":{"208":2}}],["adam",{"2":{"59":1,"77":15,"78":1,"107":1,"187":1,"195":1,"208":4,"217":1,"228":1,"234":3,"236":1,"244":1}}],["adapted",{"2":{"252":2}}],["adaptive=false",{"2":{"255":1,"256":1,"257":1,"259":1}}],["adaptivemeanpool",{"2":{"65":1}}],["adaptivemaxpool",{"2":{"65":1}}],["adaptive",{"2":{"65":3}}],["adaptivelppool",{"2":{"65":1,"105":1}}],["adaptor",{"2":{"39":4,"99":1,"193":1}}],["adapt",{"2":{"3":4,"37":6,"39":6}}],["ads",{"2":{"55":1}}],["adtype=autotracker",{"2":{"249":1}}],["adtypes",{"2":{"52":1,"144":1,"182":1,"191":1,"223":1,"231":1,"236":1,"239":1}}],["adtype",{"2":{"49":1,"258":1}}],["advances",{"2":{"17":1}}],["advancedvi",{"2":{"247":1}}],["advanced",{"0":{"201":1},"2":{"7":2,"106":1}}],["ad",{"0":{"28":1,"29":1},"2":{"16":1,"18":1,"27":6,"28":1,"29":1,"33":2,"52":3,"53":1,"55":2,"58":2,"77":2,"82":1,"83":3,"85":4,"86":2,"89":2,"144":4,"147":2,"154":1,"162":1,"170":1,"173":1,"175":2,"176":5,"177":2,"217":1,"236":1,"238":1,"240":2,"242":1,"245":1}}],["adding",{"2":{"117":1}}],["additional",{"0":{"79":1},"2":{"48":1,"49":1,"54":1,"79":1,"80":1,"133":1,"172":1,"177":1,"179":1}}],["additionally",{"2":{"7":1,"52":1,"53":1,"54":2,"85":1,"134":1,"138":1,"149":1,"153":1,"160":1,"188":1,"207":1,"242":1}}],["addition",{"2":{"15":1,"55":1}}],["address",{"2":{"81":1}}],["add",{"2":{"47":1,"55":3,"71":2,"72":5,"73":1,"76":2,"77":1,"79":1,"88":1,"105":1,"145":1,"161":3,"171":1,"172":1,"203":1}}],["added",{"2":{"15":1,"19":4,"53":2,"63":2,"65":3,"68":1,"69":4,"90":1,"105":1}}],["adds",{"2":{"14":1,"33":1,"171":1}}],["at=train",{"2":{"192":1,"213":1}}],["at=0",{"2":{"183":1}}],["atleast",{"2":{"19":1}}],["attempts",{"2":{"14":1,"18":1,"37":1}}],["at",{"2":{"4":2,"17":1,"31":1,"44":2,"53":1,"59":1,"63":2,"65":3,"66":1,"70":1,"77":1,"106":1,"109":4,"110":10,"117":1,"125":1,"130":1,"142":1,"166":2,"177":1,"184":1,"203":2,"244":4,"250":2,"254":2,"255":1}}],["april",{"2":{"74":1}}],["append",{"2":{"248":2}}],["appendix",{"0":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["appreciate",{"2":{"59":1}}],["approach",{"0":{"131":1},"2":{"106":1,"208":1}}],["approximation",{"2":{"68":1}}],["approximately",{"2":{"24":1}}],["approx",{"2":{"49":1}}],["appropriate",{"2":{"8":1,"24":1}}],["applications",{"2":{"83":1,"173":1}}],["applicable",{"2":{"2":1,"4":1}}],["applies",{"2":{"15":1,"19":1,"62":1,"69":6,"70":2,"113":1}}],["applied",{"2":{"8":1,"24":3,"53":2,"55":1,"62":1,"64":4,"69":4,"134":2}}],["applychain",{"2":{"220":3}}],["applying",{"2":{"17":2,"24":1,"62":1}}],["apply",{"0":{"13":1},"2":{"8":7,"52":5,"55":1,"62":1,"63":2,"64":2,"65":3,"77":3,"119":1,"121":1,"122":4,"123":3,"140":1,"141":1,"153":1,"166":1,"236":1,"249":1}}],["appleaccelerate",{"2":{"18":1}}],["apple",{"2":{"3":1,"80":1}}],["api",{"0":{"23":1,"52":1,"107":1,"185":1},"1":{"24":1,"25":1},"2":{"3":1,"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"37":1,"44":3,"52":2,"53":2,"62":9,"68":2,"77":3,"90":1,"92":2,"97":1,"106":2,"107":1,"125":1,"133":1,"179":1,"180":1,"181":1,"185":1,"189":1,"190":1,"204":1,"210":1,"214":1,"218":1,"220":1,"221":1,"229":1,"235":1,"237":1,"246":1,"256":2}}],["astroinformatics",{"2":{"252":1}}],["ask",{"2":{"171":3}}],["asymmetric",{"2":{"63":2,"65":3}}],["assert",{"2":{"174":1,"249":1,"254":5}}],["associated",{"2":{"81":1}}],["assume",{"2":{"106":1,"254":1}}],["assumed",{"2":{"49":1}}],["assumes",{"2":{"11":1,"62":1,"104":1}}],["assigned",{"2":{"59":1}}],["assign",{"2":{"42":1}}],["as",{"2":{"3":2,"4":2,"5":3,"7":4,"13":2,"15":2,"17":2,"19":3,"24":5,"27":2,"33":1,"34":1,"37":1,"39":1,"48":2,"49":2,"50":4,"52":2,"53":8,"54":2,"55":1,"57":2,"59":10,"62":10,"63":1,"65":3,"66":3,"68":3,"69":1,"70":1,"71":1,"74":1,"78":2,"83":2,"87":1,"90":1,"102":2,"103":2,"106":6,"107":1,"110":1,"113":1,"114":1,"116":2,"120":1,"141":3,"142":2,"145":1,"146":1,"147":2,"148":1,"152":1,"153":2,"155":2,"165":1,"169":2,"170":1,"171":1,"173":1,"174":2,"176":1,"185":1,"190":2,"196":1,"204":1,"208":1,"214":1,"220":1,"225":1,"247":1,"249":6,"250":1,"256":2}}],["agent",{"2":{"109":1,"220":3,"244":4}}],["aggressive",{"2":{"98":1}}],["aggregation",{"2":{"53":1}}],["aggregated",{"2":{"53":1}}],["agg=sum",{"2":{"53":2}}],["agg=mean",{"2":{"53":1}}],["agg",{"2":{"53":26}}],["agnostic",{"2":{"3":3,"44":3,"160":1}}],["against",{"2":{"88":1,"149":1}}],["again",{"2":{"3":1,"113":1,"146":1}}],["algebras",{"2":{"177":1}}],["algebraic",{"2":{"146":1}}],["algorithm",{"2":{"2":1,"149":1,"247":1}}],["aligned",{"2":{"70":1}}],["align",{"2":{"70":2,"104":2,"105":1}}],["alias",{"2":{"54":1}}],["aliased",{"2":{"52":3}}],["aliases",{"2":{"15":1}}],["alternate",{"0":{"218":1}}],["alternatives",{"2":{"158":1}}],["alternatively",{"2":{"49":1,"66":2,"70":1,"76":1,"113":1,"155":1}}],["alternative",{"2":{"3":1}}],["alts",{"2":{"68":2}}],["already",{"2":{"60":1,"77":1,"81":1,"174":1,"235":1}}],["almost",{"2":{"37":1}}],["always",{"2":{"27":2,"30":1,"31":1,"37":1,"53":1,"55":1,"104":1,"143":1}}],["al",{"2":{"17":2,"24":2,"53":2,"252":1}}],["alpha=0",{"2":{"232":1,"236":2,"255":2,"256":4,"259":7}}],["alphadropout",{"2":{"64":4}}],["alpha",{"2":{"17":4,"249":2}}],["along",{"2":{"15":1,"19":1,"24":1,"53":1,"54":1,"62":1,"63":1,"64":2,"66":2,"69":1,"83":1,"121":1}}],["also",{"2":{"3":1,"7":1,"15":2,"24":2,"31":1,"42":2,"54":1,"55":2,"57":1,"59":1,"62":2,"64":5,"68":1,"69":7,"70":2,"76":1,"83":2,"90":1,"98":1,"140":2,"168":1,"171":1,"185":1,"187":1,"214":1,"217":2,"249":1,"250":1}}],["allocate",{"2":{"153":1}}],["allow",{"2":{"34":1,"49":1,"55":1,"98":1,"102":1,"155":1}}],["allowscalar",{"2":{"156":1,"212":1,"223":1,"239":1}}],["allows",{"2":{"6":1,"7":1,"8":1,"27":2,"44":2,"55":1,"59":1,"62":1,"66":1,"68":1,"84":1,"105":1,"141":1,"142":1,"217":1}}],["allreduce",{"2":{"44":3,"45":1,"115":1}}],["all",{"2":{"3":1,"6":1,"7":1,"8":1,"10":3,"24":1,"30":1,"31":3,"32":2,"39":2,"44":5,"48":2,"49":1,"52":2,"55":1,"56":1,"58":1,"62":3,"63":2,"65":6,"67":1,"68":2,"70":2,"77":1,"79":2,"82":3,"83":3,"98":1,"100":1,"106":1,"113":3,"116":1,"128":2,"140":1,"160":1,"166":1,"167":1,"169":1,"171":3,"172":1,"173":1,"207":1,"241":1,"248":1,"249":2}}],["artifact",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["artificially",{"2":{"110":1}}],["artificial",{"2":{"24":2,"248":2}}],["arranged",{"2":{"171":1,"248":1}}],["arrayandtime",{"2":{"123":8,"124":1}}],["array",{"2":{"8":1,"13":5,"17":4,"19":5,"20":1,"22":3,"24":9,"28":2,"39":7,"53":1,"55":2,"59":1,"63":5,"65":6,"66":2,"67":4,"68":6,"69":16,"106":1,"107":2,"123":3,"146":1,"147":1,"169":1,"171":8,"172":1,"173":6,"194":1,"206":1,"208":1,"214":3,"215":1,"220":9,"248":4,"249":1,"255":1,"256":1,"257":1,"259":1}}],["arrays",{"0":{"156":1,"171":1,"172":1},"1":{"172":1},"2":{"3":1,"11":1,"16":2,"18":3,"20":7,"55":1,"56":1,"62":1,"67":1,"68":1,"70":5,"106":1,"134":2,"140":1,"153":2,"171":4,"173":1}}],["architectural",{"2":{"140":1}}],["architecture",{"2":{"134":1,"140":2}}],["architectures",{"2":{"83":1}}],["arbitrary",{"2":{"70":1,"217":1}}],["args",{"2":{"48":2,"49":4,"55":2,"57":6,"103":1}}],["argumenterror",{"2":{"244":1}}],["argument",{"2":{"8":2,"25":1,"59":1,"62":4,"66":1,"68":1,"95":1,"104":1,"105":1,"127":1}}],["arguments",{"2":{"2":2,"4":2,"8":1,"13":2,"15":1,"16":1,"17":2,"18":1,"19":4,"24":3,"27":2,"28":1,"31":1,"33":3,"34":1,"37":2,"39":2,"45":1,"48":3,"49":5,"52":3,"58":1,"59":2,"62":11,"63":4,"64":5,"65":9,"66":7,"67":8,"68":7,"69":9,"70":2,"92":1,"140":1,"169":1,"220":3}}],["arxiv",{"2":{"19":4,"24":1,"68":1,"69":2}}],["around",{"2":{"14":1,"27":2,"37":1,"63":2,"65":3,"83":1,"141":1,"148":1}}],["aren",{"2":{"202":1}}],["are",{"2":{"2":1,"3":2,"6":1,"7":2,"8":1,"11":1,"13":1,"18":1,"19":3,"20":1,"24":4,"28":2,"30":2,"31":6,"33":4,"34":1,"39":2,"40":1,"42":2,"49":2,"52":6,"53":6,"54":2,"55":1,"57":2,"59":2,"60":1,"67":3,"68":2,"69":5,"70":2,"76":1,"78":1,"79":1,"82":2,"83":2,"87":3,"88":5,"89":2,"95":1,"103":1,"105":1,"106":2,"107":1,"108":1,"110":1,"113":1,"116":1,"117":1,"122":1,"126":1,"130":1,"140":3,"141":2,"142":2,"143":1,"144":2,"145":2,"146":1,"149":1,"152":3,"153":1,"155":2,"158":1,"159":2,"160":1,"163":2,"164":1,"166":1,"167":4,"171":1,"173":1,"175":1,"184":2,"202":2,"203":2,"208":1,"214":1,"225":1,"249":1,"250":1,"255":1,"256":1}}],["animation",{"2":{"250":1}}],["analytical",{"2":{"243":4,"245":1}}],["anticlockwise",{"2":{"181":1,"183":4}}],["anonymous",{"2":{"102":1}}],["another",{"2":{"67":1,"141":1}}],["answers",{"2":{"84":1}}],["anything",{"2":{"140":1}}],["anywhere",{"2":{"95":1}}],["any",{"2":{"8":2,"11":1,"16":1,"18":1,"30":1,"33":1,"39":1,"49":1,"52":2,"53":3,"55":2,"59":2,"62":1,"66":3,"68":1,"82":1,"89":1,"98":2,"106":3,"107":1,"109":1,"111":1,"116":1,"122":7,"134":1,"138":2,"140":2,"142":1,"144":1,"153":1,"167":1,"172":1,"173":2,"174":1,"175":1,"188":1,"203":1,"217":1,"220":15,"226":1,"235":1}}],["an",{"2":{"2":5,"3":2,"4":1,"5":1,"8":3,"13":1,"17":4,"19":3,"24":19,"25":36,"27":2,"33":1,"41":2,"44":1,"45":1,"50":3,"52":2,"53":2,"55":1,"57":1,"59":1,"62":10,"63":3,"64":3,"65":3,"66":2,"67":8,"68":2,"69":7,"77":1,"83":1,"85":1,"87":1,"89":1,"95":1,"102":1,"104":2,"105":1,"106":1,"107":1,"108":1,"110":1,"111":2,"117":1,"120":1,"131":1,"144":1,"145":2,"146":1,"148":1,"155":1,"159":1,"161":2,"171":3,"174":1,"190":1,"202":1,"203":3,"208":1,"248":1,"249":1,"250":1}}],["andrea",{"2":{"19":1,"69":1}}],["and",{"0":{"35":1,"160":1,"186":1,"215":1,"222":1,"226":1},"1":{"36":1,"37":1,"38":1,"39":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"1":1,"2":10,"3":3,"4":7,"5":4,"7":7,"8":7,"13":1,"14":1,"15":1,"16":4,"17":2,"18":3,"19":20,"24":13,"28":1,"30":2,"31":4,"32":7,"33":5,"34":3,"37":5,"39":2,"42":4,"44":2,"46":2,"47":2,"48":2,"50":2,"52":8,"53":19,"54":2,"55":6,"56":1,"57":4,"58":1,"59":5,"60":1,"62":10,"63":9,"65":3,"66":31,"67":8,"68":3,"69":22,"70":8,"76":1,"77":4,"78":5,"81":3,"82":2,"83":6,"84":2,"87":4,"88":2,"95":5,"100":2,"102":5,"103":3,"104":3,"105":4,"106":6,"107":2,"109":3,"110":6,"111":3,"113":6,"114":2,"116":1,"117":1,"124":1,"126":1,"130":1,"131":3,"132":1,"134":10,"136":1,"138":2,"140":9,"141":8,"142":3,"143":1,"144":2,"145":5,"147":1,"148":1,"149":2,"150":2,"152":1,"153":5,"155":3,"158":2,"159":1,"160":5,"164":2,"166":2,"167":4,"169":2,"171":7,"172":2,"173":1,"174":3,"175":3,"176":1,"178":1,"179":3,"181":2,"183":3,"184":10,"185":1,"188":3,"192":1,"193":1,"196":1,"202":1,"203":3,"208":3,"213":1,"214":2,"217":2,"220":1,"232":1,"235":2,"236":1,"238":1,"241":1,"243":2,"248":1,"249":12,"250":3,"254":3,"255":2,"256":6}}],["a",{"0":{"38":1,"126":1,"129":1,"181":1,"184":1,"222":1,"225":1,"230":1,"238":1,"252":1,"256":1},"1":{"39":1,"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"0":1,"1":2,"2":5,"3":5,"4":5,"5":3,"6":1,"7":10,"8":12,"10":1,"11":4,"14":1,"15":3,"16":2,"17":4,"18":2,"19":2,"21":1,"24":20,"25":13,"27":2,"28":4,"30":1,"31":4,"32":1,"33":3,"34":4,"37":9,"39":9,"42":2,"44":3,"45":1,"47":2,"48":1,"49":4,"50":5,"52":5,"53":4,"54":5,"55":5,"57":3,"58":4,"59":21,"62":27,"63":21,"65":18,"66":43,"67":8,"68":10,"69":20,"70":6,"72":1,"73":1,"76":1,"77":2,"78":3,"81":2,"82":1,"83":4,"84":1,"90":2,"95":9,"96":2,"97":2,"100":3,"106":4,"107":3,"108":1,"109":3,"110":5,"112":2,"113":1,"116":1,"117":1,"119":2,"120":1,"123":3,"124":1,"125":1,"126":1,"128":2,"130":1,"131":7,"134":17,"136":1,"137":1,"138":3,"140":8,"141":8,"142":2,"144":6,"145":7,"146":2,"147":3,"148":1,"149":10,"152":1,"153":4,"155":5,"160":1,"163":1,"164":1,"168":2,"169":2,"170":3,"171":12,"172":1,"174":3,"175":12,"176":1,"177":2,"179":4,"181":1,"183":2,"184":3,"185":1,"190":2,"196":1,"202":1,"204":3,"207":1,"208":6,"214":3,"217":2,"218":1,"219":1,"226":1,"230":2,"233":1,"236":2,"238":2,"241":3,"242":1,"243":1,"247":1,"248":3,"249":7,"250":5,"254":9,"256":3,"257":1,"258":1}}]],"serializationVersion":2}';export{e as default}; diff --git a/dev/assets/chunks/@localSearchIndexroot.CZmaQmit.js b/dev/assets/chunks/@localSearchIndexroot.CZmaQmit.js new file mode 100644 index 0000000000..f9033aa4ee --- /dev/null +++ b/dev/assets/chunks/@localSearchIndexroot.CZmaQmit.js @@ -0,0 +1 @@ +const e='{"documentCount":261,"nextId":261,"documentIds":{"0":"/dev/api/Accelerator_Support/MLDataDevices#MLDataDevices-API","1":"/dev/api/Accelerator_Support/MLDataDevices#preferences","2":"/dev/api/Accelerator_Support/MLDataDevices#Data-Transfer","3":"/dev/api/Accelerator_Support/MLDataDevices#miscellaneous","4":"/dev/api/Accelerator_Support/MLDataDevices#Multi-GPU-Support","5":"/dev/api/Accelerator_Support/MLDataDevices#iteration","6":"/dev/api/Building_Blocks/LuxCore#luxcore","7":"/dev/api/Building_Blocks/LuxCore#Abstract-Types","8":"/dev/api/Building_Blocks/LuxCore#general","9":"/dev/api/Building_Blocks/LuxCore#parameters","10":"/dev/api/Building_Blocks/LuxCore#states","11":"/dev/api/Building_Blocks/LuxCore#Layer-size","12":"/dev/api/Building_Blocks/LuxLib#LuxLib-API","13":"/dev/api/Building_Blocks/LuxLib#Apply-Activation","14":"/dev/api/Building_Blocks/LuxLib#Batched-Operations","15":"/dev/api/Building_Blocks/LuxLib#Bias-Activation","16":"/dev/api/Building_Blocks/LuxLib#Convolutional-Layers","17":"/dev/api/Building_Blocks/LuxLib#dropout","18":"/dev/api/Building_Blocks/LuxLib#Fully-Connected-Layers","19":"/dev/api/Building_Blocks/LuxLib#normalization","20":"/dev/api/Building_Blocks/LuxLib#Helper-Functions","21":"/dev/api/Lux/autodiff#autodiff-lux-helpers","22":"/dev/api/Lux/autodiff#JVP-and-VJP-Wrappers","23":"/dev/api/Lux/autodiff#Batched-AD","24":"/dev/api/Lux/autodiff#Nested-2nd-Order-AD","25":"/dev/api/Building_Blocks/WeightInitializers#WeightInitializers-API","26":"/dev/api/Building_Blocks/WeightInitializers#Supported-RNG-Types-WeightInit","27":"/dev/api/Building_Blocks/WeightInitializers#API-Reference","28":"/dev/api/Building_Blocks/WeightInitializers#Main-Functions","29":"/dev/api/Building_Blocks/WeightInitializers#Other-Convenience-Functions","30":"/dev/api/Lux/distributed_utils#Distributed-Utils","31":"/dev/api/Lux/distributed_utils#communication-backends","32":"/dev/api/Lux/distributed_utils#initialization","33":"/dev/api/Lux/distributed_utils#Helper-Functions","34":"/dev/api/Lux/distributed_utils#Communication-Primitives","35":"/dev/api/Lux/distributed_utils#Optimizers.jl-Integration","36":"/dev/api/Lux/distributed_utils#MLUtils.jl-Integration","37":"/dev/api/Lux/contrib#Experimental-Features","38":"/dev/api/Lux/contrib#Parameter-Freezing","39":"/dev/api/Lux/contrib#Map-over-Layer","40":"/dev/api/Lux/contrib#Debugging-Functionality","41":"/dev/api/Lux/contrib#Tied-Parameters","42":"/dev/api/Lux/interop#Interoperability-between-Lux-and-other-packages","43":"/dev/api/Lux/interop#Switching-from-older-frameworks","44":"/dev/api/Lux/interop#flux-to-lux-migrate-api","45":"/dev/api/Lux/interop#Using-a-different-backend-for-Lux","46":"/dev/api/Lux/interop#Lux-Models-to-Simple-Chains","47":"/dev/api/Lux/layers#Built-In-Layers","48":"/dev/api/Lux/layers#containers","49":"/dev/api/Lux/layers#Convolutional-Layers","50":"/dev/api/Lux/layers#Dropout-Layers","51":"/dev/api/Lux/layers#Pooling-Layers","52":"/dev/api/Lux/layers#Recurrent-Layers","53":"/dev/api/Lux/layers#Linear-Layers","54":"/dev/api/Lux/layers#Misc.-Helper-Layers","55":"/dev/api/Lux/layers#Normalization-Layers","56":"/dev/api/Lux/layers#upsampling","57":"/dev/api/Testing_Functionality/LuxTestUtils#luxtestutils","58":"/dev/api/Testing_Functionality/LuxTestUtils#Testing-using-JET.jl","59":"/dev/api/Testing_Functionality/LuxTestUtils#Gradient-Correctness","60":"/dev/api/Testing_Functionality/LuxTestUtils#Extensions-to-@test","61":"/dev/api/Lux/utilities#utilities","62":"/dev/api/Lux/utilities#Training-API","63":"/dev/api/Lux/utilities#Loss-Functions","64":"/dev/api/Lux/utilities#LuxOps-Module","65":"/dev/api/Lux/utilities#Recursive-Operations","66":"/dev/api/Lux/utilities#Updating-Floating-Point-Precision","67":"/dev/api/Lux/utilities#Element-Type-Matching","68":"/dev/api/Lux/utilities#Stateful-Layer","69":"/dev/api/Lux/utilities#Compact-Layer","70":"/dev/api/Lux/utilities#miscellaneous","71":"/dev/#How-to-Install-Lux.jl?","72":"/dev/#Want-GPU-Support?","73":"/dev/#Want-XLA-Support?","74":"/dev/introduction/citation#citation","75":"/dev/introduction/#getting-started","76":"/dev/introduction/#installation","77":"/dev/introduction/#quickstart","78":"/dev/introduction/#Defining-Custom-Layers","79":"/dev/introduction/#Additional-Packages","80":"/dev/introduction/#GPU-Support","81":"/dev/introduction/overview#Why-we-wrote-Lux?","82":"/dev/introduction/overview#Design-Principles","83":"/dev/introduction/overview#Why-use-Lux-over-Flux?","84":"/dev/introduction/resources#Resources-to-Get-Started","85":"/dev/introduction/updating_to_v1#updating-to-v1","86":"/dev/introduction/updating_to_v1#LuxLib.jl","87":"/dev/introduction/updating_to_v1#Breaking-Changes","88":"/dev/introduction/updating_to_v1#New-Major-Features","89":"/dev/introduction/updating_to_v1#LuxCore.jl","90":"/dev/introduction/updating_to_v1#Breaking-Changes-2","91":"/dev/introduction/updating_to_v1#New-Major-Features-2","92":"/dev/introduction/updating_to_v1#WeightInitializers.jl","93":"/dev/introduction/updating_to_v1#MLDataDevices.jl","94":"/dev/introduction/updating_to_v1#Breaking-Changes-3","95":"/dev/introduction/updating_to_v1#New-Major-Features-3","96":"/dev/introduction/updating_to_v1#Lux.jl","97":"/dev/introduction/updating_to_v1#Breaking-Changes-(Removed-Functionality)","98":"/dev/introduction/updating_to_v1#Breaking-Changes-(Moved-Functionality)","99":"/dev/introduction/updating_to_v1#Breaking-Changes-(Changes-in-Defaults)","100":"/dev/introduction/updating_to_v1#New-Features","101":"/dev/manual/autodiff#autodiff-lux","102":"/dev/manual/autodiff#overview","103":"/dev/manual/autodiff#autodiff-recommendations","104":"/dev/manual/autodiff#Support-Class","105":"/dev/manual/autodiff#footnotes","106":"/dev/manual/compiling_lux_models#reactant-compilation","107":"/dev/manual/compiling_lux_models#compile_lux_model_trainstate","108":"/dev/manual/debugging#debug-lux-layers","109":"/dev/manual/debugging#Incorrect-Model-Specification:-Dimension-Mismatch-Problems","110":"/dev/manual/debugging#Tracking-down-NaNs","111":"/dev/manual/debugging#conclusion","112":"/dev/manual/dispatch_custom_input#Dispatching-on-Custom-Input-Types","113":"/dev/manual/dispatch_custom_input#Which-function-should-participate-in-dispatch?","114":"/dev/manual/dispatch_custom_input#Concrete-Example","115":"/dev/manual/dispatch_custom_input#Time-Dependent-Chain-Implementation","116":"/dev/manual/dispatch_custom_input#Running-the-TDChain","117":"/dev/manual/dispatch_custom_input#Writing-the-Correct-Dispatch-Rules","118":"/dev/manual/dispatch_custom_input#Using-the-Same-Input-for-Non-TD-Models","119":"/dev/manual/distributed_utils#Distributed-Data-Parallel-Training","120":"/dev/manual/distributed_utils#Guide-to-Integrating-DistributedUtils-into-your-code","121":"/dev/manual/distributed_utils#Migration-Guide-from-FluxMPI.jl","122":"/dev/manual/distributed_utils#Removed-Functionality","123":"/dev/manual/distributed_utils#Key-Differences","124":"/dev/manual/distributed_utils#Known-Shortcomings","125":"/dev/manual/freezing_model_parameters#freezing-model-parameters","126":"/dev/manual/freezing_model_parameters#Freezing-Layers-of-a-Particular-Kind","127":"/dev/manual/freezing_model_parameters#Freezing-by-Layer-Name","128":"/dev/manual/freezing_model_parameters#Freezing-Part-of-the-Parameters","129":"/dev/manual/freezing_model_parameters#Freezing-Part-of-a-Chain","130":"/dev/manual/gpu_management#GPU-Management","131":"/dev/manual/gpu_management#Automatic-Backend-Management-(Recommended-Approach)","132":"/dev/manual/gpu_management#Manual-Backend-Management","133":"/dev/manual/interface#lux-interface","134":"/dev/manual/interface#Layer-Interface","135":"/dev/manual/interface#Singular-Layer","136":"/dev/manual/interface#Container-Layer","137":"/dev/manual/interface#Parameter-Interface","138":"/dev/manual/interface#State-Interface","139":"/dev/manual/migrate_from_flux#migrate-from-flux","140":"/dev/manual/migrate_from_flux#Implementing-Custom-Layers","141":"/dev/manual/migrate_from_flux#Certain-Important-Implementation-Details","142":"/dev/manual/migrate_from_flux#Training/Inference-Mode","143":"/dev/manual/migrate_from_flux#Can-we-still-use-Flux-Layers?","144":"/dev/manual/nested_autodiff#nested_autodiff","145":"/dev/manual/nested_autodiff#Loss-Function-containing-Jacobian-Computation","146":"/dev/manual/nested_autodiff#Using-Batched-Jacobian-for-Multiple-Inputs","147":"/dev/manual/nested_autodiff#Loss-Function-contains-Gradient-Computation","148":"/dev/manual/nested_autodiff#Loss-Function-computing-the-Jacobian-of-the-Parameters","149":"/dev/manual/nested_autodiff#Hutchinson-Trace-Estimation","150":"/dev/manual/nested_autodiff#Computing-using-the-Vector-Jacobian-Product","151":"/dev/manual/nested_autodiff#Computing-using-the-Jacobian-Vector-Product","152":"/dev/manual/nested_autodiff#Computing-using-the-Full-Jacobian","153":"/dev/manual/nn_inside_gpu_kernels#Neural-Networks-Inside-GPU-Kernels","154":"/dev/manual/performance_pitfalls#Performance-Pitfalls-and-How-to-Catch-Them","155":"/dev/manual/performance_pitfalls#Spurious-Type-Promotion","156":"/dev/manual/performance_pitfalls#Scalar-Indexing-on-GPU-Arrays","157":"/dev/manual/performance_pitfalls#Type-Instabilities","158":"/dev/manual/performance_pitfalls#Faster-Primitives","159":"/dev/manual/performance_pitfalls#Optional-Dependencies-for-Performance","160":"/dev/manual/performance_pitfalls#Data-Loading-and-Device-Transfer","161":"/dev/manual/preferences#Preferences-for-Lux.jl","162":"/dev/manual/preferences#Nested-Automatic-Differentiation","163":"/dev/manual/preferences#gpu-aware-mpi-preferences","164":"/dev/manual/preferences#GPU-Backend-Selection","165":"/dev/manual/preferences#automatic-eltypes-preference","166":"/dev/manual/preferences#dispatch-doctor-preference","167":"/dev/manual/preferences#disable_loop_vectorization","168":"/dev/manual/weight_initializers#Initializing-Weights","169":"/dev/manual/weight_initializers#Quick-examples","170":"/dev/tutorials/beginner/1_Basics#Julia-and-Lux-for-the-Uninitiated","171":"/dev/tutorials/beginner/1_Basics#arrays","172":"/dev/tutorials/beginner/1_Basics#CUDA-Arrays","173":"/dev/tutorials/beginner/1_Basics#im-mutability","174":"/dev/tutorials/beginner/1_Basics#Managing-Randomness","175":"/dev/tutorials/beginner/1_Basics#Automatic-Differentiation","176":"/dev/tutorials/beginner/1_Basics#gradients","177":"/dev/tutorials/beginner/1_Basics#Jacobian-Vector-Product","178":"/dev/tutorials/beginner/1_Basics#Vector-Jacobian-Product","179":"/dev/tutorials/beginner/1_Basics#Linear-Regression","180":"/dev/tutorials/beginner/1_Basics#appendix","181":"/dev/tutorials/beginner/3_SimpleRNN#Training-a-Simple-LSTM","182":"/dev/tutorials/beginner/3_SimpleRNN#Package-Imports","183":"/dev/tutorials/beginner/3_SimpleRNN#dataset","184":"/dev/tutorials/beginner/3_SimpleRNN#Creating-a-Classifier","185":"/dev/tutorials/beginner/3_SimpleRNN#Using-the-@compact-API","186":"/dev/tutorials/beginner/3_SimpleRNN#Defining-Accuracy,-Loss-and-Optimiser","187":"/dev/tutorials/beginner/3_SimpleRNN#Training-the-Model","188":"/dev/tutorials/beginner/3_SimpleRNN#Saving-the-Model","189":"/dev/tutorials/beginner/3_SimpleRNN#appendix","190":"/dev/tutorials/beginner/4_SimpleChains#MNIST-Classification-with-SimpleChains","191":"/dev/tutorials/beginner/4_SimpleChains#Package-Imports","192":"/dev/tutorials/beginner/4_SimpleChains#Loading-MNIST","193":"/dev/tutorials/beginner/4_SimpleChains#Define-the-Model","194":"/dev/tutorials/beginner/4_SimpleChains#Helper-Functions","195":"/dev/tutorials/beginner/4_SimpleChains#Define-the-Training-Loop","196":"/dev/tutorials/beginner/4_SimpleChains#Finally-Training-the-Model","197":"/dev/tutorials/beginner/4_SimpleChains#appendix","198":"/dev/tutorials/beginner/5_OptimizationIntegration#Optimization-Lux-Tutorial","199":"/dev/tutorials/beginner/5_OptimizationIntegration#Imports-packages","200":"/dev/tutorials/beginner/5_OptimizationIntegration#Generate-some-training-data","201":"/dev/tutorials/beginner/5_OptimizationIntegration#Define-the-DataLoader","202":"/dev/tutorials/beginner/5_OptimizationIntegration#Training-the-model","203":"/dev/tutorials/beginner/5_OptimizationIntegration#Plotting-the-results","204":"/dev/tutorials/beginner/5_OptimizationIntegration#appendix","205":"/dev/tutorials/#tutorials","206":"/dev/tutorials/#beginner-tutorials","207":"/dev/tutorials/#intermediate-tutorials","208":"/dev/tutorials/#advanced-tutorials","209":"/dev/tutorials/#larger-models","210":"/dev/tutorials/#selected-3rd-party-tutorials","211":"/dev/tutorials/intermediate/1_NeuralODE#MNIST-Classification-using-Neural-ODEs","212":"/dev/tutorials/intermediate/1_NeuralODE#Package-Imports","213":"/dev/tutorials/intermediate/1_NeuralODE#Loading-MNIST","214":"/dev/tutorials/intermediate/1_NeuralODE#Define-the-Neural-ODE-Layer","215":"/dev/tutorials/intermediate/1_NeuralODE#Create-and-Initialize-the-Neural-ODE-Layer","216":"/dev/tutorials/intermediate/1_NeuralODE#Define-Utility-Functions","217":"/dev/tutorials/intermediate/1_NeuralODE#training","218":"/dev/tutorials/intermediate/1_NeuralODE#Alternate-Implementation-using-Stateful-Layer","219":"/dev/tutorials/intermediate/1_NeuralODE#Train-the-new-Stateful-Neural-ODE","220":"/dev/tutorials/intermediate/1_NeuralODE#Type-Stability","221":"/dev/tutorials/intermediate/1_NeuralODE#appendix","222":"/dev/tutorials/intermediate/3_HyperNet#Training-a-HyperNetwork-on-MNIST-and-FashionMNIST","223":"/dev/tutorials/intermediate/3_HyperNet#Package-Imports","224":"/dev/tutorials/intermediate/3_HyperNet#Loading-Datasets","225":"/dev/tutorials/intermediate/3_HyperNet#Implement-a-HyperNet-Layer","226":"/dev/tutorials/intermediate/3_HyperNet#Create-and-Initialize-the-HyperNet","227":"/dev/tutorials/intermediate/3_HyperNet#Define-Utility-Functions","228":"/dev/tutorials/intermediate/3_HyperNet#training","229":"/dev/tutorials/intermediate/3_HyperNet#appendix","230":"/dev/tutorials/intermediate/4_PINN2DPDE#Training-a-PINN-on-2D-PDE","231":"/dev/tutorials/intermediate/4_PINN2DPDE#Package-Imports","232":"/dev/tutorials/intermediate/4_PINN2DPDE#Problem-Definition","233":"/dev/tutorials/intermediate/4_PINN2DPDE#Define-the-Neural-Networks","234":"/dev/tutorials/intermediate/4_PINN2DPDE#Define-the-Loss-Functions","235":"/dev/tutorials/intermediate/4_PINN2DPDE#Generate-the-Data","236":"/dev/tutorials/intermediate/4_PINN2DPDE#training","237":"/dev/tutorials/intermediate/4_PINN2DPDE#Visualizing-the-Results","238":"/dev/tutorials/intermediate/4_PINN2DPDE#appendix","239":"/dev/tutorials/beginner/2_PolynomialFitting#Fitting-a-Polynomial-using-MLP","240":"/dev/tutorials/beginner/2_PolynomialFitting#Package-Imports","241":"/dev/tutorials/beginner/2_PolynomialFitting#dataset","242":"/dev/tutorials/beginner/2_PolynomialFitting#Neural-Network","243":"/dev/tutorials/beginner/2_PolynomialFitting#optimizer","244":"/dev/tutorials/beginner/2_PolynomialFitting#Loss-Function","245":"/dev/tutorials/beginner/2_PolynomialFitting#training","246":"/dev/tutorials/beginner/2_PolynomialFitting#appendix","247":"/dev/tutorials/intermediate/2_BayesianNN#Bayesian-Neural-Network","248":"/dev/tutorials/intermediate/2_BayesianNN#Generating-data","249":"/dev/tutorials/intermediate/2_BayesianNN#Building-the-Neural-Network","250":"/dev/tutorials/intermediate/2_BayesianNN#Prediction-Visualization","251":"/dev/tutorials/intermediate/2_BayesianNN#appendix","252":"/dev/tutorials/advanced/1_GravitationalWaveForm#Training-a-Neural-ODE-to-Model-Gravitational-Waveforms","253":"/dev/tutorials/advanced/1_GravitationalWaveForm#Package-Imports","254":"/dev/tutorials/advanced/1_GravitationalWaveForm#Define-some-Utility-Functions","255":"/dev/tutorials/advanced/1_GravitationalWaveForm#Simulating-the-True-Model","256":"/dev/tutorials/advanced/1_GravitationalWaveForm#Defiing-a-Neural-Network-Model","257":"/dev/tutorials/advanced/1_GravitationalWaveForm#Setting-Up-for-Training-the-Neural-Network","258":"/dev/tutorials/advanced/1_GravitationalWaveForm#Training-the-Neural-Network","259":"/dev/tutorials/advanced/1_GravitationalWaveForm#Visualizing-the-Results","260":"/dev/tutorials/advanced/1_GravitationalWaveForm#appendix"},"fieldIds":{"title":0,"titles":1,"text":2},"fieldLength":{"0":[1,1,14],"1":[1,1,46],"2":[2,1,156],"3":[1,1,194],"4":[3,1,93],"5":[1,1,114],"6":[1,1,38],"7":[2,1,160],"8":[1,1,214],"9":[1,1,19],"10":[1,1,62],"11":[2,1,77],"12":[1,1,5],"13":[2,1,100],"14":[2,1,44],"15":[2,1,68],"16":[2,1,128],"17":[1,1,160],"18":[3,1,132],"19":[1,1,232],"20":[2,1,91],"21":[3,1,1],"22":[4,3,72],"23":[2,3,86],"24":[4,3,13],"25":[1,1,16],"26":[3,1,27],"27":[2,1,1],"28":[2,3,342],"29":[3,3,92],"30":[2,1,11],"31":[1,2,29],"32":[1,2,81],"33":[2,2,18],"34":[2,2,58],"35":[3,2,31],"36":[3,2,35],"37":[2,1,54],"38":[2,2,131],"39":[3,2,124],"40":[2,2,140],"41":[2,2,118],"42":[6,1,1],"43":[4,6,1],"44":[4,10,172],"45":[6,6,1],"46":[5,11,188],"47":[3,1,1],"48":[1,3,292],"49":[2,3,257],"50":[2,3,115],"51":[2,3,168],"52":[2,3,361],"53":[2,3,191],"54":[3,3,243],"55":[2,3,294],"56":[1,3,165],"57":[1,1,48],"58":[4,1,91],"59":[2,1,127],"60":[3,1,33],"61":[1,1,1],"62":[2,1,228],"63":[2,1,449],"64":[2,1,134],"65":[2,1,164],"66":[4,1,45],"67":[3,1,108],"68":[2,1,112],"69":[2,1,358],"70":[1,1,54],"71":[6,1,47],"72":[4,1,34],"73":[4,1,29],"74":[1,1,67],"75":[2,1,1],"76":[1,2,51],"77":[1,2,402],"78":[3,2,181],"79":[2,2,36],"80":[2,2,17],"81":[5,1,63],"82":[2,5,58],"83":[6,5,218],"84":[4,1,48],"85":[4,1,41],"86":[2,4,1],"87":[2,6,27],"88":[3,6,22],"89":[2,4,1],"90":[2,6,119],"91":[3,6,19],"92":[2,4,30],"93":[2,4,48],"94":[2,6,22],"95":[3,6,30],"96":[2,4,1],"97":[5,5,117],"98":[5,5,54],"99":[5,5,68],"100":[2,5,63],"101":[2,1,35],"102":[1,2,34],"103":[1,2,50],"104":[2,2,65],"105":[1,2,67],"106":[6,1,493],"107":[4,6,156],"108":[3,1,76],"109":[6,3,156],"110":[3,3,190],"111":[1,3,51],"112":[5,1,1],"113":[7,5,36],"114":[2,5,40],"115":[4,7,72],"116":[3,7,73],"117":[5,7,47],"118":[8,7,46],"119":[4,1,27],"120":[7,4,122],"121":[5,4,42],"122":[2,9,28],"123":[2,9,79],"124":[2,4,52],"125":[3,1,27],"126":[6,3,123],"127":[4,3,53],"128":[5,3,43],"129":[5,3,49],"130":[2,1,71],"131":[6,2,89],"132":[3,2,44],"133":[2,1,80],"134":[2,2,1],"135":[2,3,239],"136":[2,3,158],"137":[2,2,151],"138":[2,2,47],"139":[5,1,67],"140":[3,5,164],"141":[4,5,1],"142":[3,9,49],"143":[7,5,34],"144":[3,1,169],"145":[5,3,265],"146":[6,8,155],"147":[5,3,219],"148":[7,3,216],"149":[3,3,88],"150":[6,6,46],"151":[6,6,31],"152":[5,6,113],"153":[5,1,286],"154":[7,1,18],"155":[3,7,149],"156":[5,7,32],"157":[2,7,39],"158":[2,7,43],"159":[4,7,39],"160":[5,7,75],"161":[4,1,49],"162":[3,4,21],"163":[4,4,40],"164":[3,4,33],"165":[3,4,23],"166":[2,4,61],"167":[4,4,58],"168":[2,1,133],"169":[2,2,64],"170":[6,1,77],"171":[1,6,286],"172":[2,7,86],"173":[3,6,110],"174":[2,6,130],"175":[2,6,96],"176":[1,8,72],"177":[3,8,106],"178":[3,8,30],"179":[2,6,253],"180":[1,6,150],"181":[4,1,38],"182":[2,4,12],"183":[1,4,95],"184":[3,4,160],"185":[4,4,74],"186":[5,4,56],"187":[3,4,540],"188":[3,4,48],"189":[1,4,150],"190":[4,1,33],"191":[2,4,15],"192":[2,4,60],"193":[3,4,64],"194":[2,4,38],"195":[4,4,73],"196":[4,4,132],"197":[1,4,110],"198":[6,1,78],"199":[2,6,29],"200":[4,6,58],"201":[3,6,70],"202":[3,6,244],"203":[3,6,47],"204":[1,6,150],"205":[1,1,1],"206":[2,1,1],"207":[2,1,1],"208":[2,1,1],"209":[2,1,41],"210":[4,1,65],"211":[5,1,24],"212":[2,5,23],"213":[2,5,64],"214":[5,5,139],"215":[7,5,60],"216":[3,5,37],"217":[1,5,235],"218":[5,5,60],"219":[6,5,71],"220":[2,5,180],"221":[1,5,150],"222":[7,1,1],"223":[2,7,18],"224":[2,7,41],"225":[4,7,76],"226":[5,7,42],"227":[3,7,39],"228":[1,7,223],"229":[1,7,148],"230":[6,1,45],"231":[2,6,31],"232":[2,6,26],"233":[4,6,55],"234":[4,6,71],"235":[3,6,86],"236":[1,6,1041],"237":[3,6,73],"238":[1,6,149],"239":[5,1,16],"240":[2,5,12],"241":[1,5,274],"242":[2,5,37],"243":[1,5,18],"244":[2,5,112],"245":[1,5,150],"246":[1,5,150],"247":[3,1,72],"248":[2,3,105],"249":[4,3,548],"250":[2,3,174],"251":[1,3,103],"252":[8,1,21],"253":[2,8,13],"254":[4,8,202],"255":[4,8,124],"256":[5,8,1354],"257":[7,8,73],"258":[4,8,1251],"259":[3,8,83],"260":[1,8,103]},"averageFieldLength":[2.954022988505747,3.6091954022988504,107.35632183908046],"storedFields":{"0":{"title":"MLDataDevices","titles":[]},"1":{"title":"Preferences","titles":["MLDataDevices"]},"2":{"title":"Data Transfer","titles":["MLDataDevices"]},"3":{"title":"Miscellaneous","titles":["MLDataDevices"]},"4":{"title":"Multi-GPU Support","titles":["MLDataDevices"]},"5":{"title":"Iteration","titles":["MLDataDevices"]},"6":{"title":"LuxCore","titles":[]},"7":{"title":"Abstract Types","titles":["LuxCore"]},"8":{"title":"General","titles":["LuxCore"]},"9":{"title":"Parameters","titles":["LuxCore"]},"10":{"title":"States","titles":["LuxCore"]},"11":{"title":"Layer size","titles":["LuxCore"]},"12":{"title":"LuxLib","titles":[]},"13":{"title":"Apply Activation","titles":["LuxLib"]},"14":{"title":"Batched Operations","titles":["LuxLib"]},"15":{"title":"Bias Activation","titles":["LuxLib"]},"16":{"title":"Convolutional Layers","titles":["LuxLib"]},"17":{"title":"Dropout","titles":["LuxLib"]},"18":{"title":"Fully Connected Layers","titles":["LuxLib"]},"19":{"title":"Normalization","titles":["LuxLib"]},"20":{"title":"Helper Functions","titles":["LuxLib"]},"21":{"title":"Automatic Differentiation Helpers","titles":[]},"22":{"title":"JVP & VJP Wrappers","titles":["Automatic Differentiation Helpers"]},"23":{"title":"Batched AD","titles":["Automatic Differentiation Helpers"]},"24":{"title":"Nested 2nd Order AD","titles":["Automatic Differentiation Helpers"]},"25":{"title":"WeightInitializers","titles":[]},"26":{"title":"Supported RNG Types","titles":["WeightInitializers"]},"27":{"title":"API Reference","titles":["WeightInitializers"]},"28":{"title":"Main Functions","titles":["WeightInitializers","API Reference"]},"29":{"title":"Other Convenience Functions","titles":["WeightInitializers","API Reference"]},"30":{"title":"Distributed Utils","titles":[]},"31":{"title":"Backends","titles":["Distributed Utils"]},"32":{"title":"Initialization","titles":["Distributed Utils"]},"33":{"title":"Helper Functions","titles":["Distributed Utils"]},"34":{"title":"Communication Primitives","titles":["Distributed Utils"]},"35":{"title":"Optimizers.jl Integration","titles":["Distributed Utils"]},"36":{"title":"MLUtils.jl Integration","titles":["Distributed Utils"]},"37":{"title":"Experimental Features","titles":[]},"38":{"title":"Parameter Freezing","titles":["Experimental Features"]},"39":{"title":"Map over Layer","titles":["Experimental Features"]},"40":{"title":"Debugging Functionality","titles":["Experimental Features"]},"41":{"title":"Tied Parameters","titles":["Experimental Features"]},"42":{"title":"Interoperability between Lux and other packages","titles":[]},"43":{"title":"Switching from older frameworks","titles":["Interoperability between Lux and other packages"]},"44":{"title":"Flux Models to Lux Models","titles":["Interoperability between Lux and other packages","Switching from older frameworks"]},"45":{"title":"Using a different backend for Lux","titles":["Interoperability between Lux and other packages"]},"46":{"title":"Lux Models to Simple Chains","titles":["Interoperability between Lux and other packages","Using a different backend for Lux"]},"47":{"title":"Built-In Layers","titles":[]},"48":{"title":"Containers","titles":["Built-In Layers"]},"49":{"title":"Convolutional Layers","titles":["Built-In Layers"]},"50":{"title":"Dropout Layers","titles":["Built-In Layers"]},"51":{"title":"Pooling Layers","titles":["Built-In Layers"]},"52":{"title":"Recurrent Layers","titles":["Built-In Layers"]},"53":{"title":"Linear Layers","titles":["Built-In Layers"]},"54":{"title":"Misc. Helper Layers","titles":["Built-In Layers"]},"55":{"title":"Normalization Layers","titles":["Built-In Layers"]},"56":{"title":"Upsampling","titles":["Built-In Layers"]},"57":{"title":"LuxTestUtils","titles":[]},"58":{"title":"Testing using JET.jl","titles":["LuxTestUtils"]},"59":{"title":"Gradient Correctness","titles":["LuxTestUtils"]},"60":{"title":"Extensions to @test","titles":["LuxTestUtils"]},"61":{"title":"Utilities","titles":[]},"62":{"title":"Training API","titles":["Utilities"]},"63":{"title":"Loss Functions","titles":["Utilities"]},"64":{"title":"LuxOps Module","titles":["Utilities"]},"65":{"title":"Recursive Operations","titles":["Utilities"]},"66":{"title":"Updating Floating Point Precision","titles":["Utilities"]},"67":{"title":"Element Type Matching","titles":["Utilities"]},"68":{"title":"Stateful Layer","titles":["Utilities"]},"69":{"title":"Compact Layer","titles":["Utilities"]},"70":{"title":"Miscellaneous","titles":["Utilities"]},"71":{"title":"How to Install Lux.jl?","titles":[]},"72":{"title":"Want GPU Support?","titles":[]},"73":{"title":"Want XLA Support?","titles":[]},"74":{"title":"Citation","titles":[]},"75":{"title":"Getting Started","titles":[]},"76":{"title":"Installation","titles":["Getting Started"]},"77":{"title":"Quickstart","titles":["Getting Started"]},"78":{"title":"Defining Custom Layers","titles":["Getting Started"]},"79":{"title":"Additional Packages","titles":["Getting Started"]},"80":{"title":"GPU Support","titles":["Getting Started"]},"81":{"title":"Why we wrote Lux?","titles":[]},"82":{"title":"Design Principles","titles":["Why we wrote Lux?"]},"83":{"title":"Why use Lux over Flux?","titles":["Why we wrote Lux?"]},"84":{"title":"Resources to Get Started","titles":[]},"85":{"title":"Updating to Lux v1","titles":[]},"86":{"title":"LuxLib.jl","titles":["Updating to Lux v1"]},"87":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxLib.jl"]},"88":{"title":"New Major Features","titles":["Updating to Lux v1","LuxLib.jl"]},"89":{"title":"LuxCore.jl","titles":["Updating to Lux v1"]},"90":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxCore.jl"]},"91":{"title":"New Major Features","titles":["Updating to Lux v1","LuxCore.jl"]},"92":{"title":"WeightInitializers.jl","titles":["Updating to Lux v1"]},"93":{"title":"MLDataDevices.jl","titles":["Updating to Lux v1"]},"94":{"title":"Breaking Changes","titles":["Updating to Lux v1","MLDataDevices.jl"]},"95":{"title":"New Major Features","titles":["Updating to Lux v1","MLDataDevices.jl"]},"96":{"title":"Lux.jl","titles":["Updating to Lux v1"]},"97":{"title":"Breaking Changes (Removed Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"98":{"title":"Breaking Changes (Moved Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"99":{"title":"Breaking Changes (Changes in Defaults)","titles":["Updating to Lux v1","Lux.jl"]},"100":{"title":"New Features","titles":["Updating to Lux v1","Lux.jl"]},"101":{"title":"Automatic Differentiation","titles":[]},"102":{"title":"Overview","titles":["Automatic Differentiation"]},"103":{"title":"Recommendations","titles":["Automatic Differentiation"]},"104":{"title":"Support Class","titles":["Automatic Differentiation"]},"105":{"title":"Footnotes","titles":["Automatic Differentiation"]},"106":{"title":"Compiling Lux Models using Reactant.jl","titles":[]},"107":{"title":"Using the TrainState API","titles":["Compiling Lux Models using Reactant.jl"]},"108":{"title":"Debugging Lux Models","titles":[]},"109":{"title":"Incorrect Model Specification: Dimension Mismatch Problems","titles":["Debugging Lux Models"]},"110":{"title":"Tracking down NaNs","titles":["Debugging Lux Models"]},"111":{"title":"Conclusion","titles":["Debugging Lux Models"]},"112":{"title":"Dispatching on Custom Input Types","titles":[]},"113":{"title":"Which function should participate in dispatch?","titles":["Dispatching on Custom Input Types"]},"114":{"title":"Concrete Example","titles":["Dispatching on Custom Input Types"]},"115":{"title":"Time-Dependent Chain Implementation","titles":["Dispatching on Custom Input Types","Concrete Example"]},"116":{"title":"Running the TDChain","titles":["Dispatching on Custom Input Types","Concrete Example"]},"117":{"title":"Writing the Correct Dispatch Rules","titles":["Dispatching on Custom Input Types","Concrete Example"]},"118":{"title":"Using the Same Input for Non-TD Models","titles":["Dispatching on Custom Input Types","Concrete Example"]},"119":{"title":"Distributed Data Parallel Training","titles":[]},"120":{"title":"Guide to Integrating DistributedUtils into your code","titles":["Distributed Data Parallel Training"]},"121":{"title":"Migration Guide from FluxMPI.jl","titles":["Distributed Data Parallel Training"]},"122":{"title":"Removed Functionality","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"123":{"title":"Key Differences","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"124":{"title":"Known Shortcomings","titles":["Distributed Data Parallel Training"]},"125":{"title":"Freezing Model Parameters","titles":[]},"126":{"title":"Freezing Layers of a Particular Kind","titles":["Freezing Model Parameters"]},"127":{"title":"Freezing by Layer Name","titles":["Freezing Model Parameters"]},"128":{"title":"Freezing Part of the Parameters","titles":["Freezing Model Parameters"]},"129":{"title":"Freezing Part of a Chain","titles":["Freezing Model Parameters"]},"130":{"title":"GPU Management","titles":[]},"131":{"title":"Automatic Backend Management (Recommended Approach)","titles":["GPU Management"]},"132":{"title":"Manual Backend Management","titles":["GPU Management"]},"133":{"title":"Lux Interface","titles":[]},"134":{"title":"Layer Interface","titles":["Lux Interface"]},"135":{"title":"Singular Layer","titles":["Lux Interface","Layer Interface"]},"136":{"title":"Container Layer","titles":["Lux Interface","Layer Interface"]},"137":{"title":"Parameter Interface","titles":["Lux Interface"]},"138":{"title":"State Interface","titles":["Lux Interface"]},"139":{"title":"Migrating from Flux to Lux","titles":[]},"140":{"title":"Implementing Custom Layers","titles":["Migrating from Flux to Lux"]},"141":{"title":"Certain Important Implementation Details","titles":["Migrating from Flux to Lux"]},"142":{"title":"Training/Inference Mode","titles":["Migrating from Flux to Lux","Certain Important Implementation Details"]},"143":{"title":"Can we still use Flux Layers?","titles":["Migrating from Flux to Lux"]},"144":{"title":"Nested Automatic Differentiation","titles":[]},"145":{"title":"Loss Function containing Jacobian Computation","titles":["Nested Automatic Differentiation"]},"146":{"title":"Using Batched Jacobian for Multiple Inputs","titles":["Nested Automatic Differentiation","Loss Function containing Jacobian Computation"]},"147":{"title":"Loss Function contains Gradient Computation","titles":["Nested Automatic Differentiation"]},"148":{"title":"Loss Function computing the Jacobian of the Parameters","titles":["Nested Automatic Differentiation"]},"149":{"title":"Hutchinson Trace Estimation","titles":["Nested Automatic Differentiation"]},"150":{"title":"Computing using the Vector-Jacobian Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"151":{"title":"Computing using the Jacobian-Vector Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"152":{"title":"Computing using the Full Jacobian","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"153":{"title":"Neural Networks Inside GPU Kernels","titles":[]},"154":{"title":"Performance Pitfalls & How to Catch Them","titles":[]},"155":{"title":"Spurious Type-Promotion","titles":["Performance Pitfalls & How to Catch Them"]},"156":{"title":"Scalar Indexing on GPU Arrays","titles":["Performance Pitfalls & How to Catch Them"]},"157":{"title":"Type Instabilities","titles":["Performance Pitfalls & How to Catch Them"]},"158":{"title":"Faster Primitives","titles":["Performance Pitfalls & How to Catch Them"]},"159":{"title":"Optional Dependencies for Performance","titles":["Performance Pitfalls & How to Catch Them"]},"160":{"title":"Data Loading and Device Transfer","titles":["Performance Pitfalls & How to Catch Them"]},"161":{"title":"Preferences for Lux.jl","titles":[]},"162":{"title":"Nested Automatic Differentiation","titles":["Preferences for Lux.jl"]},"163":{"title":"GPU-Aware MPI Support","titles":["Preferences for Lux.jl"]},"164":{"title":"GPU Backend Selection","titles":["Preferences for Lux.jl"]},"165":{"title":"Automatic Eltype Conversion","titles":["Preferences for Lux.jl"]},"166":{"title":"Dispatch Doctor","titles":["Preferences for Lux.jl"]},"167":{"title":"Disabling Loop Vectorization / Octavian","titles":["Preferences for Lux.jl"]},"168":{"title":"Initializing Weights","titles":[]},"169":{"title":"Quick examples","titles":["Initializing Weights"]},"170":{"title":"Julia & Lux for the Uninitiated","titles":[]},"171":{"title":"Arrays","titles":["Julia & Lux for the Uninitiated"]},"172":{"title":"CUDA Arrays","titles":["Julia & Lux for the Uninitiated","Arrays"]},"173":{"title":"(Im)mutability","titles":["Julia & Lux for the Uninitiated"]},"174":{"title":"Managing Randomness","titles":["Julia & Lux for the Uninitiated"]},"175":{"title":"Automatic Differentiation","titles":["Julia & Lux for the Uninitiated"]},"176":{"title":"Gradients","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"177":{"title":"Jacobian-Vector Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"178":{"title":"Vector-Jacobian Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"179":{"title":"Linear Regression","titles":["Julia & Lux for the Uninitiated"]},"180":{"title":"Appendix","titles":["Julia & Lux for the Uninitiated"]},"181":{"title":"Training a Simple LSTM","titles":[]},"182":{"title":"Package Imports","titles":["Training a Simple LSTM"]},"183":{"title":"Dataset","titles":["Training a Simple LSTM"]},"184":{"title":"Creating a Classifier","titles":["Training a Simple LSTM"]},"185":{"title":"Using the @compact API","titles":["Training a Simple LSTM"]},"186":{"title":"Defining Accuracy, Loss and Optimiser","titles":["Training a Simple LSTM"]},"187":{"title":"Training the Model","titles":["Training a Simple LSTM"]},"188":{"title":"Saving the Model","titles":["Training a Simple LSTM"]},"189":{"title":"Appendix","titles":["Training a Simple LSTM"]},"190":{"title":"MNIST Classification with SimpleChains","titles":[]},"191":{"title":"Package Imports","titles":["MNIST Classification with SimpleChains"]},"192":{"title":"Loading MNIST","titles":["MNIST Classification with SimpleChains"]},"193":{"title":"Define the Model","titles":["MNIST Classification with SimpleChains"]},"194":{"title":"Helper Functions","titles":["MNIST Classification with SimpleChains"]},"195":{"title":"Define the Training Loop","titles":["MNIST Classification with SimpleChains"]},"196":{"title":"Finally Training the Model","titles":["MNIST Classification with SimpleChains"]},"197":{"title":"Appendix","titles":["MNIST Classification with SimpleChains"]},"198":{"title":"Training Lux Models using Optimization.jl","titles":[]},"199":{"title":"Imports packages","titles":["Training Lux Models using Optimization.jl"]},"200":{"title":"Generate some training data","titles":["Training Lux Models using Optimization.jl"]},"201":{"title":"Define the DataLoader","titles":["Training Lux Models using Optimization.jl"]},"202":{"title":"Training the model","titles":["Training Lux Models using Optimization.jl"]},"203":{"title":"Plotting the results","titles":["Training Lux Models using Optimization.jl"]},"204":{"title":"Appendix","titles":["Training Lux Models using Optimization.jl"]},"205":{"title":"Tutorials","titles":[]},"206":{"title":"Beginner Tutorials","titles":["Tutorials"]},"207":{"title":"Intermediate Tutorials","titles":["Tutorials"]},"208":{"title":"Advanced Tutorials","titles":["Tutorials"]},"209":{"title":"Larger Models","titles":["Tutorials"]},"210":{"title":"Selected 3rd Party Tutorials","titles":["Tutorials"]},"211":{"title":"MNIST Classification using Neural ODEs","titles":[]},"212":{"title":"Package Imports","titles":["MNIST Classification using Neural ODEs"]},"213":{"title":"Loading MNIST","titles":["MNIST Classification using Neural ODEs"]},"214":{"title":"Define the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"215":{"title":"Create and Initialize the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"216":{"title":"Define Utility Functions","titles":["MNIST Classification using Neural ODEs"]},"217":{"title":"Training","titles":["MNIST Classification using Neural ODEs"]},"218":{"title":"Alternate Implementation using Stateful Layer","titles":["MNIST Classification using Neural ODEs"]},"219":{"title":"Train the new Stateful Neural ODE","titles":["MNIST Classification using Neural ODEs"]},"220":{"title":"Type Stability","titles":["MNIST Classification using Neural ODEs"]},"221":{"title":"Appendix","titles":["MNIST Classification using Neural ODEs"]},"222":{"title":"Training a HyperNetwork on MNIST and FashionMNIST","titles":[]},"223":{"title":"Package Imports","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"224":{"title":"Loading Datasets","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"225":{"title":"Implement a HyperNet Layer","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"226":{"title":"Create and Initialize the HyperNet","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"227":{"title":"Define Utility Functions","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"228":{"title":"Training","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"229":{"title":"Appendix","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"230":{"title":"Training a PINN on 2D PDE","titles":[]},"231":{"title":"Package Imports","titles":["Training a PINN on 2D PDE"]},"232":{"title":"Problem Definition","titles":["Training a PINN on 2D PDE"]},"233":{"title":"Define the Neural Networks","titles":["Training a PINN on 2D PDE"]},"234":{"title":"Define the Loss Functions","titles":["Training a PINN on 2D PDE"]},"235":{"title":"Generate the Data","titles":["Training a PINN on 2D PDE"]},"236":{"title":"Training","titles":["Training a PINN on 2D PDE"]},"237":{"title":"Visualizing the Results","titles":["Training a PINN on 2D PDE"]},"238":{"title":"Appendix","titles":["Training a PINN on 2D PDE"]},"239":{"title":"Fitting a Polynomial using MLP","titles":[]},"240":{"title":"Package Imports","titles":["Fitting a Polynomial using MLP"]},"241":{"title":"Dataset","titles":["Fitting a Polynomial using MLP"]},"242":{"title":"Neural Network","titles":["Fitting a Polynomial using MLP"]},"243":{"title":"Optimizer","titles":["Fitting a Polynomial using MLP"]},"244":{"title":"Loss Function","titles":["Fitting a Polynomial using MLP"]},"245":{"title":"Training","titles":["Fitting a Polynomial using MLP"]},"246":{"title":"Appendix","titles":["Fitting a Polynomial using MLP"]},"247":{"title":"Bayesian Neural Network","titles":[]},"248":{"title":"Generating data","titles":["Bayesian Neural Network"]},"249":{"title":"Building the Neural Network","titles":["Bayesian Neural Network"]},"250":{"title":"Prediction Visualization","titles":["Bayesian Neural Network"]},"251":{"title":"Appendix","titles":["Bayesian Neural Network"]},"252":{"title":"Training a Neural ODE to Model Gravitational Waveforms","titles":[]},"253":{"title":"Package Imports","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"254":{"title":"Define some Utility Functions","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"255":{"title":"Simulating the True Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"256":{"title":"Defiing a Neural Network Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"257":{"title":"Setting Up for Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"258":{"title":"Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"259":{"title":"Visualizing the Results","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"260":{"title":"Appendix","titles":["Training a Neural ODE to Model Gravitational Waveforms"]}},"dirtCount":0,"index":[["π",{"2":{"254":2,"255":1}}],["√",{"2":{"254":2}}],["√t",{"2":{"28":2}}],["↦",{"2":{"254":1}}],["ϕ̇",{"2":{"255":2,"256":2}}],["ϕ",{"2":{"254":5,"255":1,"256":1}}],["χ̇",{"2":{"255":2,"256":2}}],["χ",{"2":{"254":4,"255":4,"256":2}}],["₋₋₋kwargs₋₋₋",{"2":{"220":4}}],["₋₋₋no",{"2":{"220":3}}],["│",{"2":{"220":3,"236":4}}],["─",{"2":{"220":3}}],["θ|x",{"2":{"250":2}}],["θ",{"2":{"202":2,"249":2,"250":9,"257":3}}],["÷",{"2":{"183":4}}],["∥22we",{"2":{"179":1}}],["⟶∑i=1k12∥yi−fw",{"2":{"179":1}}],["∇f",{"2":{"176":3}}],["∇offending",{"2":{"110":2}}],["$",{"2":{"237":1}}],["$i",{"2":{"174":2,"250":1}}],["$name",{"2":{"39":1}}],["⋮",{"2":{"153":4}}],["∞",{"2":{"145":4,"146":4,"147":4,"148":4,"152":8}}],["∘",{"2":{"144":4,"147":1,"234":3}}],["~",{"2":{"116":3,"145":2,"249":2}}],["└──",{"2":{"220":3}}],["└",{"2":{"109":1,"145":2,"236":4}}],["┌",{"2":{"109":1,"145":2,"236":4}}],["∂w",{"2":{"234":2}}],["∂v",{"2":{"234":2}}],["∂y",{"2":{"234":4}}],["∂u",{"2":{"234":10}}],["∂t",{"2":{"147":7,"234":2}}],["∂xyt",{"2":{"234":4}}],["∂x",{"2":{"145":7,"146":7,"148":7,"152":11,"234":4}}],["∂ps",{"2":{"106":4,"145":8,"146":8,"147":8,"148":8,"152":11}}],["∂f∂x",{"2":{"22":2}}],["↩︎",{"2":{"105":8}}],["❌",{"2":{"102":5}}],["❓",{"2":{"102":6}}],["\\tfashionmnist\\ttraining",{"2":{"228":1}}],["\\tfashionmnist\\ttime",{"2":{"228":50}}],["\\ttest",{"2":{"217":45,"219":9,"228":102}}],["\\ttime",{"2":{"217":45,"219":9}}],["\\tloss",{"2":{"107":11}}],["\\t",{"2":{"78":11,"196":60,"228":51,"236":400,"245":6}}],["quantiles",{"2":{"249":1}}],["quadrupole",{"2":{"254":10}}],["quadratic",{"2":{"241":1,"245":1}}],["quadro",{"2":{"229":2,"238":2}}],["questions",{"2":{"84":3,"144":1,"175":1}}],["quick",{"0":{"169":1},"2":{"170":1}}],["quickstart",{"0":{"77":1},"2":{"84":1}}],["quite",{"2":{"81":1,"173":1,"175":1}}],["quoting",{"2":{"106":1}}],["quot",{"2":{"1":2,"8":2,"17":4,"18":1,"19":8,"28":12,"48":4,"54":2,"55":2,"63":8,"65":2,"67":16,"69":2,"70":6,"77":4,"123":2,"140":4,"155":2,"164":8}}],["δ",{"2":{"63":1,"110":3,"200":2}}],["ϵ",{"2":{"63":1}}],["ŷ",{"2":{"63":6}}],["∈",{"2":{"63":2,"254":1}}],["∗y+α∗size",{"2":{"63":1}}],["∗y+α∗0",{"2":{"63":1}}],["∗y^−logσ",{"2":{"63":1}}],["∗log⁡",{"2":{"63":1}}],["−log⁡",{"2":{"63":1}}],["−∑y~log⁡",{"2":{"63":1}}],["−",{"2":{"63":1}}],["−y~∗log⁡",{"2":{"63":1}}],["−x",{"2":{"28":1}}],["≈∑θ∼p",{"2":{"250":1}}],["≈",{"2":{"63":25,"69":1,"131":1,"132":1,"174":2}}],["✖",{"2":{"59":4}}],["✔️",{"2":{"102":13}}],["✔",{"2":{"59":8}}],["`θ`",{"2":{"250":1}}],["`x`",{"2":{"250":1}}],["`circbuff",{"2":{"236":4}}],["`carry`",{"2":{"184":2}}],["`p",{"2":{"214":1}}],["`ps",{"2":{"136":1}}],["`iterators",{"2":{"184":1}}],["`eachslice`",{"2":{"184":1}}],["`∇`",{"2":{"176":1}}],["`autoforwarddiff",{"2":{"146":1}}],["`autozygote",{"2":{"146":1}}],["`a`",{"2":{"140":3}}],["`val",{"2":{"145":2}}],["`training`",{"2":{"145":2}}],["`zygote",{"2":{"145":1}}],["`b`",{"2":{"140":2}}],["`denselayerparameters`",{"2":{"137":2}}],["`namedtuple`",{"2":{"137":2}}],["`model",{"2":{"136":1}}],["`st`",{"2":{"136":1,"256":2}}],["`st",{"2":{"136":1}}],["`lag",{"2":{"236":4}}],["`lstm",{"2":{"184":1}}],["`linear",{"2":{"136":2}}],["`l",{"2":{"135":1}}],["`luxcore",{"2":{"145":2}}],["`luxcore`",{"2":{"135":1}}],["`lux",{"2":{"145":2}}],["`lux``",{"2":{"172":1}}],["`lux`",{"2":{"135":1}}],["`",{"2":{"58":1,"77":2,"135":1,"136":1,"145":2,"146":2,"176":1,"236":8}}],["`octavian",{"2":{"18":1}}],["⋅n+z⋅hprevarguments",{"2":{"52":1}}],["↗",{"2":{"48":2}}],["↘",{"2":{"48":2}}],["→",{"2":{"48":5}}],["8389950080016263e",{"2":{"258":1}}],["839286f",{"2":{"256":1}}],["8392873f",{"2":{"256":1}}],["8393",{"2":{"171":1}}],["8393034",{"2":{"171":1}}],["839303",{"2":{"171":2}}],["831558558145079e",{"2":{"258":1}}],["8318067f",{"2":{"256":1}}],["8317274f",{"2":{"179":1}}],["837020944455422e",{"2":{"258":1}}],["837021f",{"2":{"256":1}}],["8376",{"2":{"249":1}}],["8374606",{"2":{"244":1}}],["837991",{"2":{"148":1}}],["8346457",{"2":{"241":2}}],["8340728",{"2":{"148":1}}],["832428e",{"2":{"202":1}}],["83",{"2":{"196":3,"217":6,"219":1,"228":3}}],["8361335544351264e",{"2":{"258":1}}],["8361726f",{"2":{"256":1}}],["836",{"2":{"193":2}}],["8331010723091334e",{"2":{"258":1}}],["833079f",{"2":{"256":1}}],["833518",{"2":{"172":1}}],["8337032",{"2":{"126":1}}],["833333333333333",{"2":{"63":1}}],["893460640313953e",{"2":{"258":1}}],["8935217f",{"2":{"256":1}}],["891199510196862e",{"2":{"258":1}}],["8912086f",{"2":{"256":1}}],["8915",{"2":{"249":1}}],["8946",{"2":{"249":1}}],["8904",{"2":{"249":1}}],["898531164818111e",{"2":{"258":1}}],["89858943",{"2":{"147":1}}],["898287f",{"2":{"256":1}}],["8984",{"2":{"249":1}}],["897550731873056e",{"2":{"258":1}}],["897994179061968e",{"2":{"258":1}}],["89746f",{"2":{"256":1}}],["8977897f",{"2":{"256":1}}],["8972864",{"2":{"241":1}}],["8976378",{"2":{"241":2}}],["89",{"2":{"196":2,"228":2}}],["896550579645388e",{"2":{"258":1}}],["8965212f",{"2":{"256":1}}],["896738682901885e",{"2":{"258":1}}],["896779f",{"2":{"256":1}}],["8969011",{"2":{"241":1}}],["896",{"2":{"193":2}}],["8990",{"2":{"249":1}}],["8992s\\ttraining",{"2":{"217":1}}],["899766",{"2":{"171":1}}],["8999977",{"2":{"148":1}}],["815920248644475e",{"2":{"258":1}}],["815336",{"2":{"148":1}}],["81995742580044e",{"2":{"258":1}}],["8193810330943946e",{"2":{"258":1}}],["819437f",{"2":{"256":1}}],["8181981720228886e",{"2":{"258":1}}],["818454f",{"2":{"256":1}}],["818522f",{"2":{"256":1}}],["816875457193028e",{"2":{"258":1}}],["8161495f",{"2":{"256":1}}],["816281",{"2":{"241":1}}],["817635638716272e",{"2":{"258":1}}],["8176258f",{"2":{"256":1}}],["8173783982243132",{"2":{"153":2}}],["8100952f",{"2":{"256":1}}],["8100",{"2":{"249":1}}],["810493089874772e",{"2":{"258":1}}],["810437f",{"2":{"256":1}}],["8104",{"2":{"249":1}}],["8110236",{"2":{"241":2}}],["81",{"2":{"196":3,"217":7,"219":1,"228":34,"249":1}}],["81446123",{"2":{"244":1}}],["81481",{"2":{"217":2,"219":1}}],["8146973e",{"2":{"147":2}}],["8147778",{"2":{"106":1}}],["880518264565269e",{"2":{"258":1}}],["8804572f",{"2":{"256":1}}],["8879226771262375e",{"2":{"258":1}}],["8879852053473542e",{"2":{"258":1}}],["8873688f",{"2":{"256":1}}],["8848355213978774e",{"2":{"258":1}}],["8848355f",{"2":{"256":1}}],["881097f",{"2":{"256":1}}],["8812",{"2":{"249":1}}],["885398713275422e",{"2":{"258":1}}],["8855047f",{"2":{"256":1}}],["8852221",{"2":{"148":1}}],["8883887f",{"2":{"256":1}}],["88889",{"2":{"217":1}}],["8890",{"2":{"249":1}}],["88951594",{"2":{"241":1}}],["8821303",{"2":{"241":1}}],["88214725",{"2":{"148":1}}],["88",{"2":{"228":7}}],["886188e",{"2":{"202":1}}],["883664f0",{"2":{"145":1}}],["8642770588000753e",{"2":{"258":1}}],["864361f",{"2":{"256":1}}],["865105262750728e",{"2":{"258":1}}],["8650662f",{"2":{"256":1}}],["8668151636623035e",{"2":{"258":1}}],["8669716f",{"2":{"256":1}}],["86614174",{"2":{"241":2}}],["860466134072651e",{"2":{"258":1}}],["8604027274294636e",{"2":{"258":1}}],["8602841f",{"2":{"256":1}}],["8601828553599953",{"2":{"174":1}}],["8601870861800127",{"2":{"153":2}}],["863054627037491e",{"2":{"258":1}}],["863067053716329e",{"2":{"258":1}}],["863067f",{"2":{"256":1}}],["8638",{"2":{"249":1}}],["8689",{"2":{"249":1}}],["8688824",{"2":{"241":1}}],["86",{"2":{"196":1,"228":1}}],["867422899924916e",{"2":{"258":1}}],["867407472115042e",{"2":{"258":1}}],["8674763752817414",{"2":{"153":1}}],["8674763752817416",{"2":{"153":1}}],["867507f",{"2":{"256":1}}],["8671518f",{"2":{"256":1}}],["8672",{"2":{"249":1}}],["867202",{"2":{"145":1}}],["869632",{"2":{"145":1}}],["8627415f",{"2":{"256":1}}],["862480861009647",{"2":{"153":2}}],["86290836",{"2":{"126":1}}],["8626451f",{"2":{"106":5}}],["852119549280157e",{"2":{"258":1}}],["8522256f",{"2":{"256":1}}],["8553122257375194e",{"2":{"258":1}}],["8510923759576366e",{"2":{"258":1}}],["85185",{"2":{"217":2}}],["85882f",{"2":{"256":1}}],["8534",{"2":{"249":1}}],["85344f",{"2":{"77":1}}],["85",{"2":{"217":4,"228":7}}],["850675824529835e",{"2":{"258":1}}],["8500910564565225e",{"2":{"258":1}}],["85050875",{"2":{"241":1}}],["850147f",{"2":{"256":1}}],["8501",{"2":{"236":1}}],["850",{"2":{"193":2}}],["8576185332829483e",{"2":{"258":1}}],["8576512",{"2":{"241":1}}],["8579498698038935e",{"2":{"258":1}}],["857528f",{"2":{"256":1}}],["8575416f",{"2":{"256":1}}],["857257",{"2":{"172":1}}],["857126",{"2":{"172":1}}],["8573820474674845",{"2":{"171":1}}],["859405",{"2":{"147":1,"148":1}}],["8541596",{"2":{"145":1}}],["8c79",{"2":{"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["809915413981493e",{"2":{"258":1}}],["8094",{"2":{"249":1}}],["809456",{"2":{"171":1}}],["8081411785952726e",{"2":{"258":1}}],["8080082767673397e",{"2":{"258":1}}],["808475f",{"2":{"256":1}}],["802928379427108e",{"2":{"258":1}}],["8029505f",{"2":{"256":1}}],["806580953295295e",{"2":{"258":1}}],["806665f",{"2":{"256":1}}],["8067698",{"2":{"147":1}}],["803024f",{"2":{"256":1}}],["8031547",{"2":{"241":1}}],["8031496",{"2":{"241":2}}],["8079692f",{"2":{"256":1}}],["8013471936127532e",{"2":{"258":1}}],["8013359",{"2":{"241":1}}],["8010991f",{"2":{"256":1}}],["805166068769852e",{"2":{"258":1}}],["80549214220148e",{"2":{"258":1}}],["8053244f",{"2":{"256":1}}],["8050922f",{"2":{"256":1}}],["8050",{"2":{"249":1}}],["80",{"2":{"180":1,"189":1,"196":1,"204":1,"217":10,"219":2,"221":1,"228":2,"246":1,"248":1}}],["8044134588817553e",{"2":{"258":1}}],["804539601006992e",{"2":{"258":1}}],["8045699f",{"2":{"256":1}}],["8046007f",{"2":{"256":1}}],["80465555",{"2":{"179":1}}],["8048109",{"2":{"244":1}}],["804725e",{"2":{"202":1}}],["80423f",{"2":{"77":1}}],["800467301988405e",{"2":{"258":1}}],["8002876f",{"2":{"256":1}}],["8003552f",{"2":{"256":1}}],["8003627",{"2":{"147":1}}],["80019703032551e",{"2":{"258":1}}],["80019880990992e",{"2":{"258":1}}],["8001",{"2":{"179":1,"236":1}}],["800792614736468",{"2":{"153":2}}],["800",{"2":{"107":1}}],["8f",{"2":{"107":1,"179":1}}],["8765430549467068e",{"2":{"258":1}}],["8762299f",{"2":{"256":1}}],["875698124522638e",{"2":{"258":1}}],["875689f",{"2":{"256":1}}],["875082",{"2":{"241":1}}],["878031786878868e",{"2":{"258":1}}],["878023f",{"2":{"256":1}}],["8787",{"2":{"249":1}}],["870637274429363e",{"2":{"258":1}}],["870693f",{"2":{"256":1}}],["870381652225233e",{"2":{"258":1}}],["8703817f",{"2":{"256":1}}],["8743566694758174e",{"2":{"258":1}}],["87431f",{"2":{"77":1}}],["8744627f",{"2":{"256":1}}],["8740157",{"2":{"241":2}}],["877328713490479e",{"2":{"258":1}}],["8771",{"2":{"249":1}}],["877497",{"2":{"177":1,"178":1}}],["87702435",{"2":{"106":1}}],["87",{"2":{"109":1,"196":2,"228":7}}],["87143785f0",{"2":{"77":1}}],["8245915914490362e",{"2":{"258":1}}],["8247714f",{"2":{"256":1}}],["8260223285790273e",{"2":{"258":1}}],["821967152597545e",{"2":{"258":1}}],["821931f",{"2":{"256":1}}],["8291538f",{"2":{"256":1}}],["82945824",{"2":{"77":1}}],["8275429413973564e",{"2":{"258":1}}],["8276045287786293e",{"2":{"258":1}}],["8279512f",{"2":{"256":1}}],["8278398f",{"2":{"256":1}}],["8277",{"2":{"249":1}}],["8271f",{"2":{"77":1}}],["820346f",{"2":{"256":1}}],["82030004",{"2":{"241":1}}],["8209s\\ttraining",{"2":{"217":1}}],["82293516",{"2":{"241":1}}],["82",{"2":{"196":1,"217":6,"219":2,"228":2,"249":2}}],["82805836",{"2":{"126":1}}],["8",{"2":{"48":2,"54":1,"63":1,"77":27,"106":28,"145":1,"147":2,"148":3,"171":2,"173":4,"183":1,"187":15,"196":2,"202":2,"217":5,"219":1,"228":2,"241":1,"243":1,"245":1,"249":8,"251":1,"256":73,"258":71,"260":1}}],["84866981477426e",{"2":{"258":1}}],["8484217f",{"2":{"256":1}}],["8410285438783166e",{"2":{"258":1}}],["8413417f",{"2":{"256":1}}],["841231",{"2":{"148":1}}],["843600354363898e",{"2":{"258":1}}],["8436394f",{"2":{"256":1}}],["84318924837101e",{"2":{"258":1}}],["8431494f",{"2":{"256":1}}],["8402",{"2":{"249":1}}],["840588",{"2":{"145":1}}],["8499767",{"2":{"241":1}}],["8490221",{"2":{"137":1}}],["842214216759775e",{"2":{"258":1}}],["842869798050569e",{"2":{"258":1}}],["8425274f",{"2":{"256":1}}],["8425196",{"2":{"241":2}}],["8424188",{"2":{"241":1}}],["847032e",{"2":{"202":1}}],["847106",{"2":{"148":1}}],["8454118",{"2":{"148":1}}],["84576",{"2":{"77":1}}],["846457",{"2":{"148":1}}],["84",{"2":{"46":2,"193":6,"196":3,"217":1,"219":1,"228":27}}],["9f",{"2":{"236":8}}],["9316310508200342e",{"2":{"258":1}}],["93036998395775e",{"2":{"258":1}}],["9307",{"2":{"249":1}}],["935075652421991e",{"2":{"258":1}}],["935553458910985e",{"2":{"258":1}}],["9359056528472726e",{"2":{"258":1}}],["935946f",{"2":{"256":1}}],["935245f",{"2":{"256":1}}],["9351817f",{"2":{"256":1}}],["9373120476562817e",{"2":{"258":1}}],["9377987716903e",{"2":{"258":1}}],["9372379f",{"2":{"256":1}}],["9376423f",{"2":{"256":1}}],["9370079",{"2":{"241":2}}],["93689685051731e",{"2":{"258":1}}],["936817f",{"2":{"256":1}}],["9366498533260655e",{"2":{"258":1}}],["936778f",{"2":{"256":1}}],["932041f",{"2":{"256":1}}],["934988285790656e",{"2":{"258":1}}],["934397942870884e",{"2":{"258":1}}],["9343881f",{"2":{"256":1}}],["9340093",{"2":{"147":1}}],["93",{"2":{"228":3}}],["933989f",{"2":{"256":1}}],["933464e",{"2":{"202":1}}],["9338737",{"2":{"148":1}}],["93s",{"2":{"196":2}}],["9192847618920505e",{"2":{"258":1}}],["919291837329557e",{"2":{"258":1}}],["9190291f",{"2":{"256":1}}],["912189956035279e",{"2":{"258":1}}],["912223f",{"2":{"256":1}}],["9127535f",{"2":{"256":1}}],["912781",{"2":{"152":1}}],["9127817",{"2":{"152":1}}],["912782",{"2":{"152":1}}],["911365799922083e",{"2":{"258":1}}],["91192183958915e",{"2":{"258":1}}],["9110735f",{"2":{"256":1}}],["9114s\\ttraining",{"2":{"217":1}}],["918320859302846e",{"2":{"258":1}}],["9187097f",{"2":{"256":1}}],["918918",{"2":{"168":1}}],["915987f",{"2":{"256":1}}],["9109",{"2":{"249":1}}],["9108207",{"2":{"241":1}}],["910251",{"2":{"147":1}}],["91",{"2":{"228":2}}],["916407932044661e",{"2":{"258":1}}],["9164896",{"2":{"148":1}}],["9161786f",{"2":{"256":1}}],["9161865",{"2":{"241":1}}],["9166",{"2":{"249":1}}],["9169s\\ttraining",{"2":{"217":1}}],["91s",{"2":{"196":3}}],["9130466905054665e",{"2":{"258":1}}],["9130077",{"2":{"106":2}}],["913108f",{"2":{"256":1}}],["9139614",{"2":{"148":1}}],["9534125460633956e",{"2":{"258":1}}],["9535186f",{"2":{"256":1}}],["9559060483186072e",{"2":{"258":1}}],["955220482989485e",{"2":{"258":1}}],["95526f",{"2":{"256":1}}],["958737487844602e",{"2":{"258":1}}],["958734f",{"2":{"256":1}}],["956199257790459e",{"2":{"258":1}}],["9562175f",{"2":{"256":1}}],["956209f",{"2":{"256":1}}],["9560859f",{"2":{"256":1}}],["952052499672205e",{"2":{"258":1}}],["9525932735486006e",{"2":{"258":1}}],["9521266f",{"2":{"256":1}}],["9527579f",{"2":{"256":1}}],["951161624833396e",{"2":{"258":1}}],["9511775",{"2":{"147":1}}],["9515",{"2":{"249":1}}],["9517",{"2":{"249":1}}],["9547",{"2":{"249":1}}],["95401394",{"2":{"241":1}}],["95",{"2":{"228":2}}],["95s",{"2":{"196":1}}],["950718990997625e",{"2":{"258":1}}],["950440444129138e",{"2":{"258":1}}],["9504306f",{"2":{"256":1}}],["950635f",{"2":{"256":1}}],["9508693f",{"2":{"256":1}}],["9500",{"2":{"249":1}}],["9500444",{"2":{"147":1}}],["9501",{"2":{"236":1}}],["9501405",{"2":{"148":1}}],["959182",{"2":{"148":1}}],["9577397",{"2":{"147":1}}],["9572f",{"2":{"77":1}}],["977145613106131e",{"2":{"258":1}}],["979215856403858e",{"2":{"258":1}}],["9792998f",{"2":{"256":1}}],["9799s\\ttraining",{"2":{"217":1}}],["970395539753857e",{"2":{"258":1}}],["9707843f",{"2":{"256":1}}],["9708",{"2":{"249":1}}],["97161f",{"2":{"256":1}}],["971899",{"2":{"109":1}}],["973453312900088e",{"2":{"258":1}}],["9734574",{"2":{"145":1}}],["97342f",{"2":{"256":1}}],["97378695",{"2":{"241":1}}],["976981f",{"2":{"256":1}}],["9767632",{"2":{"241":1}}],["9766699015845924",{"2":{"174":4}}],["97667",{"2":{"171":1}}],["974915285260097e",{"2":{"258":1}}],["9747",{"2":{"249":1}}],["9742292",{"2":{"241":1}}],["974361e",{"2":{"202":1}}],["97",{"2":{"228":11,"249":1}}],["97s",{"2":{"196":1}}],["975649",{"2":{"172":1}}],["9721554734769875",{"2":{"153":2}}],["9727281",{"2":{"147":1}}],["9063886060447805e",{"2":{"258":1}}],["904137003444741e",{"2":{"258":1}}],["9045",{"2":{"249":1}}],["9043228",{"2":{"145":1}}],["9050983374899115e",{"2":{"258":1}}],["90508",{"2":{"126":1,"129":1}}],["905850792586131e",{"2":{"258":1}}],["90588f",{"2":{"256":1}}],["9055119",{"2":{"241":2}}],["90",{"2":{"228":4}}],["901350552518186e",{"2":{"258":1}}],["901346e",{"2":{"202":1}}],["901844f",{"2":{"256":1}}],["9019142280758281",{"2":{"153":2}}],["90s",{"2":{"196":2}}],["902982902377702",{"2":{"153":2}}],["90910274",{"2":{"148":1}}],["9094505",{"2":{"145":1}}],["9071",{"2":{"249":1}}],["9076505",{"2":{"244":1}}],["9076533",{"2":{"145":1}}],["9079035",{"2":{"241":1}}],["90779305",{"2":{"241":1}}],["90772897",{"2":{"126":1}}],["900952f",{"2":{"256":1}}],["900935",{"2":{"168":1}}],["9001",{"2":{"179":1,"236":1}}],["900",{"2":{"107":1}}],["96367117324698e",{"2":{"258":1}}],["9633407f",{"2":{"256":1}}],["964379804412713e",{"2":{"258":1}}],["9641446f",{"2":{"256":1}}],["9640",{"2":{"249":1}}],["968654890480487e",{"2":{"258":1}}],["968056158318532e",{"2":{"258":1}}],["968066f",{"2":{"256":1}}],["968095f",{"2":{"256":1}}],["968504",{"2":{"241":2}}],["969684f",{"2":{"256":1}}],["9694",{"2":{"249":1}}],["9693878",{"2":{"148":1}}],["9679338",{"2":{"241":1}}],["967811",{"2":{"172":1}}],["96062994",{"2":{"241":2}}],["9604645f",{"2":{"106":4}}],["96",{"2":{"228":4}}],["96296",{"2":{"219":1}}],["9626998",{"2":{"147":1}}],["965008",{"2":{"244":1}}],["9653s\\ttraining",{"2":{"217":1}}],["965117",{"2":{"126":1}}],["966199e",{"2":{"202":1}}],["98728756368737e",{"2":{"258":1}}],["987929974060993e",{"2":{"258":1}}],["987908f",{"2":{"256":1}}],["9873614f",{"2":{"256":1}}],["987689",{"2":{"145":1}}],["981501507368307e",{"2":{"258":1}}],["981542f",{"2":{"256":1}}],["981339",{"2":{"171":1}}],["982676824847756e",{"2":{"258":1}}],["9826804f",{"2":{"256":1}}],["982162595",{"2":{"236":2}}],["984060427337718e",{"2":{"258":1}}],["9847",{"2":{"249":1}}],["9849229",{"2":{"244":1}}],["9849193f",{"2":{"106":1}}],["98576458235899e",{"2":{"258":1}}],["985735f",{"2":{"256":1}}],["9853",{"2":{"249":1}}],["9856883",{"2":{"241":1}}],["98",{"2":{"228":10}}],["9835s\\ttraining",{"2":{"217":1}}],["98396176",{"2":{"110":1}}],["986553e",{"2":{"202":1}}],["980320584643474e",{"2":{"258":1}}],["9803549",{"2":{"241":1}}],["98074926294218e",{"2":{"258":1}}],["980496610399982e",{"2":{"258":1}}],["980164f",{"2":{"256":1}}],["9808615164567725e",{"2":{"258":1}}],["9808005f",{"2":{"256":1}}],["9808103f",{"2":{"256":1}}],["980506f",{"2":{"256":1}}],["980",{"2":{"159":1}}],["9802322f",{"2":{"106":2}}],["98023f",{"2":{"106":4}}],["9470226826847666e",{"2":{"258":1}}],["9470896",{"2":{"129":1}}],["9420538534566074e",{"2":{"258":1}}],["942030083010798e",{"2":{"258":1}}],["94280557242975e",{"2":{"258":1}}],["942128f",{"2":{"256":1}}],["94259596",{"2":{"244":1}}],["9437603677126076e",{"2":{"258":1}}],["943035f",{"2":{"256":1}}],["9436993f",{"2":{"256":1}}],["9436797",{"2":{"179":1}}],["946031771074462e",{"2":{"258":1}}],["946028f",{"2":{"256":1}}],["9460",{"2":{"249":1}}],["941742f",{"2":{"256":1}}],["941997603407367e",{"2":{"258":1}}],["941909565047711e",{"2":{"258":1}}],["9419897f",{"2":{"256":1}}],["9419194f",{"2":{"256":1}}],["9414",{"2":{"249":1}}],["9413",{"2":{"249":1}}],["9418648",{"2":{"241":1}}],["9489941",{"2":{"241":1}}],["94s",{"2":{"196":1}}],["94",{"2":{"196":2,"228":2}}],["945782932703545e",{"2":{"258":1}}],["9459628f",{"2":{"256":1}}],["9452923791531558",{"2":{"153":2}}],["94557f",{"2":{"77":1}}],["940363332446033e",{"2":{"258":1}}],["9403927f",{"2":{"256":1}}],["9407812632078533e",{"2":{"258":1}}],["9407f",{"2":{"106":1}}],["9408554f",{"2":{"256":1}}],["9402",{"2":{"249":1}}],["9405848223512736",{"2":{"174":4}}],["940585",{"2":{"171":1}}],["940697f",{"2":{"106":1}}],["9g",{"2":{"78":1}}],["924535204588754e",{"2":{"258":1}}],["9248484f",{"2":{"256":1}}],["9248211",{"2":{"241":1}}],["926157575160787e",{"2":{"258":1}}],["926161f",{"2":{"256":1}}],["923163995862929e",{"2":{"258":1}}],["923409045649304e",{"2":{"258":1}}],["9233185f",{"2":{"256":1}}],["9230",{"2":{"249":1}}],["923950547913545",{"2":{"153":2}}],["925542248666441e",{"2":{"258":1}}],["925509f",{"2":{"256":1}}],["925144669676538e",{"2":{"258":1}}],["9251225f",{"2":{"256":1}}],["9253495",{"2":{"241":1}}],["92593",{"2":{"217":1,"219":1}}],["929220396753885e",{"2":{"258":1}}],["9296287f",{"2":{"256":1}}],["92913383",{"2":{"241":2}}],["929f",{"2":{"77":1}}],["92",{"2":{"228":4}}],["920",{"2":{"221":1}}],["9206433",{"2":{"145":1}}],["92s",{"2":{"196":1}}],["928609",{"2":{"145":1}}],["92832f",{"2":{"77":1}}],["92155594",{"2":{"176":4}}],["9214196",{"2":{"148":1}}],["921",{"2":{"69":1}}],["9988234096740654e",{"2":{"258":1}}],["998988f",{"2":{"256":1}}],["990951989528911e",{"2":{"258":1}}],["991401593183841e",{"2":{"258":1}}],["9917904f",{"2":{"256":1}}],["9913408f",{"2":{"256":1}}],["9913",{"2":{"249":1}}],["9916",{"2":{"249":1}}],["9956",{"2":{"249":1}}],["9972457875633954e",{"2":{"258":1}}],["9975381f",{"2":{"256":1}}],["9971247",{"2":{"244":1}}],["997003",{"2":{"77":6}}],["9934048426502315e",{"2":{"258":1}}],["9934452f",{"2":{"256":1}}],["9934639",{"2":{"241":1}}],["9930804",{"2":{"241":1}}],["992126",{"2":{"241":2}}],["992662",{"2":{"145":1}}],["99",{"2":{"148":1,"217":1,"228":6}}],["9999881",{"2":{"110":2}}],["999986",{"2":{"109":1}}],["99998605",{"2":{"109":1}}],["999",{"2":{"77":7,"243":1,"245":1}}],["99900760833609",{"2":{"63":1}}],["9",{"2":{"48":2,"63":4,"77":15,"97":1,"106":25,"126":2,"129":2,"145":1,"147":1,"148":2,"152":1,"171":1,"179":1,"187":14,"192":1,"195":2,"196":2,"202":1,"213":1,"217":52,"219":10,"224":2,"228":2,"243":1,"245":1,"249":3,"256":68,"258":84}}],["≤",{"2":{"28":2,"53":2,"59":2}}],["znver2",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["zip",{"2":{"107":1,"236":1}}],["zips",{"2":{"39":1}}],["zyg",{"2":{"106":2}}],["zygotevjp",{"2":{"220":13}}],["zygote",{"2":{"22":1,"23":1,"59":1,"62":1,"63":1,"69":2,"77":3,"78":1,"83":1,"102":1,"103":3,"106":3,"110":2,"139":2,"140":2,"144":3,"145":1,"146":1,"147":3,"148":3,"152":3,"170":1,"173":1,"175":3,"176":4,"181":1,"182":1,"191":1,"212":1,"223":1,"231":1,"234":3,"240":1,"245":1}}],["zenodo",{"2":{"74":3}}],["zeroing",{"2":{"39":4,"65":1}}],["zero",{"2":{"28":3,"39":3,"49":1,"50":3,"53":1,"63":2,"64":2,"65":9,"107":1,"146":1,"171":1}}],["zeroed",{"2":{"28":3}}],["zerosc64",{"2":{"29":1}}],["zerosc32",{"2":{"29":1}}],["zerosc16",{"2":{"29":1}}],["zeros64",{"2":{"29":1}}],["zeros32",{"2":{"29":1,"135":2}}],["zeros16",{"2":{"29":1}}],["zeros",{"2":{"28":3,"29":6,"153":2,"171":1,"235":3,"248":1,"249":1}}],["z=",{"2":{"59":1}}],["z=σ",{"2":{"52":1}}],["z",{"2":{"53":3,"59":2,"249":2,"250":8}}],["6e",{"2":{"202":1}}],["6d",{"2":{"179":1}}],["661524845334837e",{"2":{"258":1}}],["6617805f",{"2":{"256":1}}],["665038386102682e",{"2":{"258":1}}],["665241",{"2":{"63":10}}],["669701913376518e",{"2":{"258":1}}],["669020875018921e",{"2":{"258":1}}],["6691436f",{"2":{"256":1}}],["669152f",{"2":{"256":1}}],["669618f",{"2":{"256":1}}],["667524760792516e",{"2":{"258":1}}],["667572",{"2":{"241":1}}],["6672895f",{"2":{"256":1}}],["664891530819169e",{"2":{"258":1}}],["6644723906252673e",{"2":{"258":1}}],["6644625f",{"2":{"256":1}}],["664045442200898e",{"2":{"258":1}}],["664948f",{"2":{"256":1}}],["6645",{"2":{"249":1}}],["6642563709240284",{"2":{"153":2}}],["66852736",{"2":{"244":1}}],["66",{"2":{"196":1,"228":1,"256":1}}],["663036",{"2":{"172":1}}],["6637242",{"2":{"147":1}}],["666323587703881e",{"2":{"258":1}}],["66632f",{"2":{"256":1}}],["6663244",{"2":{"147":1}}],["66667",{"2":{"217":15,"219":4}}],["666682",{"2":{"147":1}}],["666965",{"2":{"168":1}}],["682605193857661e",{"2":{"258":1}}],["6824922f",{"2":{"256":1}}],["6824866f",{"2":{"256":1}}],["684057472544877e",{"2":{"258":1}}],["6846",{"2":{"249":1}}],["686661757944851e",{"2":{"258":1}}],["6866324f",{"2":{"256":1}}],["6867142",{"2":{"148":1}}],["680081f",{"2":{"256":1}}],["680748",{"2":{"126":1}}],["688433f",{"2":{"256":1}}],["6886621418938875e",{"2":{"258":1}}],["688665",{"2":{"148":1}}],["6886741",{"2":{"126":1}}],["6838904f",{"2":{"256":1}}],["6830945313415872",{"2":{"174":1}}],["6897",{"2":{"249":1}}],["681996531663573e",{"2":{"258":1}}],["6819965f",{"2":{"256":1}}],["6816s\\ttraining",{"2":{"217":1}}],["681728",{"2":{"172":1}}],["68",{"2":{"196":1,"217":2,"219":2,"228":5,"249":1}}],["6852",{"2":{"249":1}}],["6859683",{"2":{"241":1}}],["685057209353382e",{"2":{"258":1}}],["6850394",{"2":{"241":2}}],["685079",{"2":{"171":1}}],["6853896",{"2":{"148":1}}],["6876896843207e",{"2":{"258":1}}],["687611",{"2":{"168":1}}],["687522f",{"2":{"256":1}}],["687264",{"2":{"148":1}}],["687216",{"2":{"131":2}}],["69394756128898e",{"2":{"258":1}}],["693892615664247e",{"2":{"258":1}}],["693819f",{"2":{"256":1}}],["69549878640041e",{"2":{"258":1}}],["696747134896089e",{"2":{"258":1}}],["696847498061422e",{"2":{"258":1}}],["6965794f",{"2":{"256":1}}],["6970",{"2":{"249":1}}],["694201944784054e",{"2":{"258":1}}],["694276f",{"2":{"256":1}}],["694104f",{"2":{"256":1}}],["6941232f",{"2":{"256":1}}],["6943",{"2":{"249":1}}],["69472f",{"2":{"77":1}}],["6920981",{"2":{"241":1}}],["6983237",{"2":{"241":1}}],["6907",{"2":{"249":1}}],["6904922",{"2":{"241":1}}],["690434",{"2":{"168":1}}],["6909446526419574",{"2":{"153":2}}],["6913705",{"2":{"145":1}}],["69",{"2":{"115":1,"217":4,"228":1}}],["616056989240049e",{"2":{"258":1}}],["6160606f",{"2":{"256":1}}],["615175753504133e",{"2":{"258":1}}],["615136f",{"2":{"256":1}}],["61512",{"2":{"187":1}}],["6189",{"2":{"249":1}}],["6183s\\ttraining",{"2":{"217":1}}],["6134",{"2":{"249":1}}],["6133",{"2":{"249":1}}],["6138451",{"2":{"106":1}}],["61384505",{"2":{"106":1}}],["611404412270109e",{"2":{"258":1}}],["6113823f",{"2":{"256":1}}],["6117",{"2":{"249":1}}],["61155146",{"2":{"241":1}}],["611584190904272",{"2":{"153":2}}],["6149982940593018e",{"2":{"258":1}}],["6149884f",{"2":{"256":1}}],["6141",{"2":{"249":1}}],["61417323",{"2":{"241":2}}],["6143286",{"2":{"126":1}}],["6121381886129855e",{"2":{"258":1}}],["6129s\\ttraining",{"2":{"217":1}}],["6120",{"2":{"28":1}}],["61",{"2":{"196":1,"228":1}}],["610591118907857e",{"2":{"258":1}}],["610747081034315e",{"2":{"258":1}}],["6107692f",{"2":{"256":1}}],["61098f",{"2":{"256":1}}],["610",{"2":{"148":1}}],["6520928479197705e",{"2":{"258":1}}],["6520089f",{"2":{"256":1}}],["652251395704443e",{"2":{"258":1}}],["6582828f",{"2":{"256":1}}],["6585",{"2":{"249":1}}],["654127154409205e",{"2":{"258":1}}],["6541584",{"2":{"106":1}}],["6541586",{"2":{"106":1}}],["654245591874819e",{"2":{"258":1}}],["654088f",{"2":{"256":1}}],["65358875880763e",{"2":{"258":1}}],["6535497f",{"2":{"256":1}}],["6535434",{"2":{"241":2}}],["653837f",{"2":{"256":1}}],["6578",{"2":{"249":1}}],["6571946",{"2":{"244":1}}],["657670184695808",{"2":{"153":2}}],["651309782621265e",{"2":{"258":1}}],["651034400775136e",{"2":{"258":1}}],["6512011442468293e",{"2":{"258":1}}],["65113f",{"2":{"256":1}}],["6511s\\ttraining",{"2":{"219":1}}],["65178",{"2":{"147":1}}],["65",{"2":{"196":1,"228":2}}],["65s",{"2":{"196":2}}],["655839576295809e",{"2":{"258":1}}],["6558005f",{"2":{"256":1}}],["655881",{"2":{"171":1}}],["6556360448565952",{"2":{"174":1}}],["656613f",{"2":{"106":1}}],["650805f",{"2":{"256":1}}],["6507929f",{"2":{"256":1}}],["6501",{"2":{"236":1}}],["650",{"2":{"55":3}}],["639874536184391e",{"2":{"258":1}}],["639297+0",{"2":{"168":1}}],["633411636843179e",{"2":{"258":1}}],["6333556f",{"2":{"256":1}}],["6318724061504816e",{"2":{"258":1}}],["631842f",{"2":{"256":1}}],["631925818515209e",{"2":{"258":1}}],["6319058f",{"2":{"256":1}}],["6316393f",{"2":{"256":1}}],["6321293526218726e",{"2":{"258":1}}],["6321854f",{"2":{"256":1}}],["6323408",{"2":{"126":1}}],["6389992",{"2":{"241":1}}],["63838756",{"2":{"147":1}}],["63",{"2":{"228":4}}],["6306s\\ttraining",{"2":{"217":1}}],["634195995720673e",{"2":{"258":1}}],["6349",{"2":{"249":1}}],["6340362477836592",{"2":{"174":1}}],["634549",{"2":{"168":1}}],["6370531107315014",{"2":{"153":2}}],["6372185",{"2":{"148":1}}],["63795453",{"2":{"106":1}}],["636308",{"2":{"106":1}}],["623538343419386e",{"2":{"258":1}}],["6237684653481535e",{"2":{"258":1}}],["623225f",{"2":{"256":1}}],["6234753f",{"2":{"256":1}}],["6214",{"2":{"249":1}}],["621958",{"2":{"171":1}}],["6202136",{"2":{"241":1}}],["62",{"2":{"228":5}}],["62963",{"2":{"217":2}}],["6220861999069013e",{"2":{"258":1}}],["6220472",{"2":{"241":2}}],["622359e",{"2":{"202":1}}],["622865",{"2":{"172":1}}],["627",{"2":{"180":1}}],["62625877100596",{"2":{"171":1}}],["6247874f",{"2":{"256":1}}],["624383955429775",{"2":{"174":1}}],["624384",{"2":{"171":1}}],["62450767",{"2":{"106":1}}],["625894249282434e",{"2":{"258":1}}],["625022643448559e",{"2":{"258":1}}],["625665f",{"2":{"256":1}}],["62544",{"2":{"187":1}}],["625",{"2":{"63":1}}],["607256277320514e",{"2":{"258":1}}],["6076053f0",{"2":{"63":2}}],["603665368794104e",{"2":{"258":1}}],["6035467f",{"2":{"256":1}}],["608316f",{"2":{"256":1}}],["608310e",{"2":{"202":1}}],["602354135118717e",{"2":{"258":1}}],["6022355f",{"2":{"256":1}}],["602477f",{"2":{"256":1}}],["60590015145911e",{"2":{"258":1}}],["605733f",{"2":{"256":1}}],["60524",{"2":{"187":1}}],["6042255880650304e",{"2":{"258":1}}],["6041696f",{"2":{"256":1}}],["6049256",{"2":{"148":1}}],["6010473f",{"2":{"256":1}}],["609515865262418e",{"2":{"258":1}}],["609397f",{"2":{"256":1}}],["609",{"2":{"246":1}}],["60",{"2":{"196":1,"228":1,"249":1}}],["6006488882410714e",{"2":{"258":1}}],["6003383",{"2":{"241":1}}],["6001",{"2":{"179":1,"236":1}}],["600",{"2":{"107":1}}],["6064619",{"2":{"106":1}}],["6727633f",{"2":{"256":1}}],["6726905f",{"2":{"256":1}}],["6726513",{"2":{"145":1}}],["671630322411254e",{"2":{"258":1}}],["6716s\\ttraining",{"2":{"217":1}}],["671791213624706e",{"2":{"258":1}}],["671795f",{"2":{"256":1}}],["6713176236549e",{"2":{"258":1}}],["6712",{"2":{"249":1}}],["675358861994533e",{"2":{"258":1}}],["675337f",{"2":{"256":1}}],["6752",{"2":{"249":1}}],["67595f",{"2":{"77":1}}],["677651797705e",{"2":{"258":1}}],["6776220912718907",{"2":{"174":1}}],["6770623951350115e",{"2":{"258":1}}],["6773963105390396e",{"2":{"258":1}}],["677318f",{"2":{"256":1}}],["6772777f",{"2":{"256":1}}],["6771653",{"2":{"241":2}}],["67",{"2":{"196":2,"217":1}}],["67dffc4a8ae",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["6785566454393467e",{"2":{"258":1}}],["678466f",{"2":{"256":1}}],["6784163",{"2":{"77":1}}],["6788868f",{"2":{"256":1}}],["6788",{"2":{"249":1}}],["67861",{"2":{"171":1}}],["6799012768544025e",{"2":{"258":1}}],["6792173223071718e",{"2":{"258":1}}],["67909",{"2":{"171":1}}],["67908657",{"2":{"171":1}}],["679087",{"2":{"171":2}}],["6798847",{"2":{"147":1}}],["6708489619959403e",{"2":{"258":1}}],["67094",{"2":{"168":1}}],["6702257",{"2":{"145":1}}],["6",{"2":{"28":1,"46":2,"48":3,"77":7,"106":1,"145":1,"147":5,"148":7,"152":1,"171":2,"173":3,"179":1,"180":5,"187":14,"189":5,"193":8,"196":2,"197":1,"202":1,"204":5,"217":5,"219":1,"221":5,"228":2,"229":5,"236":4,"238":5,"241":6,"246":5,"249":11,"250":6,"251":1,"255":2,"256":139,"258":132,"260":1}}],["6477537405902708e",{"2":{"258":1}}],["6477456",{"2":{"145":1}}],["641145907561628e",{"2":{"258":1}}],["641014000943504e",{"2":{"258":1}}],["641004f",{"2":{"256":1}}],["641252f",{"2":{"256":1}}],["649362288831361e",{"2":{"258":1}}],["649333f",{"2":{"256":1}}],["6496947f",{"2":{"256":1}}],["6437044622444454e",{"2":{"258":1}}],["6436641f",{"2":{"256":1}}],["6431484",{"2":{"244":1}}],["6484575487642894e",{"2":{"258":1}}],["6484282f",{"2":{"256":1}}],["648153107789875e",{"2":{"258":1}}],["6481567f",{"2":{"256":1}}],["648693f",{"2":{"256":1}}],["6482",{"2":{"249":1}}],["6457",{"2":{"249":1}}],["6456693",{"2":{"241":2}}],["6456416023530093",{"2":{"174":1}}],["6448188f",{"2":{"256":1}}],["6448233",{"2":{"241":1}}],["64495987",{"2":{"241":1}}],["642170116693722e",{"2":{"258":1}}],["642576227731405e",{"2":{"258":1}}],["642244f",{"2":{"256":1}}],["642907f",{"2":{"256":1}}],["6423511984038721",{"2":{"153":2}}],["6424197",{"2":{"147":1}}],["646107344126006e",{"2":{"258":1}}],["64644",{"2":{"241":1}}],["6469914",{"2":{"126":1}}],["64605f",{"2":{"77":1}}],["64013f",{"2":{"256":1}}],["6409345",{"2":{"147":1}}],["6409691",{"2":{"106":1}}],["64058f",{"2":{"77":1}}],["64",{"2":{"18":1,"55":18,"78":1,"180":2,"189":2,"197":2,"204":2,"221":2,"226":2,"228":1,"229":2,"238":2,"246":2,"251":2,"256":1,"260":2}}],["×",{"2":{"23":6,"180":1,"183":2,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["×dn−2×1×1`",{"2":{"55":1}}],["×dn−2×1×1",{"2":{"19":1}}],["×dn−2×1×dn",{"2":{"19":1,"55":1}}],["≥",{"2":{"23":2,"64":1,"236":1}}],["β",{"2":{"19":4,"55":1,"200":2}}],["γ=0",{"2":{"63":2}}],["γ",{"2":{"19":4,"55":1,"63":1,"200":2}}],["762544452466263e",{"2":{"258":1}}],["762511f",{"2":{"256":1}}],["762207681806867e",{"2":{"258":1}}],["762248f",{"2":{"256":1}}],["766030390475728e",{"2":{"258":1}}],["766572836640417e",{"2":{"258":1}}],["7661975f",{"2":{"256":1}}],["7663376f",{"2":{"256":1}}],["7663455",{"2":{"241":1}}],["76158035",{"2":{"241":1}}],["76",{"2":{"196":2,"202":2,"217":6,"219":2,"228":1}}],["76s",{"2":{"196":1}}],["76425457",{"2":{"241":1}}],["76422im",{"2":{"168":1}}],["7648s\\ttraining",{"2":{"219":1}}],["7649314",{"2":{"148":1}}],["7689272356710357",{"2":{"153":2}}],["76718056212653e",{"2":{"258":1}}],["7675686f",{"2":{"256":1}}],["7670128f",{"2":{"256":1}}],["7673453",{"2":{"126":1}}],["7672513",{"2":{"106":1}}],["732576609899568e",{"2":{"258":1}}],["7325025f",{"2":{"256":1}}],["733758721019089e",{"2":{"258":1}}],["73341",{"2":{"241":1}}],["730810321451422e",{"2":{"258":1}}],["7308397f",{"2":{"256":1}}],["730941",{"2":{"148":1}}],["7313888938834663e",{"2":{"258":1}}],["731428f",{"2":{"256":1}}],["731059",{"2":{"63":2}}],["7352",{"2":{"249":1}}],["7370",{"2":{"249":1}}],["7374s\\ttraining",{"2":{"217":1}}],["737600e",{"2":{"202":1}}],["73",{"2":{"196":2,"217":1,"219":1,"228":3}}],["7384",{"2":{"249":1}}],["7384486",{"2":{"241":1}}],["73806",{"2":{"172":1}}],["738356",{"2":{"171":1}}],["7530213827364957e",{"2":{"258":1}}],["753387f",{"2":{"256":1}}],["7533",{"2":{"249":1}}],["75208065494625e",{"2":{"258":1}}],["7528156321214866e",{"2":{"258":1}}],["7526064434901786e",{"2":{"258":1}}],["7527753f",{"2":{"256":1}}],["7511532416364836e",{"2":{"258":1}}],["7511142f",{"2":{"256":1}}],["7517875f",{"2":{"256":1}}],["751377",{"2":{"147":1}}],["7555023f",{"2":{"256":1}}],["7559453",{"2":{"241":1}}],["7560",{"2":{"249":1}}],["75",{"2":{"217":2,"219":1,"228":19,"229":2,"238":2,"249":1,"255":1,"256":2,"259":4}}],["7502",{"2":{"251":1,"260":1}}],["7501",{"2":{"236":1}}],["750",{"2":{"180":1,"189":1,"204":1,"221":1,"246":1}}],["7542796970296486e",{"2":{"258":1}}],["754292474808159e",{"2":{"258":1}}],["7544443f",{"2":{"256":1}}],["754463",{"2":{"148":1}}],["7541456",{"2":{"241":1}}],["754747",{"2":{"148":1}}],["759412",{"2":{"148":1}}],["7580993408473766",{"2":{"17":1}}],["7739296243262974e",{"2":{"258":1}}],["773959801658752e",{"2":{"258":1}}],["7739687f",{"2":{"256":1}}],["7733535276924052",{"2":{"174":4}}],["773354",{"2":{"171":1}}],["7741276f",{"2":{"256":1}}],["7746656838366666",{"2":{"174":1}}],["778370592044982e",{"2":{"258":1}}],["7783706f",{"2":{"256":1}}],["7789",{"2":{"249":1}}],["77165353",{"2":{"241":2}}],["779352864711325e",{"2":{"258":1}}],["779437f",{"2":{"256":1}}],["7794657",{"2":{"126":1}}],["7795275",{"2":{"241":2}}],["77778",{"2":{"217":2}}],["77",{"2":{"196":2,"217":1,"228":1}}],["776809f",{"2":{"256":1}}],["7766",{"2":{"171":1}}],["776598",{"2":{"171":2}}],["77207",{"2":{"148":1}}],["770151847147264e",{"2":{"258":1}}],["7701346738750905",{"2":{"153":2}}],["7703",{"2":{"249":1}}],["770373f",{"2":{"256":1}}],["77037",{"2":{"147":1}}],["7700396",{"2":{"147":1}}],["7157686837513434e",{"2":{"258":1}}],["715798f",{"2":{"256":1}}],["7107122439087137e",{"2":{"258":1}}],["710968f",{"2":{"256":1}}],["711118f",{"2":{"256":1}}],["711329",{"2":{"168":1}}],["719830463519086e",{"2":{"258":1}}],["719233528922508e",{"2":{"258":1}}],["7192734f",{"2":{"256":1}}],["7196598f",{"2":{"256":1}}],["7190236f",{"2":{"256":1}}],["7199144f",{"2":{"256":1}}],["7199",{"2":{"249":1}}],["7195462169922173",{"2":{"153":1}}],["7195462169922175",{"2":{"153":1}}],["7149",{"2":{"249":1}}],["7121",{"2":{"249":1}}],["7127",{"2":{"249":1}}],["7123551",{"2":{"241":1}}],["712979078066394",{"2":{"153":2}}],["7185877115678053e",{"2":{"258":1}}],["7185877f",{"2":{"256":1}}],["7186",{"2":{"249":1}}],["7183194",{"2":{"241":1}}],["7181164",{"2":{"145":1}}],["71",{"2":{"196":1,"217":2,"228":7}}],["717863343114228",{"2":{"153":2}}],["7166861569291764e",{"2":{"258":1}}],["7166952f",{"2":{"256":1}}],["7165354",{"2":{"241":2}}],["7165512278088038",{"2":{"153":2}}],["716134755899883",{"2":{"153":2}}],["713",{"2":{"245":1}}],["713567516238075",{"2":{"153":2}}],["713316",{"2":{"106":1}}],["749413041404722e",{"2":{"258":1}}],["7495777f",{"2":{"256":1}}],["7419091598444446e",{"2":{"258":1}}],["741270119787773e",{"2":{"258":1}}],["741251439864033e",{"2":{"258":1}}],["7412987f",{"2":{"256":1}}],["741229f",{"2":{"256":1}}],["741348",{"2":{"241":1}}],["74475873611945e",{"2":{"258":1}}],["7449265f",{"2":{"256":1}}],["744489",{"2":{"145":1}}],["7474",{"2":{"249":1}}],["742316877723749e",{"2":{"258":1}}],["742277f",{"2":{"256":1}}],["7426",{"2":{"249":1}}],["7429947",{"2":{"129":1}}],["748614211944747e",{"2":{"258":1}}],["74888813316211e",{"2":{"258":1}}],["74887",{"2":{"168":1}}],["748379f",{"2":{"256":1}}],["7480315",{"2":{"241":2}}],["740307200263535e",{"2":{"258":1}}],["7405424f",{"2":{"256":1}}],["7401575",{"2":{"241":2}}],["740140e",{"2":{"202":1}}],["74074",{"2":{"217":3}}],["7402",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["74",{"2":{"196":1,"217":5,"228":2}}],["746753172601335e",{"2":{"258":1}}],["746762f",{"2":{"256":1}}],["7468008914093891",{"2":{"174":4}}],["746801",{"2":{"171":1}}],["74694291453392",{"2":{"174":4}}],["746943",{"2":{"171":1}}],["704380322734901e",{"2":{"258":1}}],["7043405f",{"2":{"256":1}}],["7024",{"2":{"249":1}}],["70866144",{"2":{"241":2}}],["7089496198123055",{"2":{"153":2}}],["7050s\\ttraining",{"2":{"217":1}}],["7054554",{"2":{"147":1}}],["707946f",{"2":{"256":1}}],["7072s\\ttraining",{"2":{"217":1}}],["707534",{"2":{"171":1}}],["70",{"2":{"217":1,"228":1}}],["70s",{"2":{"196":1}}],["7018592",{"2":{"145":1}}],["70370",{"2":{"217":1}}],["70390546",{"2":{"126":1}}],["70330536",{"2":{"77":1}}],["70020412108269e",{"2":{"258":1}}],["700688415124854e",{"2":{"258":1}}],["7006324f",{"2":{"256":1}}],["7000513f",{"2":{"256":1}}],["7004334f",{"2":{"256":1}}],["7001468",{"2":{"241":1}}],["7001",{"2":{"179":1,"236":1}}],["700",{"2":{"107":1}}],["722913111142489e",{"2":{"258":1}}],["722839f",{"2":{"256":1}}],["72137428008044e",{"2":{"258":1}}],["7214146f",{"2":{"256":1}}],["725419114659701e",{"2":{"258":1}}],["7255755f",{"2":{"256":1}}],["7252903f",{"2":{"106":8}}],["727649944397793e",{"2":{"258":1}}],["7274265f",{"2":{"256":1}}],["727885f",{"2":{"256":1}}],["7273",{"2":{"249":1}}],["723422966062034e",{"2":{"258":1}}],["723484f",{"2":{"256":1}}],["7237308297251812",{"2":{"174":1}}],["72373",{"2":{"171":1}}],["723731",{"2":{"171":2}}],["7200",{"2":{"249":1}}],["72",{"2":{"217":3,"228":1}}],["72s",{"2":{"196":1}}],["726525218174617e",{"2":{"258":1}}],["72669f",{"2":{"256":1}}],["726115",{"2":{"168":1}}],["7262075",{"2":{"147":1}}],["724508877495954e",{"2":{"258":1}}],["724518f",{"2":{"256":1}}],["724099",{"2":{"168":1}}],["72471225",{"2":{"147":1}}],["72428167",{"2":{"126":1}}],["72908926",{"2":{"148":1}}],["7296474",{"2":{"147":1}}],["7292344",{"2":{"145":1}}],["729",{"2":{"77":6}}],["798337146897226e",{"2":{"258":1}}],["798027",{"2":{"147":1}}],["790873702257031e",{"2":{"258":1}}],["799489273675791e",{"2":{"258":1}}],["7995953f",{"2":{"256":1}}],["7990667",{"2":{"244":1}}],["7919546091982902e",{"2":{"258":1}}],["7917452f",{"2":{"256":1}}],["7917748f",{"2":{"256":1}}],["7911913230679333e",{"2":{"258":1}}],["7911294f",{"2":{"256":1}}],["7911112",{"2":{"147":1}}],["791447f",{"2":{"256":1}}],["797467871033851e",{"2":{"258":1}}],["7970278876400933e",{"2":{"258":1}}],["797303f",{"2":{"256":1}}],["7977",{"2":{"249":1}}],["7966732093004386e",{"2":{"258":1}}],["7966196f",{"2":{"256":1}}],["7964175f",{"2":{"256":1}}],["79656f",{"2":{"77":1}}],["7956222344861515e",{"2":{"258":1}}],["7954133544459536e",{"2":{"258":1}}],["7955612f",{"2":{"256":1}}],["7953574f",{"2":{"256":1}}],["794150635463917e",{"2":{"258":1}}],["7942",{"2":{"249":1}}],["794908",{"2":{"147":1}}],["79",{"2":{"196":1,"217":4,"228":2,"249":1}}],["79250443",{"2":{"147":1}}],["793481744757494e",{"2":{"258":1}}],["793463f",{"2":{"77":1}}],["7937424f",{"2":{"256":1}}],["793151f",{"2":{"256":1}}],["793159",{"2":{"171":1}}],["7939677f",{"2":{"106":1}}],["786793890260071e",{"2":{"258":1}}],["785104734791074e",{"2":{"258":1}}],["785261f",{"2":{"256":1}}],["7853945f",{"2":{"256":1}}],["7833874f",{"2":{"256":1}}],["7834",{"2":{"249":1}}],["7834251",{"2":{"147":1}}],["787880135302134e",{"2":{"258":1}}],["787567f",{"2":{"256":1}}],["787202f",{"2":{"256":1}}],["787159",{"2":{"145":1}}],["7829985725519594e",{"2":{"258":1}}],["7826983568923675e",{"2":{"258":1}}],["7826984f",{"2":{"256":1}}],["78226817",{"2":{"135":1}}],["78908600843019e",{"2":{"258":1}}],["789192f",{"2":{"256":1}}],["7891",{"2":{"249":1}}],["7897024",{"2":{"241":1}}],["78818554",{"2":{"241":1}}],["78814f",{"2":{"106":1}}],["78",{"2":{"196":1,"217":4,"219":1,"228":10}}],["7817315740767116",{"2":{"174":1}}],["7811481",{"2":{"145":1}}],["78000563",{"2":{"136":1}}],["7808904",{"2":{"74":2}}],["7842128027304344e",{"2":{"258":1}}],["7849862038479912e",{"2":{"258":1}}],["7841387f",{"2":{"256":1}}],["784768",{"2":{"147":1,"148":1}}],["784",{"2":{"55":6,"215":1,"220":6,"226":1}}],["7",{"2":{"19":9,"48":4,"63":5,"77":17,"106":10,"145":5,"147":2,"148":5,"152":1,"173":3,"179":1,"180":4,"187":14,"189":4,"196":2,"197":1,"202":1,"204":4,"217":5,"219":1,"221":4,"228":2,"229":4,"238":4,"241":5,"246":4,"249":6,"251":1,"256":75,"258":70,"260":1}}],["5g",{"2":{"245":1}}],["5gb",{"2":{"180":1,"189":1,"204":1,"221":1,"246":1}}],["5fs",{"2":{"228":1}}],["5f",{"2":{"187":3,"217":2}}],["5f0",{"2":{"126":1,"129":1,"152":1,"186":1,"248":12}}],["543181105451772e",{"2":{"258":1}}],["5430165f",{"2":{"256":1}}],["545963694062543e",{"2":{"258":1}}],["5453541817767217e",{"2":{"258":1}}],["547369202278632e",{"2":{"258":1}}],["547049137928025e",{"2":{"258":1}}],["5476045f",{"2":{"256":1}}],["5476424498276177",{"2":{"174":4}}],["547642",{"2":{"171":1}}],["544035526592878e",{"2":{"258":1}}],["5442154f",{"2":{"256":1}}],["54452",{"2":{"172":1}}],["546131652318562e",{"2":{"258":1}}],["54681844470009e",{"2":{"258":1}}],["546048f",{"2":{"256":1}}],["5460823f",{"2":{"256":1}}],["5469855f",{"2":{"256":1}}],["54691",{"2":{"187":1}}],["5467187f",{"2":{"256":1}}],["548698094675063e",{"2":{"258":1}}],["548707f",{"2":{"256":1}}],["54823",{"2":{"187":1}}],["5424696",{"2":{"241":1}}],["549",{"2":{"229":1,"238":1}}],["540684460542714e",{"2":{"258":1}}],["540366429758654e",{"2":{"258":1}}],["5403886f",{"2":{"256":1}}],["540",{"2":{"220":8}}],["54",{"2":{"217":1,"228":2}}],["5×3",{"2":{"171":8,"172":1}}],["5×5",{"2":{"28":1,"171":3}}],["58587310390065e",{"2":{"258":1}}],["5858413f",{"2":{"256":1}}],["585174f",{"2":{"256":1}}],["585929f",{"2":{"256":1}}],["5852299069862294e",{"2":{"258":1}}],["5852",{"2":{"249":1}}],["586542847795008e",{"2":{"258":1}}],["5865265807660498",{"2":{"153":2}}],["5862495801101184e",{"2":{"258":1}}],["586674812241179e",{"2":{"258":1}}],["586675f",{"2":{"256":1}}],["586627f",{"2":{"256":1}}],["5847995800928283e",{"2":{"258":1}}],["584730802686131e",{"2":{"258":1}}],["5847308f",{"2":{"256":1}}],["5847257f",{"2":{"256":1}}],["5840",{"2":{"249":1}}],["5839544373761774e",{"2":{"258":1}}],["58391",{"2":{"187":1}}],["58324503875292e",{"2":{"258":1}}],["583864f",{"2":{"256":1}}],["5834",{"2":{"249":1}}],["5812",{"2":{"249":1}}],["582849052010059e",{"2":{"258":1}}],["582888f",{"2":{"256":1}}],["5821",{"2":{"249":1}}],["58263564",{"2":{"244":1}}],["5826772",{"2":{"241":2}}],["5805858326411852e",{"2":{"258":1}}],["5808926f",{"2":{"256":1}}],["5806152f",{"2":{"256":1}}],["5806",{"2":{"249":1}}],["5809",{"2":{"249":1}}],["58",{"2":{"217":3,"219":1,"228":1}}],["58s",{"2":{"196":1}}],["587083052398726e",{"2":{"258":1}}],["587022f",{"2":{"256":1}}],["58721",{"2":{"171":1}}],["58720636",{"2":{"171":1}}],["58720636f0",{"2":{"171":1}}],["587206",{"2":{"171":2}}],["5878153",{"2":{"147":1}}],["5879354f",{"2":{"106":1}}],["523991656924728e",{"2":{"258":1}}],["5237564f",{"2":{"256":1}}],["525685027746876e",{"2":{"258":1}}],["5257072f",{"2":{"256":1}}],["52874031880243e",{"2":{"258":1}}],["528666f",{"2":{"256":1}}],["5281",{"2":{"74":2}}],["529384870565706e",{"2":{"258":1}}],["529854110296813e",{"2":{"258":1}}],["529197184863753e",{"2":{"258":1}}],["529565f",{"2":{"256":1}}],["529562f",{"2":{"256":1}}],["5294324f",{"2":{"256":1}}],["52628386",{"2":{"241":1}}],["5267549745189349",{"2":{"153":2}}],["52024",{"2":{"241":1}}],["527559",{"2":{"241":2}}],["52717",{"2":{"147":1}}],["52",{"2":{"217":1}}],["5218s\\ttraining",{"2":{"217":1}}],["5210742834996898",{"2":{"153":2}}],["5221656",{"2":{"179":1}}],["522139",{"2":{"148":1}}],["5242909957219376e",{"2":{"258":1}}],["5243203f",{"2":{"256":1}}],["52434",{"2":{"187":1}}],["524008",{"2":{"171":1}}],["524781",{"2":{"148":1}}],["5731702796634066e",{"2":{"258":1}}],["5734184f",{"2":{"256":1}}],["5752533863499285e",{"2":{"258":1}}],["575224f",{"2":{"256":1}}],["575809f",{"2":{"256":1}}],["576203407699251e",{"2":{"258":1}}],["576237f",{"2":{"256":1}}],["57671577",{"2":{"137":1}}],["570196224301594e",{"2":{"258":1}}],["5704723128396553e",{"2":{"258":1}}],["5704822f",{"2":{"256":1}}],["5703",{"2":{"249":1}}],["57142954495314e",{"2":{"258":1}}],["571922f",{"2":{"256":1}}],["5715973f",{"2":{"256":1}}],["571883067229534e",{"2":{"258":1}}],["5718",{"2":{"254":2}}],["571838e",{"2":{"202":1}}],["5794",{"2":{"249":1}}],["57882756",{"2":{"241":1}}],["578374863",{"2":{"236":2}}],["57",{"2":{"217":5,"228":1}}],["577116166104724e",{"2":{"258":1}}],["5771772f",{"2":{"256":1}}],["577181",{"2":{"171":1}}],["57736",{"2":{"187":1}}],["5779067839062536",{"2":{"153":2}}],["5776367e",{"2":{"148":1}}],["5776052f0",{"2":{"63":1}}],["5642660290027388e",{"2":{"258":1}}],["564832f",{"2":{"256":1}}],["565123976295764e",{"2":{"258":1}}],["561653f",{"2":{"256":1}}],["561137f",{"2":{"106":1}}],["5681158103063064e",{"2":{"258":1}}],["5685252152966485e",{"2":{"258":1}}],["5682064f",{"2":{"256":1}}],["5680085614623662",{"2":{"153":2}}],["569484790716378e",{"2":{"258":1}}],["5691726f",{"2":{"256":1}}],["569154f",{"2":{"256":1}}],["5692906f",{"2":{"256":1}}],["5623467412146784e",{"2":{"258":1}}],["5625114f",{"2":{"256":1}}],["5627227",{"2":{"145":1}}],["5668716f",{"2":{"256":1}}],["5663",{"2":{"249":1}}],["56",{"2":{"217":1,"228":4}}],["56702805278856e",{"2":{"258":1}}],["5675401f",{"2":{"256":1}}],["5675557670686644",{"2":{"174":1}}],["5673",{"2":{"249":1}}],["56779",{"2":{"171":1}}],["567794",{"2":{"171":2}}],["560794e",{"2":{"202":1}}],["560",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["56055",{"2":{"171":1}}],["56069607",{"2":{"106":1}}],["5937554825901e",{"2":{"258":1}}],["5937278842412807e",{"2":{"258":1}}],["593723f",{"2":{"256":1}}],["59399403006389e",{"2":{"258":1}}],["5934998f",{"2":{"256":1}}],["5938376f",{"2":{"256":1}}],["5938956f",{"2":{"256":1}}],["590096764028852e",{"2":{"258":1}}],["590735504586477e",{"2":{"258":1}}],["590181f",{"2":{"256":1}}],["590809f",{"2":{"256":1}}],["5905511",{"2":{"241":2}}],["594661298169121e",{"2":{"258":1}}],["5948909956864962e",{"2":{"258":1}}],["594587f",{"2":{"256":1}}],["5949308f",{"2":{"256":1}}],["594261",{"2":{"148":1}}],["5956",{"2":{"249":1}}],["59543324",{"2":{"244":1}}],["59259",{"2":{"219":1}}],["598161470656944e",{"2":{"258":1}}],["5981s\\ttraining",{"2":{"217":1}}],["598046f",{"2":{"256":1}}],["598043f",{"2":{"256":1}}],["598338328679915e",{"2":{"258":1}}],["5983",{"2":{"249":1}}],["598505020",{"2":{"236":2}}],["5984s\\ttraining",{"2":{"217":1}}],["59",{"2":{"148":1,"196":1,"228":2}}],["597602022491649e",{"2":{"258":1}}],["597894f",{"2":{"256":1}}],["5979974",{"2":{"145":1}}],["5975106",{"2":{"77":1}}],["59984816741856e",{"2":{"258":1}}],["5995452626037e",{"2":{"258":1}}],["5991",{"2":{"249":1}}],["5999714",{"2":{"126":1}}],["5994072",{"2":{"106":1}}],["5994074",{"2":{"106":1}}],["5395698882187308e",{"2":{"258":1}}],["5326060372288575e",{"2":{"258":1}}],["537753f",{"2":{"256":1}}],["5378918",{"2":{"77":1}}],["534096345463743e",{"2":{"258":1}}],["534006f",{"2":{"256":1}}],["5345716",{"2":{"77":1}}],["530184260451466e",{"2":{"258":1}}],["530175f",{"2":{"256":1}}],["530635098688162e",{"2":{"258":1}}],["530635f",{"2":{"256":1}}],["538551986733346e",{"2":{"258":1}}],["538478f",{"2":{"256":1}}],["5386",{"2":{"249":1}}],["5381101016248151",{"2":{"153":2}}],["531887714956414e",{"2":{"258":1}}],["5318069357061005e",{"2":{"258":1}}],["531554f",{"2":{"256":1}}],["5316505f",{"2":{"256":1}}],["53167",{"2":{"187":1}}],["5310851",{"2":{"179":1}}],["53144f",{"2":{"77":1}}],["536413597195021e",{"2":{"258":1}}],["5365934f",{"2":{"256":1}}],["5365157",{"2":{"148":1}}],["536743e",{"2":{"152":1}}],["5367613",{"2":{"147":1}}],["53",{"2":{"110":1,"228":1,"249":1}}],["5∗δ",{"2":{"63":1}}],["5∗|y−y^|2if",{"2":{"63":1}}],["550988215516858e",{"2":{"258":1}}],["5506959f",{"2":{"256":1}}],["5501",{"2":{"236":1}}],["5524723f",{"2":{"256":1}}],["556705254773653e",{"2":{"258":1}}],["556034612313352e",{"2":{"258":1}}],["5560125f",{"2":{"256":1}}],["5564",{"2":{"249":1}}],["55658394",{"2":{"241":1}}],["55469999437402e",{"2":{"258":1}}],["554471f",{"2":{"256":1}}],["5541",{"2":{"249":1}}],["55476f",{"2":{"77":1}}],["551667591807606e",{"2":{"258":1}}],["551033116481576e",{"2":{"258":1}}],["5515529326924823e",{"2":{"258":1}}],["5518474f",{"2":{"256":1}}],["55185s\\ttraining",{"2":{"228":1}}],["5511866f",{"2":{"256":1}}],["5511811",{"2":{"241":2}}],["551152f",{"2":{"256":1}}],["5519417f",{"2":{"256":1}}],["555637038745822e",{"2":{"258":1}}],["5557935f",{"2":{"256":1}}],["5557236f",{"2":{"256":1}}],["555567189552139e",{"2":{"258":1}}],["55556",{"2":{"217":2}}],["5553797706980106",{"2":{"174":1}}],["5594571",{"2":{"241":1}}],["5590551",{"2":{"241":2}}],["5595843665394066",{"2":{"174":1}}],["55989707",{"2":{"148":1}}],["5537708314034874e",{"2":{"258":1}}],["553532171218581e",{"2":{"258":1}}],["5539273f",{"2":{"256":1}}],["553496f",{"2":{"256":1}}],["55321077984955e",{"2":{"258":1}}],["5532",{"2":{"171":1}}],["553631",{"2":{"147":1}}],["5530689",{"2":{"147":1}}],["55",{"2":{"63":1,"196":1,"219":1}}],["5where",{"2":{"63":1}}],["5d",{"2":{"56":1,"202":1,"236":2,"257":1}}],["5108233386230962e",{"2":{"258":1}}],["5108455f",{"2":{"256":1}}],["5182278f",{"2":{"256":1}}],["51852",{"2":{"217":1}}],["5172274497928558e",{"2":{"258":1}}],["5171533f",{"2":{"256":1}}],["5178",{"2":{"249":1}}],["5118190320989156e",{"2":{"258":1}}],["511986f",{"2":{"256":1}}],["51165134",{"2":{"244":1}}],["5111s\\ttraining",{"2":{"217":1}}],["51968503",{"2":{"241":2}}],["515201207261988e",{"2":{"258":1}}],["51524514",{"2":{"179":1}}],["515369f",{"2":{"256":1}}],["5156s\\ttraining",{"2":{"217":1}}],["5167208",{"2":{"244":1}}],["51671",{"2":{"187":1}}],["5164032",{"2":{"179":1}}],["5124s\\ttraining",{"2":{"217":1}}],["512",{"2":{"69":3,"235":1}}],["51",{"2":{"55":3,"109":1,"202":2,"228":2,"245":1}}],["509565277511446e",{"2":{"258":1}}],["509532f",{"2":{"256":1}}],["508029397665399e",{"2":{"258":1}}],["508659549173389e",{"2":{"258":1}}],["5086595f",{"2":{"256":1}}],["5036827f",{"2":{"256":1}}],["503186",{"2":{"172":1}}],["5031421",{"2":{"106":1}}],["507989f",{"2":{"256":1}}],["50798f",{"2":{"77":1}}],["5066772933633293e",{"2":{"258":1}}],["506195040126744e",{"2":{"258":1}}],["5061914f",{"2":{"256":1}}],["5064943f",{"2":{"256":1}}],["506485",{"2":{"172":1}}],["5067176f",{"2":{"256":1}}],["505843671305004e",{"2":{"258":1}}],["5058837933777822e",{"2":{"258":1}}],["505687f",{"2":{"256":1}}],["5053",{"2":{"249":1}}],["505488",{"2":{"168":1}}],["5010715f",{"2":{"256":1}}],["501",{"2":{"236":1}}],["5016017",{"2":{"147":1}}],["50498",{"2":{"187":1}}],["500496522416472e",{"2":{"258":1}}],["500759539220936e",{"2":{"258":1}}],["500241f",{"2":{"256":1}}],["500523f",{"2":{"256":1}}],["5005s\\ttraining",{"2":{"217":1}}],["500987535104989e",{"2":{"258":1}}],["500931079654415e",{"2":{"258":1}}],["500979971255219e",{"2":{"258":1}}],["500971f",{"2":{"256":1}}],["5009556",{"2":{"244":1}}],["5000×30×1",{"2":{"249":1}}],["50000",{"2":{"236":100}}],["5000",{"2":{"229":2,"236":1,"238":2,"249":5}}],["5001",{"2":{"179":1,"236":1}}],["500",{"2":{"107":1,"183":2,"236":1}}],["502312852219817",{"2":{"63":1}}],["50",{"2":{"49":2,"55":3,"196":13,"228":110,"245":1,"249":1}}],["5",{"2":{"19":9,"28":8,"39":2,"46":4,"49":2,"55":6,"63":15,"69":11,"74":1,"77":16,"97":1,"102":1,"106":5,"115":5,"129":1,"130":1,"147":1,"148":2,"168":6,"171":29,"172":1,"173":3,"177":3,"179":6,"180":3,"187":14,"189":3,"193":14,"196":2,"199":1,"200":1,"202":1,"204":3,"217":5,"218":2,"219":1,"220":40,"221":3,"228":2,"229":3,"231":1,"238":3,"241":8,"245":2,"246":3,"248":4,"249":19,"250":1,"254":4,"255":2,"256":795,"258":754,"259":3}}],["^2",{"2":{"255":2,"256":1}}],["^",{"2":{"19":9,"69":1,"78":1,"106":1,"107":1,"254":2}}],["λβ",{"2":{"17":1}}],["α",{"2":{"17":3,"63":3,"200":2,"250":3}}],["+=",{"2":{"194":2,"216":2,"227":2,"236":1,"249":1}}],["+ϵ∗γ+βwhere",{"2":{"55":1}}],["+ϵ∗γ+βand",{"2":{"19":1}}],["+",{"2":{"16":1,"18":1,"28":3,"39":1,"44":1,"48":5,"49":4,"51":6,"53":4,"56":1,"59":2,"62":1,"65":1,"69":4,"88":1,"135":2,"145":1,"146":1,"158":2,"171":4,"179":3,"183":2,"200":1,"234":4,"235":3,"237":1,"241":1,"248":4,"249":1,"254":13,"255":1,"256":2}}],["y=x2−2x",{"2":{"241":1}}],["y=x−e",{"2":{"19":1,"55":1}}],["ylabel=",{"2":{"237":1,"241":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["ys",{"2":{"237":6}}],["y∈",{"2":{"235":1}}],["yᵢ",{"2":{"107":2}}],["yes",{"2":{"82":1}}],["year",{"2":{"74":2}}],["yet",{"2":{"44":1,"106":1}}],["yann",{"2":{"63":1}}],["y+ϵ",{"2":{"63":1}}],["yi∈rm",{"2":{"179":1}}],["yi",{"2":{"63":2,"179":1}}],["y~=",{"2":{"63":2}}],["y~",{"2":{"63":4}}],["y^2+y∗max",{"2":{"63":1}}],["y^−y∗log⁡",{"2":{"63":1}}],["y^−y",{"2":{"63":1}}],["y^",{"2":{"63":7}}],["y^+ϵ",{"2":{"63":3}}],["ŷ",{"2":{"63":11,"145":2,"146":2,"147":2,"186":2,"187":3}}],["y3",{"2":{"48":1}}],["y2",{"2":{"48":1,"248":2}}],["y1",{"2":{"48":1,"248":2}}],["yoshua",{"2":{"28":2}}],["yourself",{"2":{"210":1}}],["your",{"0":{"120":1},"2":{"44":2,"55":3,"57":1,"69":2,"85":1,"108":2,"110":1,"124":1,"133":1,"135":2,"136":2,"137":3,"143":2,"144":1,"145":2,"157":1,"161":1,"163":2,"188":1}}],["you",{"2":{"2":1,"6":2,"40":2,"44":1,"57":1,"58":1,"62":3,"69":5,"71":3,"74":2,"76":3,"77":1,"78":1,"79":1,"97":1,"105":1,"106":2,"107":1,"108":3,"109":2,"110":2,"111":1,"120":2,"124":1,"127":1,"133":1,"135":8,"136":6,"137":5,"139":2,"143":2,"144":5,"145":4,"146":1,"153":1,"156":1,"157":2,"163":2,"168":1,"171":1,"172":3,"173":1,"175":1,"181":1,"185":1,"188":1,"209":1,"210":1,"217":2,"225":1,"235":1,"242":1,"256":1}}],["yuxin",{"2":{"19":1}}],["y",{"2":{"14":2,"15":2,"19":1,"23":5,"28":3,"48":8,"49":2,"51":9,"53":10,"54":6,"55":4,"59":3,"63":63,"64":3,"65":6,"68":1,"69":10,"77":1,"78":4,"106":11,"107":2,"110":2,"117":4,"135":3,"136":4,"145":7,"146":7,"153":2,"160":5,"179":11,"183":4,"184":5,"185":3,"186":12,"187":5,"192":6,"194":2,"195":4,"200":4,"213":6,"216":2,"217":2,"224":4,"225":2,"227":2,"228":2,"235":7,"237":1,"241":5,"245":5,"248":1,"254":6,"256":3}}],["σ=identity",{"2":{"19":1}}],["σ",{"2":{"13":10,"15":5,"16":3,"18":3,"19":7,"158":3}}],["know",{"2":{"104":1,"109":1,"144":1,"256":1}}],["known",{"0":{"124":1},"2":{"28":2,"62":1,"104":1}}],["knet",{"2":{"81":1}}],["kwarg",{"2":{"97":2,"98":1}}],["kwargs",{"2":{"29":24,"32":2,"40":1,"48":9,"54":2,"59":4,"69":3,"98":1,"168":2,"169":2,"195":1,"214":7,"217":2,"218":4,"220":4}}],["kw",{"2":{"69":2}}],["kullback",{"2":{"63":1}}],["kl",{"2":{"63":1}}],["kldivergenceloss",{"2":{"63":5}}],["klambauer",{"2":{"17":1}}],["k",{"2":{"49":6,"53":4,"179":1,"214":2,"218":2}}],["ki−1",{"2":{"49":1,"51":3}}],["kind",{"0":{"126":1},"2":{"41":1,"46":1,"48":1,"69":2,"109":1,"126":1}}],["kinds",{"2":{"8":1}}],["kiros",{"2":{"19":1}}],["kaiming",{"2":{"19":1,"28":4,"49":2,"53":1,"168":6}}],["keith",{"2":{"252":1}}],["keep",{"2":{"142":1}}],["kept",{"2":{"139":1}}],["kernelabstractions",{"2":{"20":1,"153":7}}],["kernels",{"0":{"153":1},"2":{"16":1,"153":3}}],["kernel",{"2":{"16":1,"20":1,"28":1,"49":5,"153":6}}],["keys",{"2":{"97":1,"115":1,"138":2}}],["keypath=keypath",{"2":{"40":1}}],["keypath",{"2":{"39":7,"97":1,"98":1,"109":4,"110":10,"127":2,"128":4}}],["key",{"0":{"123":1},"2":{"10":2,"119":1}}],["keywords",{"2":{"56":1,"69":1,"169":1}}],["keyword",{"2":{"2":1,"10":1,"40":2,"44":1,"48":5,"49":2,"50":4,"51":3,"52":2,"53":4,"54":2,"55":4,"56":2,"58":1,"59":2,"87":1,"99":1,"100":1,"169":1}}],["hₓ",{"2":{"254":4}}],["h₊",{"2":{"254":4}}],["h12",{"2":{"254":12}}],["h11",{"2":{"254":12}}],["h22",{"2":{"254":12}}],["hmc",{"2":{"249":2}}],["hcat",{"2":{"235":1,"249":1}}],["hn",{"2":{"225":2}}],["hnew",{"2":{"52":6}}],["hnew=activation",{"2":{"52":1}}],["hnew=",{"2":{"52":1}}],["hypernet",{"0":{"225":1,"226":1},"2":{"225":4,"226":1}}],["hypernetwork",{"0":{"222":1},"1":{"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1}}],["home",{"2":{"137":1}}],["hosts",{"2":{"79":1}}],["hold",{"2":{"77":1,"82":1}}],["hot",{"2":{"65":1}}],["how",{"0":{"71":1,"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"46":1,"48":2,"52":3,"55":8,"63":2,"65":1,"76":1,"77":1,"83":1,"106":1,"107":1,"108":1,"110":2,"125":1,"135":1,"137":1,"140":1,"144":1,"148":1,"153":1,"157":1,"161":2,"171":2,"177":1,"190":1,"198":1,"214":1,"225":1,"247":1,"249":1,"250":1}}],["however",{"2":{"8":1,"11":1,"48":1,"65":1,"69":1,"77":1,"81":1,"99":1,"105":1,"111":1,"123":1,"148":1,"167":1,"173":2,"175":1,"198":3,"209":1,"214":1,"230":1,"254":1,"256":2}}],["hutchinson",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":2,"150":3,"151":2,"152":8}}],["huber",{"2":{"63":1}}],["huberloss",{"2":{"63":3}}],["human",{"2":{"28":2}}],["hh",{"2":{"52":6,"100":1}}],["h",{"2":{"51":6,"52":3,"56":4,"192":1,"213":1,"254":14}}],["https",{"2":{"28":1,"52":1,"54":1,"71":1,"74":1,"177":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"254":2,"260":1}}],["heatmap",{"2":{"237":1}}],["head",{"2":{"83":1,"184":1}}],["heavily",{"2":{"68":1}}],["heavy",{"2":{"6":1}}],["height",{"2":{"49":1}}],["hence",{"2":{"44":1,"57":1,"140":1,"142":1,"150":1,"174":1,"175":1,"196":1,"202":1,"214":1,"256":1}}],["helps",{"2":{"214":1}}],["help",{"2":{"37":1,"38":1,"39":1,"40":2,"48":6,"49":2,"51":3,"52":2,"54":1,"55":4,"56":1,"63":2,"67":1,"69":1,"108":1,"157":1}}],["helpers",{"0":{"21":1},"1":{"22":1,"23":1,"24":1},"2":{"64":1}}],["helper",{"0":{"20":1,"33":1,"54":1,"194":1},"2":{"62":1,"67":1,"250":1}}],["here",{"2":{"37":1,"40":1,"62":2,"64":1,"69":3,"77":1,"81":1,"104":1,"114":1,"120":1,"136":1,"145":3,"146":1,"147":1,"160":1,"161":1,"171":3,"177":2,"202":1,"214":1,"233":1,"248":1,"249":2}}],["hessian",{"2":{"28":1}}],["he",{"2":{"19":1,"28":2}}],["historical",{"2":{"214":1}}],["historically",{"2":{"37":1}}],["hierarchically",{"2":{"69":1}}],["hinge",{"2":{"63":2}}],["hingeloss",{"2":{"63":2}}],["hinton",{"2":{"19":1}}],["hidden",{"2":{"52":43,"184":6,"185":3,"233":11,"236":2,"249":1}}],["highlight",{"2":{"123":1}}],["highly",{"2":{"63":1}}],["higher",{"2":{"56":1,"147":1,"155":1}}],["highest",{"2":{"16":1,"104":1,"155":1,"250":1}}],["high",{"2":{"48":1,"153":1,"171":1,"177":1}}],["hi",{"2":{"28":2}}],["hit",{"2":{"8":1}}],["hamiltonian",{"2":{"249":2}}],["hat",{"2":{"203":2}}],["had",{"2":{"175":1}}],["hadsell",{"2":{"63":1}}],["hand",{"2":{"171":1}}],["handling",{"2":{"64":1,"67":1,"133":1,"155":1,"165":1}}],["handles",{"2":{"83":1,"185":1}}],["handle",{"2":{"52":1,"130":1,"153":1,"214":1,"249":1}}],["happened",{"2":{"155":1}}],["happening",{"2":{"146":1,"160":1}}],["happens",{"2":{"109":1,"144":1,"160":1,"165":1}}],["haven",{"2":{"44":1}}],["have",{"2":{"23":1,"28":1,"37":2,"49":1,"52":1,"54":1,"83":2,"84":1,"90":1,"94":2,"97":3,"98":1,"99":2,"100":1,"104":3,"105":1,"109":1,"110":2,"111":2,"124":1,"135":1,"136":1,"137":1,"139":2,"143":1,"144":2,"145":1,"146":1,"152":1,"155":1,"172":1,"173":1,"174":1,"175":3,"184":3,"202":1,"226":1,"249":1,"250":2,"254":1}}],["having",{"2":{"6":1,"41":1,"69":1}}],["harder",{"2":{"83":1,"108":1}}],["hardware",{"2":{"82":1}}],["hard",{"2":{"11":1,"63":1,"110":1,"135":1,"180":2,"189":2,"197":1,"204":2,"221":2,"229":2,"238":2,"246":2,"251":1,"260":1}}],["has",{"2":{"8":3,"38":1,"44":2,"49":1,"51":3,"56":1,"69":1,"81":1,"83":1,"87":2,"90":6,"94":1,"97":7,"98":4,"99":1,"100":1,"108":1,"109":4,"119":1,"122":1,"130":1,"136":1,"148":1,"170":1,"171":2,"175":1,"249":1,"252":1,"256":1}}],["2f",{"2":{"195":2,"228":4}}],["2fs",{"2":{"195":1}}],["2`",{"2":{"136":1}}],["2732927063860792e",{"2":{"258":1}}],["27373654",{"2":{"126":1}}],["2729995f",{"2":{"256":1}}],["278945542483876e",{"2":{"258":1}}],["278971733222418e",{"2":{"258":1}}],["278149249557925e",{"2":{"258":1}}],["278153f",{"2":{"256":1}}],["278327363249883e",{"2":{"258":1}}],["2783274f",{"2":{"256":1}}],["2788714f",{"2":{"256":1}}],["278853f",{"2":{"256":1}}],["271804261219849e",{"2":{"258":1}}],["2718603f",{"2":{"256":1}}],["2714",{"2":{"249":1}}],["2717",{"2":{"249":1}}],["27471745",{"2":{"244":1}}],["27",{"2":{"217":3,"228":2,"249":1}}],["276791524081877e",{"2":{"258":1}}],["276754f",{"2":{"256":1}}],["276788f",{"2":{"256":1}}],["276838f",{"2":{"256":1}}],["276",{"2":{"202":1}}],["2755905",{"2":{"241":2}}],["27501",{"2":{"236":1}}],["27590",{"2":{"187":1}}],["27545732",{"2":{"106":1}}],["2700544089169337e",{"2":{"258":1}}],["27001",{"2":{"236":1}}],["2709",{"2":{"249":1}}],["2701178",{"2":{"241":1}}],["27035674",{"2":{"179":1}}],["27051234",{"2":{"179":1}}],["2792237406003304e",{"2":{"258":1}}],["2791332f",{"2":{"256":1}}],["2791573282471997",{"2":{"153":2}}],["27993",{"2":{"172":1}}],["2797048270773437",{"2":{"153":2}}],["277047259411739e",{"2":{"258":1}}],["27708576306609e",{"2":{"258":1}}],["2771789",{"2":{"179":1}}],["2771862617010155",{"2":{"153":2}}],["2774315",{"2":{"145":1}}],["27th",{"2":{"28":1}}],["2×4",{"2":{"173":1}}],["2×5",{"2":{"168":6}}],["2×32",{"2":{"106":2}}],["2×2",{"2":{"63":2,"171":1}}],["29481262530745e",{"2":{"258":1}}],["2947895f",{"2":{"256":1}}],["2946562f",{"2":{"256":1}}],["2977874f",{"2":{"256":1}}],["297959481822617",{"2":{"153":2}}],["291560524770234e",{"2":{"258":1}}],["291850669067434e",{"2":{"258":1}}],["2918086f",{"2":{"256":1}}],["2916829f",{"2":{"256":1}}],["29135",{"2":{"187":1}}],["2913357",{"2":{"106":1}}],["29315922",{"2":{"244":1}}],["2932029",{"2":{"244":1}}],["293211",{"2":{"106":1}}],["2938747",{"2":{"244":1}}],["292981203171935e",{"2":{"258":1}}],["29206026",{"2":{"244":1}}],["292002",{"2":{"168":1}}],["2922556",{"2":{"241":1}}],["290620075873599e",{"2":{"258":1}}],["290227507396954e",{"2":{"258":1}}],["290237f",{"2":{"256":1}}],["290726f",{"2":{"256":1}}],["29001",{"2":{"236":1}}],["2904677",{"2":{"129":1}}],["29",{"2":{"217":2,"228":2,"249":1}}],["2958108520232484e",{"2":{"258":1}}],["2958072f",{"2":{"256":1}}],["2950",{"2":{"249":1}}],["29501",{"2":{"236":1}}],["2952975296256756e",{"2":{"258":1}}],["2952234f",{"2":{"256":1}}],["29523",{"2":{"187":1}}],["29527",{"2":{"187":1}}],["29561827",{"2":{"179":1}}],["2980226051005725e",{"2":{"258":1}}],["29870",{"2":{"187":1}}],["298787",{"2":{"148":1}}],["29828635",{"2":{"179":1}}],["299408633732433e",{"2":{"258":1}}],["29944375",{"2":{"179":1}}],["2992029407862942e",{"2":{"258":1}}],["2992126",{"2":{"241":2}}],["299325f",{"2":{"256":1}}],["2993707f",{"2":{"256":1}}],["2991",{"2":{"249":1}}],["2990853",{"2":{"179":1}}],["29955",{"2":{"168":1}}],["29630",{"2":{"217":2,"219":1}}],["296372",{"2":{"168":1}}],["29689",{"2":{"187":1}}],["296496",{"2":{"171":1}}],["263269962070584e",{"2":{"258":1}}],["263231f",{"2":{"256":1}}],["26311737",{"2":{"106":1}}],["269739589208838e",{"2":{"258":1}}],["2696785f",{"2":{"256":1}}],["2695",{"2":{"249":1}}],["264962280622851e",{"2":{"258":1}}],["264279379569514e",{"2":{"258":1}}],["264283f",{"2":{"256":1}}],["2646691f",{"2":{"256":1}}],["2647",{"2":{"249":1}}],["26477236",{"2":{"106":2}}],["2648",{"2":{"249":1}}],["26",{"2":{"196":1,"202":2,"228":4,"249":1}}],["2681656",{"2":{"241":1}}],["26873",{"2":{"187":1}}],["2689521",{"2":{"106":2}}],["268941",{"2":{"63":2}}],["2601805483492784e",{"2":{"258":1}}],["2601067f",{"2":{"256":1}}],["260598",{"2":{"244":1}}],["26001",{"2":{"236":1}}],["26049",{"2":{"187":1}}],["2606924",{"2":{"148":1}}],["262623",{"2":{"168":1}}],["2615936",{"2":{"147":1}}],["261273",{"2":{"131":2}}],["2672",{"2":{"249":1}}],["26793814",{"2":{"244":1}}],["26771653",{"2":{"241":2}}],["267644",{"2":{"147":1,"148":1}}],["26700416",{"2":{"106":1}}],["26700422",{"2":{"106":1}}],["26657984",{"2":{"106":1}}],["265707583622401e",{"2":{"258":1}}],["2657076f",{"2":{"256":1}}],["265788",{"2":{"168":1}}],["26501",{"2":{"236":1}}],["265372",{"2":{"168":1}}],["265",{"2":{"78":1}}],["221811f",{"2":{"256":1}}],["229501296823788e",{"2":{"258":1}}],["2295013f",{"2":{"256":1}}],["2290224145974982",{"2":{"153":2}}],["2271",{"2":{"249":1}}],["227513",{"2":{"168":1}}],["22001",{"2":{"236":1}}],["220",{"2":{"220":6}}],["22068",{"2":{"187":1}}],["2261",{"2":{"249":1}}],["226",{"2":{"202":1}}],["22635892",{"2":{"179":1}}],["226381",{"2":{"77":1}}],["2287953f",{"2":{"256":1}}],["2287",{"2":{"249":1}}],["22878",{"2":{"187":1}}],["22807482",{"2":{"241":1}}],["22846f",{"2":{"77":1}}],["22501",{"2":{"236":1}}],["225",{"2":{"184":1,"201":2,"214":1,"220":18,"233":1}}],["22543387",{"2":{"77":1}}],["222040352394661e",{"2":{"258":1}}],["22222",{"2":{"217":2,"219":1}}],["222534",{"2":{"168":1}}],["222528",{"2":{"148":1}}],["2227837233928576",{"2":{"153":2}}],["2221465",{"2":{"148":1}}],["2228831",{"2":{"106":1}}],["22",{"2":{"145":1,"187":14,"196":1,"228":4,"254":14}}],["223914941069638e",{"2":{"258":1}}],["223999f",{"2":{"256":1}}],["22363997",{"2":{"106":1}}],["2236399",{"2":{"106":1}}],["2237958",{"2":{"77":1}}],["224768420825822e",{"2":{"258":1}}],["2248088f",{"2":{"256":1}}],["2243",{"2":{"249":1}}],["22406094",{"2":{"106":1}}],["22459084",{"2":{"77":1}}],["254673416812025e",{"2":{"258":1}}],["254381072079152e",{"2":{"258":1}}],["254381f",{"2":{"256":1}}],["2545f",{"2":{"256":1}}],["2545935",{"2":{"77":1}}],["258292437220608e",{"2":{"258":1}}],["2582407f",{"2":{"256":1}}],["258470031298464e",{"2":{"258":1}}],["2583666f",{"2":{"256":1}}],["2581186",{"2":{"241":1}}],["258",{"2":{"238":1}}],["2586653",{"2":{"106":1}}],["255258549986294e",{"2":{"258":1}}],["25564992604587e",{"2":{"258":1}}],["25561f",{"2":{"256":1}}],["255185f",{"2":{"256":1}}],["2557",{"2":{"249":1}}],["2555",{"2":{"249":1}}],["25501",{"2":{"236":1}}],["25587f",{"2":{"77":1}}],["250",{"2":{"245":2,"250":1,"255":1}}],["25001",{"2":{"236":1}}],["2501",{"2":{"236":1}}],["25055695",{"2":{"177":1,"178":1}}],["25926",{"2":{"217":3,"219":1}}],["251",{"2":{"202":1}}],["25",{"2":{"171":1,"187":15,"202":2,"228":38,"249":1}}],["25278285",{"2":{"179":1}}],["2525357728685884",{"2":{"153":2}}],["252985",{"2":{"148":1}}],["252925",{"2":{"148":1}}],["2523673",{"2":{"148":1}}],["253706635036125e",{"2":{"258":1}}],["25371f",{"2":{"256":1}}],["25378025",{"2":{"106":2}}],["25350",{"2":{"187":1}}],["25398",{"2":{"187":1}}],["25389904",{"2":{"106":1}}],["25385493",{"2":{"106":1}}],["257099221082406e",{"2":{"258":1}}],["257044f",{"2":{"256":1}}],["257347495428192e",{"2":{"258":1}}],["257",{"2":{"77":1}}],["256901898073669e",{"2":{"258":1}}],["256987713359704e",{"2":{"258":1}}],["256911f",{"2":{"256":1}}],["256807f",{"2":{"256":1}}],["256751im",{"2":{"168":1}}],["25662464",{"2":{"126":1}}],["256",{"2":{"46":1,"69":1,"77":6,"193":3,"226":2}}],["249120398230365e",{"2":{"258":1}}],["243532f",{"2":{"256":1}}],["245295366089095e",{"2":{"258":1}}],["245325f",{"2":{"256":1}}],["24501",{"2":{"236":1}}],["248712f",{"2":{"256":1}}],["2440945",{"2":{"241":2}}],["244728",{"2":{"63":10}}],["24640",{"2":{"187":1}}],["2464001",{"2":{"106":1}}],["24797",{"2":{"187":1}}],["24",{"2":{"115":2,"180":3,"187":14,"189":3,"197":2,"204":3,"221":3,"228":2,"229":3,"238":3,"246":3,"249":2}}],["241862833138554e",{"2":{"258":1}}],["2416237979383453e",{"2":{"258":1}}],["241614f",{"2":{"256":1}}],["24169",{"2":{"106":1}}],["241937",{"2":{"241":1}}],["241977",{"2":{"168":1}}],["2415697f",{"2":{"256":1}}],["2415948",{"2":{"148":1}}],["24150778",{"2":{"107":1}}],["240325231090934e",{"2":{"258":1}}],["240495011599202e",{"2":{"258":1}}],["2404157686012208e",{"2":{"258":1}}],["240431f",{"2":{"256":1}}],["2408228f",{"2":{"256":1}}],["240517f",{"2":{"256":1}}],["24001",{"2":{"236":1}}],["24099025",{"2":{"77":1}}],["240",{"2":{"55":3}}],["2154322768126957e",{"2":{"258":1}}],["21501",{"2":{"236":1}}],["2140641303782147e",{"2":{"258":1}}],["2141221075579763e",{"2":{"258":1}}],["214328e",{"2":{"202":1}}],["2174169344313458e",{"2":{"258":1}}],["2177645f",{"2":{"256":1}}],["2173807f",{"2":{"256":1}}],["2170883f",{"2":{"256":1}}],["2170",{"2":{"249":1}}],["2130",{"2":{"249":1}}],["2125984",{"2":{"241":2}}],["21252f",{"2":{"77":1}}],["210389202432504e",{"2":{"258":1}}],["210349f",{"2":{"256":1}}],["21001",{"2":{"236":1}}],["210",{"2":{"220":18}}],["21061",{"2":{"187":1}}],["211",{"2":{"220":6}}],["21100",{"2":{"187":1}}],["2111273",{"2":{"148":1}}],["218447824674669e",{"2":{"258":1}}],["21823",{"2":{"187":1}}],["21856f",{"2":{"77":1}}],["216931854108423e",{"2":{"258":1}}],["2169951484519065e",{"2":{"258":1}}],["216995f",{"2":{"256":1}}],["21690917",{"2":{"179":1}}],["21672015",{"2":{"126":1}}],["2191863f",{"2":{"256":1}}],["2191",{"2":{"249":1}}],["21940619",{"2":{"241":1}}],["21983",{"2":{"187":1}}],["2197981041108443",{"2":{"171":1}}],["2192001",{"2":{"148":1}}],["21950458",{"2":{"106":1}}],["21",{"2":{"55":1,"58":1,"187":14,"196":1,"217":2,"228":2,"249":6}}],["2091991845930406e",{"2":{"258":1}}],["209747366051371e",{"2":{"258":1}}],["2097083f",{"2":{"256":1}}],["20949235f",{"2":{"256":1}}],["20472442",{"2":{"241":2}}],["2042089",{"2":{"106":1}}],["20867",{"2":{"187":1}}],["20881107",{"2":{"126":1}}],["2036972998469065e",{"2":{"258":1}}],["203405f",{"2":{"256":1}}],["203011f",{"2":{"256":1}}],["203036",{"2":{"106":1}}],["20337",{"2":{"187":1}}],["2050",{"2":{"249":1}}],["20501",{"2":{"236":1}}],["2058456",{"2":{"244":1}}],["20579764",{"2":{"244":1}}],["20570",{"2":{"187":1}}],["205454im",{"2":{"168":1}}],["20548",{"2":{"168":1}}],["202781848859344e",{"2":{"258":1}}],["20277858",{"2":{"179":1}}],["2021",{"2":{"252":1}}],["2024",{"2":{"180":2,"189":2,"197":1,"204":2,"221":2,"229":2,"238":2,"246":2,"251":1,"260":1}}],["2023",{"2":{"74":2}}],["2067",{"2":{"249":1}}],["206476",{"2":{"168":1}}],["20668754",{"2":{"147":1}}],["2003056f",{"2":{"256":1}}],["2005206640083375e",{"2":{"258":1}}],["20055f",{"2":{"256":1}}],["20053944",{"2":{"106":1}}],["200019f",{"2":{"256":1}}],["20001",{"2":{"236":1}}],["20001543",{"2":{"179":1}}],["2000",{"2":{"192":1}}],["2001",{"2":{"179":1,"236":1}}],["200",{"2":{"107":1,"220":18}}],["2006",{"2":{"63":2,"249":1}}],["20",{"2":{"48":1,"55":3,"77":1,"116":1,"147":1,"171":2,"179":3,"187":14,"215":5,"220":24,"228":2,"249":7}}],["201",{"2":{"202":1,"220":18,"245":1}}],["20114",{"2":{"171":1}}],["201145",{"2":{"171":2}}],["2010",{"2":{"28":3}}],["201611+0",{"2":{"168":1}}],["2016",{"2":{"19":2,"55":1,"63":2}}],["20188306",{"2":{"106":1}}],["2018",{"2":{"19":1}}],["2015050068933292e",{"2":{"258":1}}],["2015",{"2":{"19":1,"28":2}}],["201444f",{"2":{"256":1}}],["2014",{"2":{"17":1,"28":1}}],["2017",{"2":{"17":1,"63":2}}],["2346334950529898e",{"2":{"258":1}}],["2349672f",{"2":{"256":1}}],["2332278f",{"2":{"256":1}}],["23325463",{"2":{"126":1}}],["2329721141036216e",{"2":{"258":1}}],["2321886f",{"2":{"256":1}}],["23234403",{"2":{"244":1}}],["23242",{"2":{"147":1}}],["2354317",{"2":{"241":1}}],["23501",{"2":{"236":1}}],["2351742f",{"2":{"106":3}}],["2396502902144953e",{"2":{"258":1}}],["2393571f",{"2":{"256":1}}],["23992883",{"2":{"179":1}}],["239756",{"2":{"172":1}}],["23921265",{"2":{"147":1}}],["230775314602677e",{"2":{"258":1}}],["230482f",{"2":{"256":1}}],["23001",{"2":{"236":1}}],["23055",{"2":{"187":1}}],["23095",{"2":{"171":1}}],["230954",{"2":{"171":2}}],["23021114",{"2":{"106":1}}],["231614989239022e",{"2":{"258":1}}],["23162955",{"2":{"106":1}}],["2319405095342203e",{"2":{"258":1}}],["2313593f",{"2":{"256":1}}],["231723",{"2":{"168":1}}],["23710957",{"2":{"137":1}}],["2366",{"2":{"249":1}}],["23622048",{"2":{"241":2}}],["23626f",{"2":{"77":1}}],["23672",{"2":{"187":1}}],["2361472",{"2":{"126":1}}],["23",{"2":{"48":2,"180":1,"187":14,"189":1,"197":1,"204":1,"221":1,"228":2,"229":1,"238":1,"246":1,"249":2,"251":1,"260":1}}],["288003555528041e",{"2":{"258":1}}],["2882388f",{"2":{"256":1}}],["28839",{"2":{"187":1}}],["287408413153271e",{"2":{"258":1}}],["287375f",{"2":{"256":1}}],["2877",{"2":{"249":1}}],["28725442",{"2":{"106":1}}],["289725906583554e",{"2":{"258":1}}],["2897161f",{"2":{"256":1}}],["289482122236741e",{"2":{"258":1}}],["2894528f",{"2":{"256":1}}],["28946856",{"2":{"244":1}}],["2899156",{"2":{"244":1}}],["2896426",{"2":{"241":1}}],["2896142",{"2":{"147":1}}],["2852",{"2":{"249":1}}],["28501",{"2":{"236":1}}],["28596",{"2":{"187":1}}],["281252",{"2":{"241":1}}],["28169",{"2":{"187":1}}],["281053",{"2":{"168":1}}],["2803448180066917e",{"2":{"258":1}}],["2805929f",{"2":{"256":1}}],["28058",{"2":{"187":1}}],["28001",{"2":{"236":1}}],["28042415",{"2":{"106":1}}],["2833",{"2":{"249":1}}],["2835",{"2":{"249":1}}],["28355134",{"2":{"106":1}}],["28370",{"2":{"187":1}}],["2869913410456831",{"2":{"153":2}}],["2864522",{"2":{"148":1}}],["28682f",{"2":{"77":1}}],["284708940473613e",{"2":{"258":1}}],["2847383f",{"2":{"256":1}}],["28402749670361e",{"2":{"258":1}}],["2840365f",{"2":{"256":1}}],["2849667",{"2":{"148":1}}],["2842",{"2":{"147":1}}],["28289196",{"2":{"77":1}}],["28",{"2":{"46":4,"180":1,"189":1,"193":2,"195":2,"196":3,"197":1,"204":1,"217":1,"220":2,"221":1,"224":4,"228":3,"229":1,"238":1,"246":1,"249":1,"251":1,"260":1}}],["2=dense",{"2":{"39":1}}],["2d",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"238":1},"2":{"28":2,"49":3,"50":2,"51":3,"195":2,"230":1}}],["2nd",{"0":{"24":1},"2":{"102":1,"115":1,"232":2,"234":1}}],["2",{"2":{"5":5,"23":2,"28":5,"38":5,"39":4,"41":8,"44":4,"46":4,"48":30,"49":9,"51":15,"52":11,"53":2,"54":15,"55":6,"56":3,"58":1,"59":1,"63":22,"69":13,"77":49,"78":5,"102":7,"106":32,"107":4,"109":21,"110":41,"115":1,"116":1,"117":2,"126":3,"127":2,"128":2,"129":4,"131":2,"132":1,"135":3,"136":12,"137":2,"139":6,"140":6,"145":7,"146":4,"147":13,"148":13,"152":1,"153":23,"155":5,"168":8,"171":20,"172":1,"173":4,"174":3,"176":1,"177":1,"180":4,"183":5,"184":1,"185":1,"187":15,"188":1,"189":4,"192":1,"193":18,"196":2,"197":2,"200":6,"202":10,"203":5,"204":4,"213":1,"214":1,"215":1,"217":5,"219":1,"220":67,"221":4,"226":1,"228":4,"229":5,"234":4,"235":10,"237":4,"238":5,"241":14,"242":1,"244":6,"245":3,"246":4,"248":2,"249":34,"254":50,"255":8,"256":111,"258":113}}],["v=v",{"2":{"234":1}}],["vtav",{"2":{"149":1}}],["vvt",{"2":{"149":1}}],["v∈rd",{"2":{"149":1}}],["vs",{"2":{"133":1,"202":1}}],["v0",{"2":{"74":1,"97":1,"130":1,"218":1}}],["voila",{"2":{"109":1}}],["volterra",{"2":{"200":2}}],["vol",{"2":{"63":1}}],["volumetric",{"2":{"63":1}}],["vocabulary",{"2":{"53":2}}],["vcat",{"2":{"52":1,"183":1,"254":1}}],["v",{"2":{"22":2,"55":1,"63":1,"64":3,"90":1,"150":5,"151":5,"152":10,"176":4,"177":2,"178":1,"233":4,"234":10,"254":21}}],["vjp",{"0":{"22":1},"2":{"22":1,"150":7,"152":16,"175":3,"178":4,"245":3}}],["visualization",{"0":{"250":1}}],["visualize",{"2":{"241":1,"250":1,"259":1}}],["visualizing",{"0":{"237":1,"259":1}}],["vision",{"2":{"19":1,"28":2,"63":4}}],["virtual",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["vi",{"2":{"149":1}}],["vitj",{"2":{"149":1}}],["viewaxis",{"2":{"220":68}}],["views",{"2":{"214":2,"234":1,"254":1}}],["view",{"2":{"54":4,"173":1,"249":1,"250":1}}],["victor",{"2":{"19":1,"55":1}}],["via",{"2":{"3":1,"5":2,"6":1,"7":1,"18":1,"28":1,"30":1,"62":2,"67":2,"79":2,"83":2,"97":1,"98":1,"100":1,"106":1,"120":1,"123":1,"153":1,"174":1,"176":4,"177":1,"249":1}}],["vanilla",{"2":{"106":1}}],["various",{"2":{"79":1,"83":1,"101":1,"198":1}}],["variants",{"2":{"158":1}}],["variance",{"2":{"17":2,"19":8,"55":6,"249":1}}],["variable",{"2":{"69":2,"97":1,"249":1}}],["variables",{"2":{"62":2,"69":1,"77":1,"83":2,"233":1,"249":1,"254":1}}],["variationalhiddendropout",{"2":{"50":5}}],["var",{"2":{"19":5,"55":7,"69":1,"109":2,"110":2,"126":1,"129":1,"184":1,"201":2,"214":1,"220":24,"233":1,"236":4}}],["validate",{"2":{"187":1}}],["validated",{"2":{"1":1}}],["validation",{"2":{"183":1,"187":101}}],["valid",{"2":{"55":1,"135":1,"164":1}}],["val",{"2":{"10":2,"17":4,"19":2,"40":3,"46":1,"48":7,"50":1,"63":4,"64":5,"65":4,"109":1,"110":2,"126":2,"129":2,"183":4,"184":1,"185":1,"187":2,"201":1,"220":13,"235":8,"237":3}}],["valued",{"2":{"176":1}}],["value>",{"2":{"161":3}}],["value",{"2":{"3":1,"8":1,"10":2,"17":1,"19":4,"34":2,"40":1,"48":1,"52":5,"55":6,"60":1,"63":6,"64":1,"65":3,"69":6,"110":1,"146":2,"179":16,"183":1,"233":1,"236":4,"249":1,"250":4}}],["valuestorage",{"2":{"220":3}}],["values",{"2":{"3":2,"32":2,"56":1,"62":4,"63":1,"65":1,"66":3,"244":1}}],["vec",{"2":{"152":2,"184":1,"185":1,"214":4,"225":1,"235":1,"237":1}}],["vectors",{"2":{"52":1,"53":2,"55":3,"149":1,"174":6,"247":1}}],["vectorization",{"0":{"167":1},"2":{"20":1,"159":1,"167":1}}],["vectorize",{"2":{"13":1}}],["vector",{"0":{"150":1,"151":1,"177":1,"178":1},"2":{"15":1,"18":1,"22":9,"28":2,"41":2,"52":16,"53":5,"58":1,"63":3,"69":2,"83":1,"116":2,"144":2,"149":4,"150":1,"151":1,"153":3,"171":4,"174":2,"175":5,"176":1,"177":3,"178":1,"179":1,"188":1,"201":1,"214":3,"217":1,"249":4,"250":1,"254":1}}],["vendor",{"2":{"20":1}}],["vedaldi",{"2":{"19":1,"55":1}}],["verbatim",{"2":{"69":1}}],["verified",{"2":{"152":1}}],["verification",{"2":{"8":1}}],["verify",{"2":{"65":1,"145":1,"146":1,"147":1,"148":1,"152":1}}],["versioninfo",{"2":{"180":3,"189":3,"197":3,"204":3,"221":3,"229":3,"238":3,"246":3,"251":3,"260":3}}],["versioning",{"2":{"145":2}}],["versions",{"2":{"97":1,"159":1,"167":2}}],["version",{"2":{"38":1,"62":2,"65":1,"69":1,"71":3,"74":1,"76":1,"90":1,"114":1,"135":1,"150":1,"180":1,"189":1,"197":1,"204":1,"210":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["very",{"2":{"7":1,"53":1,"77":1,"83":1,"106":2,"108":1,"109":1,"110":2,"139":1,"140":1,"170":1,"171":1,"254":1,"258":1}}],["v1",{"0":{"85":1},"1":{"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1},"2":{"7":1,"49":2,"76":4,"85":2,"88":1,"130":1}}],["||",{"2":{"78":1,"107":1,"179":1,"236":1,"245":1}}],["|>",{"2":{"77":2,"78":3,"106":4,"107":2,"131":2,"132":3,"153":2,"160":2,"179":1,"187":3,"201":1,"202":2,"203":1,"215":2,"217":1,"225":2,"228":3,"236":3,"244":1,"245":1,"256":1}}],["|y^−y|",{"2":{"63":1}}],["|y−y^|−0",{"2":{"63":1}}],["|y−y^|≤δδ∗",{"2":{"63":1}}],["|p|−di×",{"2":{"49":1,"51":3}}],["|",{"2":{"3":4,"95":1,"109":5,"110":13,"237":1}}],["x~",{"2":{"250":1}}],["x~|θ",{"2":{"250":1}}],["x~|x",{"2":{"250":1}}],["x∈",{"2":{"235":1}}],["xyt",{"2":{"234":17,"235":18,"236":10}}],["xi∈rn",{"2":{"179":1}}],["xi",{"2":{"179":2}}],["xᵢ",{"2":{"107":4}}],["xdev",{"2":{"106":5,"107":3}}],["xlogy",{"2":{"64":1}}],["xlogx",{"2":{"64":1}}],["xlabel=",{"2":{"237":1,"241":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["xladevice",{"2":{"2":2}}],["xla",{"0":{"73":1},"2":{"2":1,"73":3,"106":5,"107":1}}],["x=",{"2":{"59":1}}],["x3",{"2":{"48":1}}],["x3c",{"2":{"3":3,"4":2,"7":2,"16":1,"18":1,"32":3,"49":4,"52":2,"115":2,"117":2,"135":1,"136":1,"140":1,"161":6,"176":1,"184":1,"202":1,"214":2,"218":2,"220":2,"233":1,"236":2,"254":2}}],["x2s",{"2":{"248":8}}],["x2",{"2":{"48":3,"248":2,"250":18}}],["x26",{"2":{"41":6,"74":1,"127":2,"128":4,"180":4,"189":4,"197":4,"202":2,"204":4,"221":4,"229":4,"236":4,"238":4,"246":4,"251":4,"254":4,"260":4}}],["x1s",{"2":{"248":8}}],["x1",{"2":{"48":3,"248":2,"250":18}}],["xoshiro",{"2":{"28":3,"41":1,"69":5,"78":1,"109":1,"126":2,"153":1,"155":1,"174":3,"187":1,"228":1}}],["xavier",{"2":{"28":4}}],["xs",{"2":{"20":2,"237":6,"248":1,"249":3}}],["xt0s",{"2":{"248":5}}],["xt1s",{"2":{"248":5}}],["xt",{"2":{"19":2,"117":1,"118":1}}],["x86",{"2":{"18":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["x",{"2":{"3":12,"5":12,"7":3,"8":12,"11":2,"13":10,"14":2,"15":10,"16":3,"17":8,"18":3,"19":26,"20":1,"22":6,"23":10,"28":7,"44":2,"46":2,"48":24,"49":23,"50":9,"51":27,"52":20,"53":15,"54":30,"55":13,"56":14,"59":5,"63":2,"64":18,"65":16,"68":2,"69":31,"77":4,"78":12,"106":14,"107":3,"109":4,"110":10,"113":3,"115":9,"116":2,"117":9,"121":3,"126":3,"129":2,"131":4,"132":9,"135":6,"136":5,"137":7,"139":6,"140":10,"144":6,"145":10,"146":12,"148":8,"150":2,"151":2,"152":9,"153":3,"155":3,"158":3,"160":5,"171":7,"172":2,"173":4,"176":7,"177":5,"178":1,"179":11,"183":6,"184":8,"185":8,"186":2,"187":4,"192":6,"194":2,"195":4,"200":4,"213":6,"214":10,"216":2,"217":2,"218":2,"220":13,"224":4,"225":2,"227":2,"228":2,"235":7,"237":1,"241":9,"245":8,"248":1,"249":5,"250":4,"254":6,"258":2}}],["3+0",{"2":{"180":2,"189":2,"204":2,"221":2,"229":2,"238":2,"246":2}}],["378156353770916e",{"2":{"258":1}}],["3779917f",{"2":{"256":1}}],["37796",{"2":{"187":1}}],["3791565390392404e",{"2":{"258":1}}],["3791175f",{"2":{"256":1}}],["3794s\\ttraining",{"2":{"217":1}}],["37501",{"2":{"236":1}}],["3708327975928686e",{"2":{"258":1}}],["37026f",{"2":{"256":1}}],["3705",{"2":{"249":1}}],["37009271020907e",{"2":{"258":1}}],["3700787",{"2":{"241":2}}],["37001",{"2":{"236":1}}],["37037",{"2":{"217":2}}],["3707023",{"2":{"145":1}}],["37",{"2":{"217":5,"219":1,"228":3}}],["376",{"2":{"202":1}}],["376383",{"2":{"148":1}}],["3727",{"2":{"249":1}}],["37275374",{"2":{"241":1}}],["37263",{"2":{"187":1}}],["3729234",{"2":{"147":1}}],["3743544377026143e",{"2":{"258":1}}],["3740988f",{"2":{"256":1}}],["37452",{"2":{"187":1}}],["3742405e",{"2":{"179":1}}],["37335",{"2":{"187":1}}],["3719285",{"2":{"148":1}}],["3rd",{"0":{"210":1},"2":{"147":1}}],["3465126f",{"2":{"256":1}}],["3454915757587986e",{"2":{"258":1}}],["3452623f",{"2":{"256":1}}],["34501",{"2":{"236":1}}],["345025",{"2":{"148":1}}],["344698775352445e",{"2":{"258":1}}],["344538425259823e",{"2":{"258":1}}],["3447893f",{"2":{"256":1}}],["344208f",{"2":{"256":1}}],["34490",{"2":{"187":1}}],["340165f",{"2":{"256":1}}],["3401",{"2":{"249":1}}],["34001",{"2":{"236":1}}],["34",{"2":{"228":3,"249":1}}],["34123",{"2":{"187":1}}],["3410565",{"2":{"136":1}}],["34717593",{"2":{"179":1}}],["34254",{"2":{"171":1}}],["34253937",{"2":{"171":1}}],["342539",{"2":{"171":2}}],["348932223189744e",{"2":{"258":1}}],["3489726f",{"2":{"256":1}}],["348918im",{"2":{"168":1}}],["34855f",{"2":{"77":1}}],["3432243f",{"2":{"256":1}}],["343517419753061e",{"2":{"258":1}}],["34351700231383653",{"2":{"174":1}}],["343531210380995e",{"2":{"258":1}}],["3435276f",{"2":{"256":1}}],["3436353225087703",{"2":{"153":2}}],["34383082",{"2":{"126":1}}],["3494582244101045e",{"2":{"258":1}}],["3491451f",{"2":{"256":1}}],["349021082084834e",{"2":{"258":1}}],["3490952f",{"2":{"256":1}}],["3490",{"2":{"249":1}}],["349",{"2":{"131":2}}],["3679823189400309e",{"2":{"258":1}}],["368849663732603e",{"2":{"258":1}}],["368759f",{"2":{"256":1}}],["3685573f",{"2":{"256":1}}],["36829436",{"2":{"179":1}}],["3644179731121464e",{"2":{"258":1}}],["3647802f",{"2":{"256":1}}],["3645461",{"2":{"241":1}}],["36428708",{"2":{"145":1}}],["3637495",{"2":{"241":1}}],["36001",{"2":{"236":1}}],["3602331",{"2":{"147":1}}],["36",{"2":{"217":1,"228":5}}],["3669",{"2":{"249":1}}],["36606",{"2":{"187":1}}],["36610",{"2":{"187":1}}],["36179",{"2":{"187":1}}],["3694",{"2":{"249":1}}],["3690",{"2":{"249":1}}],["3693867",{"2":{"241":1}}],["36938f",{"2":{"77":1}}],["3691778381831775",{"2":{"174":1}}],["369178",{"2":{"171":2}}],["36918",{"2":{"171":1}}],["365699682256304e",{"2":{"258":1}}],["3654065f",{"2":{"256":1}}],["36542922",{"2":{"147":1}}],["3653",{"2":{"249":1}}],["36535",{"2":{"187":1}}],["36501",{"2":{"236":1}}],["3650193059617517",{"2":{"153":2}}],["3657s\\ttraining",{"2":{"217":1}}],["362543189600014e",{"2":{"258":1}}],["362593",{"2":{"106":1}}],["3623786f",{"2":{"256":1}}],["3628",{"2":{"249":1}}],["36220473",{"2":{"241":2}}],["36222499022985155",{"2":{"153":2}}],["3142036093340545e",{"2":{"258":1}}],["314238376897153e",{"2":{"258":1}}],["3148644f",{"2":{"256":1}}],["314009f",{"2":{"256":1}}],["314708016779633e",{"2":{"258":1}}],["3147144225444096e",{"2":{"258":1}}],["3147543f",{"2":{"256":1}}],["3147238",{"2":{"77":1}}],["313490590391579e",{"2":{"258":1}}],["3139113f",{"2":{"256":1}}],["3136577f",{"2":{"256":1}}],["313226f",{"2":{"106":3}}],["31906623362331e",{"2":{"258":1}}],["3190884f",{"2":{"256":1}}],["3191016f",{"2":{"256":1}}],["319268656932053e",{"2":{"258":1}}],["3192",{"2":{"249":1}}],["3170582",{"2":{"241":1}}],["315824425403808e",{"2":{"258":1}}],["3155",{"2":{"249":1}}],["31550723",{"2":{"106":1}}],["31501",{"2":{"236":1}}],["31001",{"2":{"236":1}}],["3108876",{"2":{"147":1}}],["31",{"2":{"228":2,"249":2}}],["31185",{"2":{"187":1}}],["31128f",{"2":{"77":1}}],["31861955",{"2":{"145":1}}],["3539522397538664e",{"2":{"258":1}}],["35395628",{"2":{"241":1}}],["353919f",{"2":{"256":1}}],["3549777652255285e",{"2":{"258":1}}],["3541768488869374e",{"2":{"258":1}}],["35431f",{"2":{"256":1}}],["3540122f",{"2":{"256":1}}],["354464",{"2":{"106":1}}],["352328128651912e",{"2":{"258":1}}],["3523193",{"2":{"179":1}}],["352244f",{"2":{"256":1}}],["3526998f",{"2":{"256":1}}],["352485609995227e",{"2":{"258":1}}],["3524440909899965e",{"2":{"258":1}}],["3524",{"2":{"249":1}}],["356901393761471e",{"2":{"258":1}}],["35653025",{"2":{"241":1}}],["35638642",{"2":{"147":1}}],["35505145941473e",{"2":{"258":1}}],["3550838f",{"2":{"256":1}}],["35501",{"2":{"236":1}}],["3558193508137715",{"2":{"153":2}}],["3505003f",{"2":{"256":1}}],["35001",{"2":{"236":1}}],["350111472519508e",{"2":{"258":1}}],["3501",{"2":{"236":1}}],["359",{"2":{"204":1}}],["359568e",{"2":{"202":1}}],["35",{"2":{"180":2,"189":2,"204":2,"221":2,"228":4,"229":2,"238":2,"246":2,"249":1}}],["35817",{"2":{"171":1}}],["35836592",{"2":{"77":1}}],["351",{"2":{"202":1}}],["35149138733595564",{"2":{"174":4}}],["351491",{"2":{"171":1}}],["351662",{"2":{"168":1}}],["35181466",{"2":{"126":1}}],["35188f",{"2":{"77":1}}],["35152f",{"2":{"77":1}}],["381059892683991e",{"2":{"258":1}}],["38187847",{"2":{"126":1}}],["384531561969631e",{"2":{"258":1}}],["384679157608353e",{"2":{"258":1}}],["3841858f",{"2":{"106":2}}],["38693978535884e",{"2":{"258":1}}],["3869788f",{"2":{"256":1}}],["3864",{"2":{"249":1}}],["389818034523583e",{"2":{"258":1}}],["38984f",{"2":{"256":1}}],["38911813",{"2":{"106":1}}],["3891182",{"2":{"106":1}}],["385879612518597e",{"2":{"258":1}}],["385712f",{"2":{"256":1}}],["38501",{"2":{"236":1}}],["3850993",{"2":{"126":1}}],["3886",{"2":{"249":1}}],["3877474645670614e",{"2":{"258":1}}],["3877268f",{"2":{"256":1}}],["3878173204562125e",{"2":{"258":1}}],["3875676f",{"2":{"256":1}}],["3871873",{"2":{"244":1}}],["38798f",{"2":{"77":1}}],["38",{"2":{"228":28}}],["3822335f",{"2":{"256":1}}],["382474477404435e",{"2":{"258":1}}],["3824835f",{"2":{"256":1}}],["38242024",{"2":{"148":1}}],["38200346",{"2":{"241":1}}],["38291",{"2":{"187":1}}],["382971",{"2":{"168":1}}],["3834447782571341",{"2":{"153":1}}],["3834447782571344",{"2":{"153":1}}],["380969f",{"2":{"256":1}}],["380597f",{"2":{"256":1}}],["38001",{"2":{"236":1}}],["380777f0",{"2":{"146":1}}],["38068f",{"2":{"77":1}}],["325505215689046e",{"2":{"258":1}}],["325607074653012e",{"2":{"258":1}}],["325691f",{"2":{"256":1}}],["3257898826220205e",{"2":{"258":1}}],["325399f",{"2":{"256":1}}],["32501",{"2":{"236":1}}],["3226475742017294e",{"2":{"258":1}}],["3226349f",{"2":{"256":1}}],["3228642419102902e",{"2":{"258":1}}],["3228825f",{"2":{"256":1}}],["3229032f",{"2":{"256":1}}],["323151662688528e",{"2":{"258":1}}],["3232504790464034e",{"2":{"258":1}}],["32327131567796e",{"2":{"258":1}}],["32322538",{"2":{"126":1}}],["323061f",{"2":{"256":1}}],["3230212f",{"2":{"256":1}}],["3274710235328225e",{"2":{"258":1}}],["32741f",{"2":{"256":1}}],["3270",{"2":{"249":1}}],["3279041356366077",{"2":{"153":2}}],["3295454745602193e",{"2":{"258":1}}],["3292323f",{"2":{"256":1}}],["3294",{"2":{"249":1}}],["3291591",{"2":{"244":1}}],["32984197",{"2":{"77":1}}],["328542",{"2":{"241":1}}],["32839986131789933",{"2":{"153":2}}],["32001",{"2":{"236":1}}],["320",{"2":{"220":6}}],["32687877041703e",{"2":{"258":1}}],["3265313f",{"2":{"256":1}}],["32676f",{"2":{"256":1}}],["326",{"2":{"202":1}}],["32623518",{"2":{"126":1}}],["32480",{"2":{"187":1}}],["32103915017989e",{"2":{"258":1}}],["3210428f",{"2":{"256":1}}],["321",{"2":{"220":6}}],["32186",{"2":{"187":1}}],["321506",{"2":{"168":1}}],["32",{"2":{"44":2,"69":2,"78":13,"106":9,"107":2,"109":1,"110":1,"193":2,"200":1,"202":4,"226":2,"228":2,"236":1,"242":1,"251":1,"256":8,"260":1}}],["3=dense",{"2":{"39":1}}],["3dv",{"2":{"63":1}}],["3d",{"2":{"19":1,"23":1,"63":1,"146":1,"187":1,"228":2,"245":1}}],["303357824034544e",{"2":{"258":1}}],["303284f",{"2":{"256":1}}],["3035",{"2":{"249":1}}],["304244639740679e",{"2":{"258":1}}],["3042545f",{"2":{"256":1}}],["3041183f",{"2":{"256":1}}],["3040",{"2":{"249":1}}],["3048440391400475e",{"2":{"258":1}}],["3048",{"2":{"249":1}}],["307611007594211e",{"2":{"258":1}}],["30762f",{"2":{"256":1}}],["3070866",{"2":{"241":2}}],["3071294",{"2":{"148":1}}],["3050475",{"2":{"244":1}}],["30501",{"2":{"236":1}}],["30569053",{"2":{"244":1}}],["305844",{"2":{"147":1,"148":1}}],["308636e",{"2":{"202":1}}],["30215",{"2":{"171":1}}],["30673835",{"2":{"241":1}}],["30674052",{"2":{"179":1}}],["306641",{"2":{"168":1}}],["306147",{"2":{"168":1}}],["3092",{"2":{"249":1}}],["3096",{"2":{"249":1}}],["3097294",{"2":{"179":1}}],["309",{"2":{"145":2}}],["3008",{"2":{"249":1}}],["30001",{"2":{"236":1}}],["3001",{"2":{"179":1,"236":1}}],["300",{"2":{"107":1}}],["301186682732619e",{"2":{"258":1}}],["3011372860824714e",{"2":{"258":1}}],["3011274f",{"2":{"256":1}}],["301",{"2":{"77":1,"202":1}}],["30",{"2":{"17":1,"28":1,"196":1,"217":1,"228":2,"249":1}}],["3×2",{"2":{"69":2,"131":2}}],["3×5",{"2":{"63":4,"69":1}}],["3×3×1×1",{"2":{"28":1}}],["3×7",{"2":{"5":2}}],["3×13",{"2":{"5":4}}],["339554482719717e",{"2":{"258":1}}],["339079",{"2":{"148":1}}],["338426930109939e",{"2":{"258":1}}],["338467f",{"2":{"256":1}}],["3385826",{"2":{"241":2}}],["335871266180674e",{"2":{"258":1}}],["3356914f",{"2":{"256":1}}],["3356",{"2":{"249":1}}],["33544478",{"2":{"241":1}}],["33501",{"2":{"236":1}}],["3343754",{"2":{"241":1}}],["3344351",{"2":{"148":1}}],["33442986",{"2":{"106":1}}],["33054202260911e",{"2":{"258":1}}],["330651926769951e",{"2":{"258":1}}],["330360405708549e",{"2":{"258":1}}],["330436f",{"2":{"256":1}}],["3304215f",{"2":{"256":1}}],["330819f",{"2":{"256":1}}],["33070865",{"2":{"241":2}}],["33001",{"2":{"236":1}}],["3331402971243175e",{"2":{"258":1}}],["333284418874019e",{"2":{"258":1}}],["333262f",{"2":{"256":1}}],["3333755f",{"2":{"256":1}}],["3333333333333335",{"2":{"249":1}}],["33333",{"2":{"217":15,"219":2}}],["3335",{"2":{"249":1}}],["33349",{"2":{"187":1}}],["336818557921545e",{"2":{"258":1}}],["336628174683956e",{"2":{"258":1}}],["336678675997424e",{"2":{"258":1}}],["3366885f",{"2":{"256":1}}],["3366508f",{"2":{"256":1}}],["33645",{"2":{"187":1}}],["3367505",{"2":{"126":1}}],["33758628",{"2":{"118":1}}],["33",{"2":{"5":1,"77":2,"78":1,"196":2,"228":2}}],["397297774624188e",{"2":{"258":1}}],["3973716f",{"2":{"256":1}}],["3976",{"2":{"249":1}}],["3977319",{"2":{"148":1}}],["398038",{"2":{"241":1}}],["39800483",{"2":{"148":1}}],["3935",{"2":{"249":1}}],["39300445",{"2":{"244":1}}],["39370078",{"2":{"241":2}}],["3939896",{"2":{"148":1}}],["395735431163909e",{"2":{"258":1}}],["395571f",{"2":{"256":1}}],["3956",{"2":{"249":1}}],["3952",{"2":{"249":1}}],["39588368",{"2":{"241":1}}],["39501",{"2":{"236":1}}],["395306",{"2":{"145":1}}],["39966",{"2":{"187":1}}],["3960",{"2":{"249":1}}],["39644",{"2":{"187":1}}],["396323",{"2":{"168":1}}],["3969839f",{"2":{"106":1}}],["39001",{"2":{"236":1}}],["390200114697191",{"2":{"153":2}}],["3903124",{"2":{"106":1}}],["394313533953629e",{"2":{"258":1}}],["394660341156399e",{"2":{"258":1}}],["394991f",{"2":{"256":1}}],["3942307655311696",{"2":{"153":2}}],["3940224213371761",{"2":{"153":2}}],["3928175",{"2":{"145":1}}],["39",{"2":{"2":1,"4":2,"8":1,"10":1,"13":2,"16":1,"18":1,"29":1,"39":1,"41":1,"44":1,"48":1,"52":1,"54":1,"55":3,"56":1,"57":1,"62":1,"63":1,"64":1,"67":3,"69":7,"77":2,"83":2,"90":2,"93":1,"97":1,"104":3,"105":1,"106":2,"107":1,"109":1,"110":1,"111":1,"113":1,"114":1,"121":2,"123":1,"124":1,"126":1,"135":5,"136":1,"137":3,"138":1,"140":2,"142":1,"143":1,"144":4,"145":4,"146":1,"147":2,"148":1,"152":3,"153":2,"155":1,"170":5,"171":8,"173":2,"176":1,"177":1,"179":1,"184":3,"186":1,"188":2,"196":2,"198":2,"201":1,"214":1,"217":1,"219":1,"225":1,"228":3,"241":1,"242":1,"245":1,"248":1,"249":4,"254":1,"255":1,"256":2}}],["3",{"2":{"2":1,"5":3,"28":5,"39":7,"44":2,"46":1,"48":17,"49":2,"50":2,"52":4,"54":4,"55":3,"58":1,"62":1,"63":30,"69":27,"77":10,"78":4,"102":2,"106":16,"107":1,"109":12,"110":7,"115":3,"116":1,"118":1,"126":4,"129":6,"131":1,"132":1,"135":1,"137":1,"145":4,"147":12,"148":7,"153":2,"171":23,"172":1,"173":3,"174":7,"180":6,"184":1,"185":1,"187":14,"189":6,"192":1,"193":9,"195":1,"196":3,"200":1,"202":1,"204":6,"213":1,"215":2,"217":5,"219":1,"220":56,"221":7,"228":7,"229":6,"233":3,"234":2,"235":1,"236":2,"238":6,"241":7,"244":4,"246":6,"249":17,"250":1,"254":12,"255":1,"256":125,"258":127}}],["4t",{"2":{"235":1}}],["4th",{"2":{"127":1}}],["4fs",{"2":{"217":1}}],["4x",{"2":{"196":1}}],["462655359833105e",{"2":{"258":1}}],["469378314397726e",{"2":{"258":1}}],["469065f",{"2":{"256":1}}],["469896f",{"2":{"256":1}}],["46981",{"2":{"187":1}}],["4673",{"2":{"249":1}}],["46430754548516e",{"2":{"258":1}}],["464391f",{"2":{"256":1}}],["4641405f",{"2":{"256":1}}],["464424972716262e",{"2":{"258":1}}],["4644",{"2":{"249":1}}],["464567",{"2":{"241":2}}],["46061087",{"2":{"241":1}}],["46001",{"2":{"236":1}}],["4605s\\ttraining",{"2":{"219":1}}],["4604845",{"2":{"148":1}}],["465299499211943e",{"2":{"258":1}}],["4658514f",{"2":{"256":1}}],["46501",{"2":{"236":1}}],["4650s\\ttraining",{"2":{"219":1}}],["465132f",{"2":{"256":1}}],["4651s\\ttraining",{"2":{"217":1}}],["46514034",{"2":{"179":1}}],["466007803367306e",{"2":{"258":1}}],["4664s\\ttraining",{"2":{"219":1}}],["4662s\\ttraining",{"2":{"217":1}}],["46",{"2":{"196":1,"228":4,"249":1}}],["46396",{"2":{"187":1}}],["46327f",{"2":{"77":1}}],["4612s\\ttraining",{"2":{"219":1}}],["461908",{"2":{"171":1}}],["4619778701480337",{"2":{"153":2}}],["4614",{"2":{"147":1}}],["4687857",{"2":{"148":1}}],["483040474483915e",{"2":{"258":1}}],["48351598",{"2":{"179":1}}],["48983693",{"2":{"241":1}}],["4894s\\ttraining",{"2":{"217":1}}],["485971575869152e",{"2":{"258":1}}],["485373f",{"2":{"256":1}}],["485815f",{"2":{"256":1}}],["48501",{"2":{"236":1}}],["4851346",{"2":{"137":1}}],["480965476587697e",{"2":{"258":1}}],["4809s\\ttraining",{"2":{"217":1}}],["480445915249443e",{"2":{"258":1}}],["480437f",{"2":{"256":1}}],["480524072890363e",{"2":{"258":1}}],["480524f",{"2":{"256":1}}],["480119f",{"2":{"256":1}}],["48027995",{"2":{"241":1}}],["48001",{"2":{"236":1}}],["480",{"2":{"220":3}}],["4846s\\ttraining",{"2":{"217":1}}],["484056",{"2":{"168":1}}],["4877852684387734e",{"2":{"258":1}}],["487726e",{"2":{"202":1}}],["4878186f",{"2":{"256":1}}],["48786",{"2":{"187":1}}],["48",{"2":{"180":3,"189":3,"197":3,"204":3,"221":3,"228":3,"229":3,"238":3,"246":3}}],["482267727648984e",{"2":{"258":1}}],["4827924f",{"2":{"256":1}}],["482031e",{"2":{"202":1}}],["48203f",{"2":{"77":1}}],["48263642",{"2":{"179":1}}],["486484535291516e",{"2":{"258":1}}],["486451f",{"2":{"256":1}}],["486546120877551e",{"2":{"258":1}}],["486550215883336",{"2":{"153":2}}],["486313f",{"2":{"256":1}}],["486379f",{"2":{"256":1}}],["486214",{"2":{"168":1}}],["4880",{"2":{"249":1}}],["488028235902512",{"2":{"153":2}}],["48818898",{"2":{"241":2}}],["488105",{"2":{"147":1,"148":1}}],["4886s\\ttraining",{"2":{"217":1}}],["488623",{"2":{"148":1}}],["481771629997701e",{"2":{"258":1}}],["4818556f",{"2":{"256":1}}],["4816s\\ttraining",{"2":{"217":1}}],["48148",{"2":{"217":6,"219":1}}],["48193252",{"2":{"145":1}}],["4810884",{"2":{"145":1}}],["48137343",{"2":{"126":1}}],["4231267103847656e",{"2":{"258":1}}],["423201f",{"2":{"256":1}}],["4280803756635753e",{"2":{"258":1}}],["4281542f",{"2":{"256":1}}],["42589974148187e",{"2":{"258":1}}],["42519686",{"2":{"241":2}}],["42501",{"2":{"236":1}}],["426589476427531e",{"2":{"258":1}}],["426819f",{"2":{"256":1}}],["4260675f",{"2":{"256":1}}],["426",{"2":{"202":1}}],["42669478",{"2":{"106":1}}],["4225",{"2":{"249":1}}],["422",{"2":{"189":1}}],["422918",{"2":{"171":1}}],["42725",{"2":{"187":1}}],["424662174224403e",{"2":{"258":1}}],["4248958388686553e",{"2":{"258":1}}],["4248398f",{"2":{"256":1}}],["424506f",{"2":{"256":1}}],["42407",{"2":{"187":1}}],["4242767",{"2":{"179":1}}],["421906372894266e",{"2":{"258":1}}],["421658f",{"2":{"256":1}}],["4216236",{"2":{"126":1}}],["42189494",{"2":{"179":1}}],["42",{"2":{"168":2,"196":1,"217":1,"228":2}}],["42001",{"2":{"236":1}}],["420058350197999",{"2":{"153":1}}],["42005835019799886",{"2":{"153":1}}],["420698",{"2":{"147":1,"148":1}}],["403694321425882e",{"2":{"258":1}}],["4036489824359646e",{"2":{"258":1}}],["403785f",{"2":{"256":1}}],["4033567f",{"2":{"256":1}}],["40354207",{"2":{"179":1}}],["4061",{"2":{"249":1}}],["40623155",{"2":{"244":1}}],["4066515",{"2":{"148":1}}],["4055689200670786e",{"2":{"258":1}}],["4059",{"2":{"249":1}}],["40501",{"2":{"236":1}}],["4051151",{"2":{"176":4}}],["40785212729435e",{"2":{"258":1}}],["40789893",{"2":{"179":1}}],["407778f",{"2":{"256":1}}],["40727592",{"2":{"244":1}}],["40741",{"2":{"217":3}}],["40",{"2":{"217":5,"219":1,"228":2}}],["4015367816808866e",{"2":{"258":1}}],["4015749",{"2":{"241":2}}],["4017039f",{"2":{"256":1}}],["4017",{"2":{"249":1}}],["4018757",{"2":{"241":1}}],["401",{"2":{"202":1}}],["401215",{"2":{"117":1}}],["40983",{"2":{"187":1}}],["4028312402278282e",{"2":{"258":1}}],["4029983f",{"2":{"256":1}}],["40209",{"2":{"187":1}}],["40229",{"2":{"171":1}}],["40gb",{"2":{"180":1,"189":1,"204":1,"221":1,"246":1}}],["40473",{"2":{"171":1}}],["404728",{"2":{"171":2}}],["400320559905991e",{"2":{"258":1}}],["400657428789124e",{"2":{"258":1}}],["4002467f",{"2":{"256":1}}],["4004093f",{"2":{"256":1}}],["40001",{"2":{"236":1}}],["4001",{"2":{"179":1,"236":1}}],["4007292",{"2":{"147":1}}],["4007905",{"2":{"145":1}}],["400",{"2":{"107":1}}],["4572842f",{"2":{"256":1}}],["453620448364157e",{"2":{"258":1}}],["4535365f",{"2":{"256":1}}],["45325348",{"2":{"135":1}}],["458806582705095e",{"2":{"258":1}}],["4588464f",{"2":{"256":1}}],["4585s\\ttraining",{"2":{"217":1}}],["4585489",{"2":{"126":1}}],["454155f",{"2":{"256":1}}],["4543",{"2":{"249":1}}],["454679",{"2":{"168":1}}],["4566929",{"2":{"241":2}}],["45583528",{"2":{"241":1}}],["45501",{"2":{"236":1}}],["4552384158732863",{"2":{"174":4}}],["455238",{"2":{"171":1}}],["451716296991064e",{"2":{"258":1}}],["45166f",{"2":{"256":1}}],["451",{"2":{"202":1}}],["4510427",{"2":{"126":1}}],["45",{"2":{"196":1,"228":3}}],["45s",{"2":{"196":1}}],["452119641151299e",{"2":{"258":1}}],["4529514f",{"2":{"256":1}}],["45298263",{"2":{"77":1}}],["452355f",{"2":{"256":1}}],["4524014",{"2":{"241":1}}],["45247704",{"2":{"179":1}}],["45229",{"2":{"187":1}}],["459899260876196e",{"2":{"258":1}}],["4598972775180253e",{"2":{"258":1}}],["45988",{"2":{"187":1}}],["4596436f",{"2":{"256":1}}],["4593505f",{"2":{"256":1}}],["4593922",{"2":{"176":4}}],["4597295f",{"2":{"256":1}}],["4597s\\ttraining",{"2":{"219":1}}],["459019",{"2":{"171":1}}],["450398200839809e",{"2":{"258":1}}],["450661f",{"2":{"256":1}}],["450551207952779e",{"2":{"258":1}}],["4505414f",{"2":{"256":1}}],["450581f",{"2":{"106":5}}],["450085f",{"2":{"256":1}}],["45001",{"2":{"236":1}}],["4500963619011972",{"2":{"153":2}}],["4501",{"2":{"236":1}}],["45016125",{"2":{"179":1}}],["45026",{"2":{"187":1}}],["413206168565263e",{"2":{"258":1}}],["413197f",{"2":{"256":1}}],["418102901866489e",{"2":{"258":1}}],["418967f",{"2":{"256":1}}],["4180625f",{"2":{"256":1}}],["419257849271984e",{"2":{"258":1}}],["4192013f",{"2":{"256":1}}],["4194936534334495e",{"2":{"258":1}}],["4190932f",{"2":{"256":1}}],["41919118",{"2":{"179":1}}],["4117173f",{"2":{"256":1}}],["4112944",{"2":{"106":1}}],["4112945",{"2":{"106":1}}],["4152326060594614e",{"2":{"258":1}}],["4152362f",{"2":{"256":1}}],["41501",{"2":{"236":1}}],["4158693023143286",{"2":{"153":2}}],["415858",{"2":{"106":1}}],["41001",{"2":{"236":1}}],["4101033",{"2":{"147":1}}],["41",{"2":{"228":2}}],["416",{"2":{"193":2}}],["41602",{"2":{"187":1}}],["414476326908213e",{"2":{"258":1}}],["4144732",{"2":{"148":1}}],["414069818099648e",{"2":{"258":1}}],["414769",{"2":{"171":1}}],["41238892",{"2":{"147":1}}],["41246277",{"2":{"126":1}}],["434606304695145e",{"2":{"258":1}}],["434725f",{"2":{"256":1}}],["4320318620669685e",{"2":{"258":1}}],["4320717f",{"2":{"256":1}}],["4322f",{"2":{"77":1}}],["437423713223206e",{"2":{"258":1}}],["4371106f",{"2":{"256":1}}],["4371308",{"2":{"241":1}}],["4379",{"2":{"249":1}}],["4376572711003852",{"2":{"153":2}}],["4357573778699496e",{"2":{"258":1}}],["4353992",{"2":{"241":1}}],["43501",{"2":{"236":1}}],["43001",{"2":{"236":1}}],["4305115e",{"2":{"152":1}}],["4305781",{"2":{"126":1}}],["4364566327798674e",{"2":{"258":1}}],["4364273f",{"2":{"256":1}}],["436657671371589e",{"2":{"258":1}}],["4365966f",{"2":{"256":1}}],["4360055f",{"2":{"256":1}}],["43611",{"2":{"187":1}}],["43676835",{"2":{"147":1}}],["4330709",{"2":{"241":2}}],["4339262",{"2":{"179":1}}],["433656",{"2":{"168":1}}],["43350348",{"2":{"147":1}}],["4319346874291314",{"2":{"171":1}}],["4314566",{"2":{"148":1}}],["4318977",{"2":{"126":1}}],["4397957",{"2":{"145":1}}],["4391564",{"2":{"106":1}}],["43af",{"2":{"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["43",{"2":{"90":1,"196":1,"228":2}}],["4382721",{"2":{"241":1}}],["43846482",{"2":{"179":1}}],["438519",{"2":{"106":1}}],["43866f",{"2":{"77":1}}],["4389",{"2":{"54":1}}],["4779666542064923e",{"2":{"258":1}}],["477876f",{"2":{"256":1}}],["4778283",{"2":{"241":1}}],["473383660049485e",{"2":{"258":1}}],["473091f",{"2":{"256":1}}],["47308",{"2":{"187":1}}],["4730743722547668",{"2":{"153":2}}],["470304002208984e",{"2":{"258":1}}],["47001",{"2":{"236":1}}],["474291846836073e",{"2":{"258":1}}],["474296178099866e",{"2":{"258":1}}],["474001f",{"2":{"256":1}}],["474048f",{"2":{"256":1}}],["474314f",{"2":{"256":1}}],["4791",{"2":{"249":1}}],["475820012643837e",{"2":{"258":1}}],["475479418392951e",{"2":{"258":1}}],["475787f",{"2":{"256":1}}],["47525f",{"2":{"256":1}}],["47501",{"2":{"236":1}}],["47551f",{"2":{"77":1}}],["4780s\\ttraining",{"2":{"217":1}}],["47871",{"2":{"187":1}}],["47872233",{"2":{"145":1}}],["4766496300919e",{"2":{"258":1}}],["476559f",{"2":{"256":1}}],["4761s\\ttraining",{"2":{"219":1}}],["476",{"2":{"202":1}}],["4763277207638013",{"2":{"153":1}}],["4763277207638008",{"2":{"153":1}}],["47",{"2":{"193":2,"228":2}}],["4726s\\ttraining",{"2":{"217":1}}],["4721s\\ttraining",{"2":{"217":1}}],["472940619324102e",{"2":{"258":1}}],["47294",{"2":{"187":1}}],["472",{"2":{"52":1}}],["4945065063297755e",{"2":{"258":1}}],["4940985f",{"2":{"256":1}}],["497101058174528e",{"2":{"258":1}}],["497115883027829e",{"2":{"258":1}}],["4971752f",{"2":{"256":1}}],["492708f",{"2":{"256":1}}],["4968866f",{"2":{"256":1}}],["4968",{"2":{"249":1}}],["496063",{"2":{"241":2}}],["495236057110264e",{"2":{"258":1}}],["4952027f",{"2":{"256":1}}],["4953518517909694e",{"2":{"258":1}}],["495184f",{"2":{"256":1}}],["49501",{"2":{"236":1}}],["4958s\\ttraining",{"2":{"217":1}}],["49001",{"2":{"236":1}}],["4907s\\ttraining",{"2":{"217":1}}],["4901161f",{"2":{"106":6}}],["49012f",{"2":{"106":2}}],["4936395820894294e",{"2":{"258":1}}],["4936414",{"2":{"241":1}}],["493931403578232e",{"2":{"258":1}}],["4938067f",{"2":{"256":1}}],["4937",{"2":{"249":1}}],["4934s\\ttraining",{"2":{"217":1}}],["49311",{"2":{"187":1}}],["49804282",{"2":{"241":1}}],["49884",{"2":{"187":1}}],["4981351",{"2":{"106":1}}],["499163956770091e",{"2":{"258":1}}],["499174f",{"2":{"256":1}}],["49913f",{"2":{"77":1}}],["49998975",{"2":{"106":1}}],["49",{"2":{"69":1,"228":2,"242":1,"245":1}}],["4d",{"2":{"56":3,"107":2}}],["4496882239097885e",{"2":{"258":1}}],["4496549f",{"2":{"256":1}}],["449373932950347e",{"2":{"258":1}}],["449255f",{"2":{"256":1}}],["4466310155355671e",{"2":{"258":1}}],["4467216f",{"2":{"256":1}}],["446214",{"2":{"147":1}}],["442299435870841e",{"2":{"258":1}}],["442865419446369e",{"2":{"258":1}}],["44239f",{"2":{"256":1}}],["4420",{"2":{"249":1}}],["44407824800849e",{"2":{"258":1}}],["444711997549296e",{"2":{"258":1}}],["444673f",{"2":{"256":1}}],["44439f",{"2":{"77":1}}],["4415182871570897e",{"2":{"258":1}}],["4415452028142198e",{"2":{"258":1}}],["441302581421965e",{"2":{"258":1}}],["4416792f",{"2":{"256":1}}],["4419071f",{"2":{"256":1}}],["4410673f",{"2":{"256":1}}],["4417892998606726e",{"2":{"258":1}}],["4417893f",{"2":{"256":1}}],["441713f",{"2":{"256":1}}],["4418456",{"2":{"179":1}}],["4472",{"2":{"249":1}}],["44746",{"2":{"171":1}}],["44096947",{"2":{"244":1}}],["44078717",{"2":{"241":1}}],["44001",{"2":{"236":1}}],["44048917",{"2":{"179":1}}],["4456",{"2":{"249":1}}],["44501",{"2":{"236":1}}],["4458s\\ttraining",{"2":{"217":1}}],["44571686",{"2":{"148":1}}],["4488101521328277",{"2":{"174":1}}],["44807646",{"2":{"107":1}}],["443927507361983e",{"2":{"258":1}}],["443972f",{"2":{"256":1}}],["44397",{"2":{"148":1}}],["4437604f",{"2":{"256":1}}],["44374",{"2":{"187":1}}],["4437156",{"2":{"147":1}}],["44",{"2":{"41":1,"196":4,"228":2,"249":1}}],["4",{"2":{"2":1,"28":1,"38":1,"39":1,"41":8,"50":1,"52":4,"54":1,"55":5,"62":2,"63":2,"69":2,"77":3,"78":1,"102":1,"106":2,"107":4,"115":8,"118":6,"126":5,"129":8,"131":1,"135":2,"136":2,"139":8,"140":4,"145":9,"146":2,"147":3,"148":11,"152":7,"153":16,"155":2,"171":15,"173":5,"176":2,"179":1,"180":3,"187":17,"189":3,"193":2,"195":1,"196":2,"204":3,"217":5,"219":1,"220":33,"221":2,"224":1,"228":2,"229":1,"236":9,"238":1,"241":6,"244":1,"246":3,"248":9,"249":20,"254":7,"255":1,"256":100,"258":92}}],["0f4",{"2":{"255":1}}],["0f",{"2":{"195":1,"215":2}}],["0f0",{"2":{"69":2,"78":2,"117":2,"118":1,"152":2,"183":2,"200":4,"214":4,"218":2,"235":8,"237":6,"241":2,"248":4,"255":1}}],["0+560",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["0x1911b814c02405e8",{"2":{"174":1}}],["0x12c522b8034ae186",{"2":{"126":1}}],["0x8c49bc52dc8a77ea",{"2":{"174":1}}],["0x8e0c3a65079041bb",{"2":{"126":1}}],["0x48d73dc42d195740",{"2":{"174":1}}],["0x4fa3403dd074e603",{"2":{"126":1}}],["0xdb2fa90498613fdf",{"2":{"174":1}}],["0x22a21880af5dc689",{"2":{"126":1,"174":1}}],["0x21617f7747d97206",{"2":{"126":1}}],["0e",{"2":{"77":7,"243":1,"245":1}}],["0835724714158276e",{"2":{"258":1}}],["0837523f",{"2":{"256":1}}],["08368",{"2":{"187":1}}],["0836414",{"2":{"168":1}}],["0813",{"2":{"249":1}}],["08131",{"2":{"187":1}}],["08259142636652e",{"2":{"258":1}}],["082086",{"2":{"245":1}}],["0829282",{"2":{"145":1}}],["0845460802201694e",{"2":{"258":1}}],["08456781",{"2":{"77":1}}],["0841573f",{"2":{"256":1}}],["0847823f",{"2":{"256":1}}],["0842",{"2":{"249":1}}],["0843f",{"2":{"256":1}}],["0843",{"2":{"249":1}}],["08486",{"2":{"187":1}}],["08",{"2":{"177":1}}],["085946251633898e",{"2":{"258":1}}],["0851128023826387e",{"2":{"258":1}}],["08517",{"2":{"171":1}}],["085170805",{"2":{"171":1}}],["0851708",{"2":{"171":2}}],["08569",{"2":{"187":1}}],["08526564",{"2":{"77":1}}],["0885",{"2":{"249":1}}],["0887",{"2":{"249":1}}],["08820387",{"2":{"241":1}}],["088205874",{"2":{"106":1}}],["0884834705765853",{"2":{"153":2}}],["0860570433619975e",{"2":{"258":1}}],["0860472f",{"2":{"256":1}}],["0867",{"2":{"249":1}}],["0865",{"2":{"249":1}}],["0866141",{"2":{"241":2}}],["08667634",{"2":{"106":2}}],["08694684883050086",{"2":{"174":4}}],["0869468",{"2":{"171":1}}],["086429894",{"2":{"147":1}}],["0876",{"2":{"249":1}}],["08796",{"2":{"187":1}}],["08791955",{"2":{"106":1}}],["08713347",{"2":{"137":1}}],["089009336341235e",{"2":{"258":1}}],["089302f",{"2":{"256":1}}],["08922641",{"2":{"241":1}}],["089530945",{"2":{"106":1}}],["08953094",{"2":{"106":1}}],["08993f",{"2":{"77":1}}],["080294160037954e",{"2":{"258":1}}],["08022",{"2":{"19":1,"55":1}}],["0803379",{"2":{"241":1}}],["0801143f",{"2":{"256":1}}],["08010",{"2":{"187":1}}],["08012",{"2":{"187":1}}],["0801",{"2":{"78":1}}],["08085155",{"2":{"78":1}}],["03683116765555e",{"2":{"258":1}}],["03f0",{"2":{"243":1}}],["03",{"2":{"202":5,"243":1,"245":1}}],["037173817773901e",{"2":{"258":1}}],["0371243f",{"2":{"256":1}}],["03704",{"2":{"217":3}}],["03776",{"2":{"187":1}}],["03731",{"2":{"187":1}}],["037500158",{"2":{"106":1}}],["034966860497914e",{"2":{"258":1}}],["03473",{"2":{"187":1}}],["03488",{"2":{"187":1}}],["034259234",{"2":{"179":1}}],["035324779008351e",{"2":{"258":1}}],["035999415113797e",{"2":{"258":1}}],["035966f",{"2":{"256":1}}],["0359876f",{"2":{"256":1}}],["0351315f",{"2":{"256":1}}],["035557814",{"2":{"236":1}}],["0355555",{"2":{"171":1}}],["03550",{"2":{"187":1}}],["03507",{"2":{"187":1}}],["0350433",{"2":{"77":1}}],["03574",{"2":{"187":1}}],["03575",{"2":{"187":1}}],["0333516733809932e",{"2":{"258":1}}],["0334577f",{"2":{"256":1}}],["03342",{"2":{"187":1}}],["03322105",{"2":{"241":1}}],["03375s\\ttraining",{"2":{"228":1}}],["03377",{"2":{"187":1}}],["03378",{"2":{"187":1}}],["0338569",{"2":{"147":1}}],["03306",{"2":{"187":1}}],["03306928",{"2":{"77":1}}],["0330944",{"2":{"77":1}}],["03192185969298e",{"2":{"258":1}}],["0319617f",{"2":{"256":1}}],["03193",{"2":{"187":1}}],["031860039",{"2":{"236":1}}],["03181",{"2":{"187":1}}],["03174",{"2":{"187":1}}],["03160",{"2":{"187":1}}],["031414255389526885",{"2":{"153":2}}],["031403534",{"2":{"106":1}}],["031403527",{"2":{"106":1}}],["031501673",{"2":{"236":1}}],["031503614",{"2":{"106":1}}],["03156",{"2":{"187":1}}],["0315817",{"2":{"77":1}}],["03209",{"2":{"187":1}}],["0320963",{"2":{"106":1}}],["03212",{"2":{"187":1}}],["03216",{"2":{"187":1}}],["03210",{"2":{"187":1}}],["032705",{"2":{"241":1}}],["03270",{"2":{"187":1}}],["03265",{"2":{"187":1}}],["03261588",{"2":{"106":1}}],["032615878",{"2":{"106":1}}],["03258145",{"2":{"179":1}}],["032846544",{"2":{"147":1}}],["032405667",{"2":{"106":1}}],["032405674",{"2":{"106":1}}],["0324213",{"2":{"106":1}}],["03242131",{"2":{"106":1}}],["03967",{"2":{"187":1}}],["03966",{"2":{"187":1}}],["039647065",{"2":{"106":2}}],["039132368",{"2":{"77":1}}],["030308052229863e",{"2":{"258":1}}],["03038",{"2":{"187":1}}],["0306004f",{"2":{"256":1}}],["030016",{"2":{"245":1}}],["03001s\\ttraining",{"2":{"228":1}}],["030411797",{"2":{"236":1}}],["030424416",{"2":{"236":1}}],["03082",{"2":{"187":1}}],["030881835",{"2":{"106":2}}],["0305068f",{"2":{"256":1}}],["03052",{"2":{"187":1}}],["03051",{"2":{"187":1}}],["03070261",{"2":{"107":1}}],["030926049",{"2":{"106":1}}],["030926052",{"2":{"106":1}}],["03094203",{"2":{"77":1}}],["030193644566294e",{"2":{"258":1}}],["0301",{"2":{"78":1}}],["0302464",{"2":{"77":1}}],["03861",{"2":{"187":1}}],["03839",{"2":{"187":1}}],["03821",{"2":{"187":1}}],["0381317",{"2":{"148":1}}],["0381727",{"2":{"147":1}}],["03804328",{"2":{"77":1}}],["038",{"2":{"55":3}}],["02f0",{"2":{"237":2}}],["02",{"2":{"202":17}}],["021537139061442e",{"2":{"258":1}}],["02151",{"2":{"187":1}}],["021353512286594e",{"2":{"258":1}}],["021357f",{"2":{"256":1}}],["02139s\\ttraining",{"2":{"228":1}}],["02172139980526e",{"2":{"258":1}}],["0217423f",{"2":{"256":1}}],["021707822",{"2":{"236":1}}],["021692f",{"2":{"256":1}}],["021622123",{"2":{"236":1}}],["021639718",{"2":{"236":1}}],["021128871",{"2":{"236":1}}],["021179065",{"2":{"236":1}}],["021835698",{"2":{"236":1}}],["021062549",{"2":{"236":1}}],["021029573",{"2":{"236":1}}],["02121",{"2":{"187":1}}],["02120",{"2":{"187":1}}],["02142",{"2":{"187":1}}],["02143306",{"2":{"107":1}}],["02195",{"2":{"187":1}}],["0256",{"2":{"249":1}}],["025311429",{"2":{"236":1}}],["025349757",{"2":{"236":1}}],["025398642",{"2":{"236":1}}],["02535537",{"2":{"179":1}}],["02535769",{"2":{"147":1}}],["025281416",{"2":{"236":1}}],["02525",{"2":{"187":1}}],["025583766",{"2":{"236":1}}],["02550",{"2":{"187":1}}],["02595",{"2":{"187":1}}],["02597",{"2":{"187":1}}],["02505",{"2":{"187":1}}],["029747024",{"2":{"236":1}}],["029854555",{"2":{"236":1}}],["02960",{"2":{"187":1}}],["02964765308691042",{"2":{"174":4}}],["0296477",{"2":{"171":1}}],["02924",{"2":{"187":1}}],["029290024",{"2":{"77":1}}],["02933",{"2":{"187":1}}],["0288",{"2":{"249":1}}],["02861",{"2":{"187":1}}],["02803",{"2":{"187":1}}],["0285962",{"2":{"147":1}}],["028738942",{"2":{"147":1}}],["028461456",{"2":{"126":1}}],["02827571",{"2":{"106":1}}],["028275706",{"2":{"106":1}}],["028251555",{"2":{"77":1}}],["0278612f",{"2":{"256":1}}],["02788",{"2":{"187":1}}],["02795",{"2":{"187":1}}],["0277897",{"2":{"241":1}}],["02777",{"2":{"187":1}}],["027702922",{"2":{"77":1}}],["02758",{"2":{"187":1}}],["027250724194729e",{"2":{"258":1}}],["027273539",{"2":{"236":1}}],["02724",{"2":{"187":1}}],["027226416",{"2":{"106":1}}],["027226413",{"2":{"106":1}}],["02711",{"2":{"187":1}}],["0204",{"2":{"249":1}}],["020470668",{"2":{"236":1}}],["02054s\\ttraining",{"2":{"228":1}}],["02028",{"2":{"187":1}}],["02020",{"2":{"187":1}}],["0209",{"2":{"249":1}}],["020919006",{"2":{"236":1}}],["020974938",{"2":{"236":1}}],["02098",{"2":{"187":1}}],["02096",{"2":{"187":1}}],["02093",{"2":{"187":1}}],["02094",{"2":{"187":1}}],["02081",{"2":{"187":1}}],["02034",{"2":{"187":1}}],["02074",{"2":{"187":1}}],["02073",{"2":{"187":1}}],["0207279",{"2":{"135":1}}],["0207926",{"2":{"77":1}}],["020194370",{"2":{"236":1}}],["02010",{"2":{"187":1}}],["02011",{"2":{"187":1}}],["0201",{"2":{"78":1}}],["023622",{"2":{"241":2}}],["023531647",{"2":{"236":1}}],["02354",{"2":{"187":1}}],["02317",{"2":{"187":1}}],["0231775",{"2":{"77":1}}],["02377",{"2":{"187":1}}],["02374",{"2":{"187":1}}],["02339",{"2":{"187":1}}],["023031702",{"2":{"77":1}}],["022515260",{"2":{"236":1}}],["022515845",{"2":{"236":1}}],["022652669",{"2":{"236":1}}],["02263",{"2":{"187":1}}],["022343241",{"2":{"236":1}}],["022420531",{"2":{"236":1}}],["022408014",{"2":{"236":1}}],["022434518",{"2":{"77":1}}],["022167893",{"2":{"236":1}}],["022159226",{"2":{"236":1}}],["022175739",{"2":{"236":1}}],["02210",{"2":{"187":1}}],["02206s\\ttraining",{"2":{"228":1}}],["02200",{"2":{"187":1}}],["022215",{"2":{"245":1}}],["02221",{"2":{"187":1}}],["02274",{"2":{"187":1}}],["0227737",{"2":{"77":1}}],["02288",{"2":{"187":1}}],["02284",{"2":{"187":1}}],["0228109822209213",{"2":{"153":2}}],["02298",{"2":{"187":1}}],["02294",{"2":{"187":2}}],["026318f",{"2":{"256":1}}],["026282769",{"2":{"236":1}}],["026193798",{"2":{"236":1}}],["026170360",{"2":{"236":1}}],["02618",{"2":{"187":1}}],["02615",{"2":{"187":1}}],["026762877",{"2":{"236":1}}],["02676",{"2":{"187":1}}],["02678",{"2":{"187":1}}],["02662",{"2":{"187":1}}],["02645",{"2":{"187":1}}],["026851978",{"2":{"77":1}}],["026516732",{"2":{"77":1}}],["0242",{"2":{"249":1}}],["0246403f",{"2":{"256":1}}],["0246",{"2":{"249":1}}],["024626993",{"2":{"236":1}}],["0240",{"2":{"249":1}}],["02477",{"2":{"187":1}}],["02439",{"2":{"187":1}}],["02436",{"2":{"187":1}}],["02484",{"2":{"187":1}}],["024822138",{"2":{"236":1}}],["02482",{"2":{"187":1}}],["0244548f",{"2":{"106":1}}],["0244214",{"2":{"77":1}}],["024",{"2":{"77":1}}],["0943527292192e",{"2":{"258":1}}],["0944266f",{"2":{"256":1}}],["09474",{"2":{"187":1}}],["09",{"2":{"228":2,"249":1}}],["093940150376217e",{"2":{"258":1}}],["093925126",{"2":{"179":1}}],["093884f",{"2":{"256":1}}],["0935077",{"2":{"254":2}}],["09351656",{"2":{"177":1,"178":1}}],["098333749713619e",{"2":{"258":1}}],["098487989318162e",{"2":{"258":1}}],["098702223620674e",{"2":{"258":1}}],["0987538",{"2":{"168":1}}],["098295f",{"2":{"256":1}}],["09851f",{"2":{"256":1}}],["09852",{"2":{"171":1}}],["0985227",{"2":{"171":2}}],["098693f",{"2":{"256":1}}],["098861866",{"2":{"77":1}}],["0912873278510902e",{"2":{"258":1}}],["0912197f",{"2":{"256":1}}],["091627955358855e",{"2":{"258":1}}],["0913612f",{"2":{"256":1}}],["0913593358445652",{"2":{"153":1}}],["091359335844565",{"2":{"153":1}}],["09154",{"2":{"187":1}}],["0914626",{"2":{"148":1}}],["092454119983505e",{"2":{"258":1}}],["092215427875536e",{"2":{"258":1}}],["0922765f",{"2":{"256":1}}],["0926352f",{"2":{"256":1}}],["0927673f",{"2":{"256":1}}],["0923590437726185e",{"2":{"258":1}}],["09235f",{"2":{"256":1}}],["09235",{"2":{"187":1}}],["0928834",{"2":{"168":1}}],["09289699",{"2":{"106":1}}],["09299411",{"2":{"107":1}}],["09648f",{"2":{"256":1}}],["09643307",{"2":{"106":2}}],["096519953127111e",{"2":{"258":1}}],["0965",{"2":{"249":1}}],["09693718",{"2":{"244":1}}],["09699093",{"2":{"147":1}}],["096988946",{"2":{"126":1}}],["09692798",{"2":{"77":1}}],["09591761099469e",{"2":{"258":1}}],["09599f",{"2":{"77":1}}],["095683348120538e",{"2":{"258":1}}],["0958435f",{"2":{"256":1}}],["0952946f",{"2":{"256":1}}],["095568806",{"2":{"244":1}}],["0954186",{"2":{"106":1}}],["0991955",{"2":{"106":1}}],["09919551",{"2":{"106":1}}],["099906564",{"2":{"106":1}}],["09926938",{"2":{"106":1}}],["097892033761677e",{"2":{"258":1}}],["09785451",{"2":{"77":1}}],["0976364f",{"2":{"256":1}}],["0974325678648426e",{"2":{"258":1}}],["0974424f",{"2":{"256":1}}],["09745",{"2":{"187":1}}],["0973",{"2":{"249":1}}],["0971",{"2":{"249":1}}],["097182155",{"2":{"106":1}}],["09703",{"2":{"187":1}}],["09701932",{"2":{"179":1}}],["097253",{"2":{"126":1}}],["09724942",{"2":{"106":1}}],["09724939",{"2":{"106":1}}],["097701035",{"2":{"106":2}}],["090212652653063e",{"2":{"258":1}}],["09038",{"2":{"187":1}}],["0903271",{"2":{"106":2}}],["0901905f",{"2":{"256":1}}],["0901",{"2":{"78":1}}],["09098f",{"2":{"77":1}}],["0900306",{"2":{"63":10}}],["076103048767231e",{"2":{"258":1}}],["076042f",{"2":{"256":1}}],["076404385",{"2":{"106":1}}],["07640441",{"2":{"106":1}}],["075994766110806e",{"2":{"258":1}}],["075956f",{"2":{"256":1}}],["07565363",{"2":{"244":1}}],["075135306",{"2":{"147":1}}],["07",{"2":{"228":1}}],["07270",{"2":{"187":1}}],["0728675615927385",{"2":{"63":1}}],["0733",{"2":{"249":1}}],["07375115",{"2":{"244":1}}],["07391",{"2":{"187":1}}],["073442355",{"2":{"77":1}}],["0782",{"2":{"249":1}}],["0787",{"2":{"249":1}}],["07874016",{"2":{"241":2}}],["07846",{"2":{"187":1}}],["07868",{"2":{"187":1}}],["0771742893068196e",{"2":{"258":1}}],["07743",{"2":{"187":1}}],["0770072f",{"2":{"256":1}}],["07702",{"2":{"171":1}}],["0770206",{"2":{"171":2}}],["07704306",{"2":{"77":1}}],["07945452398759e",{"2":{"258":1}}],["079785f",{"2":{"256":1}}],["079723e",{"2":{"202":1}}],["0796850313797592e",{"2":{"258":1}}],["0796941f",{"2":{"256":1}}],["07964921",{"2":{"107":1}}],["079208925",{"2":{"118":1}}],["0742047",{"2":{"241":1}}],["07425846",{"2":{"106":1}}],["07425845",{"2":{"106":1}}],["07412187",{"2":{"77":1}}],["0709",{"2":{"241":1}}],["0701",{"2":{"78":1}}],["07058401",{"2":{"77":1}}],["07057328",{"2":{"77":1}}],["0715908737533902e",{"2":{"258":1}}],["0715298f",{"2":{"256":1}}],["071482725",{"2":{"179":1}}],["07133968",{"2":{"109":1}}],["07167f",{"2":{"77":1}}],["07125676",{"2":{"77":1}}],["0444",{"2":{"249":1}}],["04416",{"2":{"245":1}}],["04414",{"2":{"187":1}}],["04455",{"2":{"187":1}}],["04453",{"2":{"187":1}}],["049318592096048e",{"2":{"258":1}}],["0493577f",{"2":{"256":1}}],["0499726f",{"2":{"256":1}}],["04903387",{"2":{"179":1}}],["049417756",{"2":{"106":1}}],["049417753",{"2":{"106":1}}],["0435",{"2":{"249":1}}],["0435721",{"2":{"77":1}}],["043140102",{"2":{"236":1}}],["04306",{"2":{"187":1}}],["043883",{"2":{"171":1}}],["04191",{"2":{"187":1}}],["04199602",{"2":{"179":1}}],["04194im",{"2":{"168":1}}],["041031003",{"2":{"145":1}}],["04108f",{"2":{"77":1}}],["046537939462137e",{"2":{"258":1}}],["046538f",{"2":{"256":1}}],["04674",{"2":{"187":1}}],["04691",{"2":{"187":1}}],["0469301",{"2":{"148":1}}],["0462954",{"2":{"148":1}}],["040289409033149e",{"2":{"258":1}}],["0402095",{"2":{"147":1}}],["0407607765980987e",{"2":{"258":1}}],["0406422f",{"2":{"256":1}}],["040918142",{"2":{"236":1}}],["04043",{"2":{"187":1}}],["04086",{"2":{"187":1}}],["0401096f",{"2":{"256":1}}],["0401",{"2":{"78":1}}],["04019f",{"2":{"77":1}}],["0451268516464432e",{"2":{"258":1}}],["0451178f",{"2":{"256":1}}],["045591548",{"2":{"236":1}}],["04551310",{"2":{"107":1}}],["045338e",{"2":{"202":1}}],["04527",{"2":{"187":1}}],["04566476",{"2":{"106":1}}],["0473993552825355e",{"2":{"258":1}}],["047356773",{"2":{"77":1}}],["0471922024922518e",{"2":{"258":1}}],["0471824f",{"2":{"256":1}}],["0470862f",{"2":{"256":1}}],["047244094",{"2":{"241":2}}],["04776",{"2":{"187":1}}],["047897488",{"2":{"106":2}}],["0488062668986844e",{"2":{"258":1}}],["048816f",{"2":{"256":1}}],["048811335",{"2":{"106":2}}],["04829",{"2":{"187":1}}],["04815",{"2":{"187":1}}],["048109",{"2":{"148":1}}],["048133414",{"2":{"106":2}}],["048522998",{"2":{"129":1}}],["04853325",{"2":{"106":1}}],["048533253",{"2":{"106":1}}],["0480501",{"2":{"77":1}}],["04d",{"2":{"78":1}}],["0428307",{"2":{"241":1}}],["04214",{"2":{"187":1}}],["04265762",{"2":{"179":1}}],["0425432",{"2":{"147":1}}],["0420834",{"2":{"77":1}}],["042254835",{"2":{"106":1}}],["04225484",{"2":{"106":1}}],["042256515",{"2":{"106":1}}],["04224006",{"2":{"77":1}}],["04226003",{"2":{"77":1}}],["0612031156531617e",{"2":{"258":1}}],["0610467f",{"2":{"256":1}}],["06113f",{"2":{"256":1}}],["0618",{"2":{"249":1}}],["0613",{"2":{"249":1}}],["0672",{"2":{"249":1}}],["067872316",{"2":{"77":1}}],["0690823",{"2":{"244":1}}],["06942138",{"2":{"77":1}}],["0646901052186956e",{"2":{"258":1}}],["064526767928963e",{"2":{"258":1}}],["06450",{"2":{"19":1}}],["064764f",{"2":{"256":1}}],["064775f",{"2":{"256":1}}],["06433817",{"2":{"241":1}}],["0686756",{"2":{"241":1}}],["06830",{"2":{"187":1}}],["062894453548825e",{"2":{"258":1}}],["0628546f",{"2":{"256":1}}],["062221f",{"2":{"256":1}}],["062907",{"2":{"245":1}}],["06252",{"2":{"187":1}}],["062155924699545",{"2":{"153":1}}],["0621559246995447",{"2":{"153":1}}],["062079933",{"2":{"147":1}}],["066689924848949e",{"2":{"258":1}}],["06669",{"2":{"131":2}}],["066303726145864e",{"2":{"258":1}}],["06634",{"2":{"187":1}}],["0665713f",{"2":{"256":1}}],["066128",{"2":{"148":1}}],["06612328",{"2":{"107":1}}],["06618893",{"2":{"106":1}}],["06675106",{"2":{"106":1}}],["066751055",{"2":{"106":1}}],["063228054300107e",{"2":{"258":1}}],["06325f",{"2":{"77":1}}],["0633",{"2":{"249":1}}],["06335",{"2":{"187":1}}],["06301335",{"2":{"106":1}}],["063480526",{"2":{"77":1}}],["060015640986507e",{"2":{"258":1}}],["060059104442697e",{"2":{"258":1}}],["0600895f",{"2":{"256":1}}],["060099f",{"2":{"256":1}}],["06049",{"2":{"187":1}}],["06027697",{"2":{"106":1}}],["06026962",{"2":{"77":1}}],["060143257221312e",{"2":{"258":1}}],["0601",{"2":{"78":1,"249":1}}],["060364496",{"2":{"77":1}}],["06053336",{"2":{"77":1}}],["06574688238783e",{"2":{"258":1}}],["065769f",{"2":{"256":1}}],["06519687",{"2":{"107":1}}],["06550143",{"2":{"77":1}}],["065317236",{"2":{"77":1}}],["06",{"2":{"63":1,"196":1}}],["0584",{"2":{"249":1}}],["0585f",{"2":{"77":1}}],["0510362f",{"2":{"256":1}}],["0510",{"2":{"249":1}}],["05111",{"2":{"187":1}}],["05f0",{"2":{"236":2,"237":1}}],["052803104856694e",{"2":{"258":1}}],["05207",{"2":{"187":1}}],["05218964",{"2":{"179":1}}],["055919900797779e",{"2":{"258":1}}],["05596f",{"2":{"256":1}}],["05594",{"2":{"187":1}}],["0551181",{"2":{"241":2}}],["05514",{"2":{"187":1}}],["0555s\\ttraining",{"2":{"217":1}}],["05569",{"2":{"187":1}}],["05566",{"2":{"187":1}}],["05580",{"2":{"187":1}}],["0545969271800749e",{"2":{"258":1}}],["05451",{"2":{"187":1}}],["0548262f",{"2":{"256":1}}],["05485",{"2":{"187":1}}],["0546",{"2":{"249":1}}],["0540",{"2":{"249":1}}],["0540537",{"2":{"77":1}}],["059800547535577e",{"2":{"258":1}}],["0590504757222437e",{"2":{"258":1}}],["0595074f",{"2":{"256":1}}],["059518166",{"2":{"106":2}}],["0592182f",{"2":{"256":1}}],["05944",{"2":{"187":1}}],["057242040728919e",{"2":{"258":1}}],["05729505",{"2":{"106":1}}],["0579",{"2":{"249":1}}],["0578545",{"2":{"244":1}}],["05747",{"2":{"187":1}}],["057005208",{"2":{"177":1,"178":1}}],["05779423",{"2":{"126":1}}],["0534",{"2":{"249":1}}],["0536",{"2":{"249":1}}],["053627778",{"2":{"106":1}}],["05399",{"2":{"187":1}}],["053747915",{"2":{"106":2}}],["056504248706273e",{"2":{"258":1}}],["05657739601024536",{"2":{"174":1}}],["0563847f",{"2":{"256":1}}],["0568",{"2":{"249":1}}],["0569",{"2":{"249":1}}],["05660",{"2":{"187":1}}],["0564903986057956",{"2":{"153":2}}],["056",{"2":{"78":3,"256":1}}],["050228301562713e",{"2":{"258":1}}],["0506279855235563e",{"2":{"258":1}}],["0501",{"2":{"78":1}}],["05058f",{"2":{"77":1}}],["050089",{"2":{"77":1}}],["05",{"2":{"63":1,"228":2,"249":1}}],["00",{"2":{"196":8,"228":47}}],["008555712",{"2":{"236":1}}],["00853",{"2":{"187":1}}],["008431979",{"2":{"236":1}}],["008468037",{"2":{"236":1}}],["008468761",{"2":{"236":1}}],["008464416",{"2":{"236":1}}],["008424449",{"2":{"236":1}}],["00848",{"2":{"187":1}}],["008130345",{"2":{"236":1}}],["008149015",{"2":{"236":1}}],["00815",{"2":{"187":1}}],["00831",{"2":{"187":1}}],["00830",{"2":{"187":1}}],["00832",{"2":{"187":1}}],["008759783488835e",{"2":{"258":1}}],["008753551",{"2":{"236":1}}],["008758042",{"2":{"236":1}}],["008725122",{"2":{"236":1}}],["008737788",{"2":{"236":1}}],["008712809",{"2":{"236":1}}],["00879",{"2":{"187":1}}],["008777075",{"2":{"236":1}}],["00877",{"2":{"187":1}}],["00874",{"2":{"187":1}}],["008019534",{"2":{"236":1}}],["008029928",{"2":{"236":1}}],["008085096",{"2":{"236":1}}],["00808",{"2":{"187":1}}],["00805",{"2":{"187":1}}],["00800",{"2":{"187":1}}],["00804",{"2":{"187":1}}],["008093885",{"2":{"236":1}}],["00809",{"2":{"187":1}}],["00807",{"2":{"187":1}}],["008276105",{"2":{"236":1}}],["008223927",{"2":{"236":1}}],["00825",{"2":{"187":1}}],["00824",{"2":{"187":1}}],["00823197",{"2":{"171":1}}],["008846317298144e",{"2":{"258":1}}],["008846736",{"2":{"236":1}}],["0088503f",{"2":{"256":1}}],["008868203",{"2":{"236":1}}],["008868832",{"2":{"236":1}}],["008882683",{"2":{"236":1}}],["00883",{"2":{"187":1}}],["00880",{"2":{"187":1}}],["008870317",{"2":{"77":1}}],["008658717",{"2":{"236":1}}],["00868",{"2":{"187":1}}],["00866",{"2":{"187":1}}],["008956734",{"2":{"236":1}}],["00897",{"2":{"187":1}}],["00896",{"2":{"187":1}}],["007414200",{"2":{"236":1}}],["00746",{"2":{"187":1}}],["007530828",{"2":{"236":1}}],["007592318",{"2":{"236":1}}],["007622246",{"2":{"236":1}}],["007691749",{"2":{"236":1}}],["007675199",{"2":{"236":1}}],["007164373",{"2":{"236":1}}],["007191297",{"2":{"236":1}}],["007141389",{"2":{"236":1}}],["007100587",{"2":{"236":1}}],["007786f",{"2":{"256":1}}],["007742645",{"2":{"236":1}}],["007718043",{"2":{"236":1}}],["007729640",{"2":{"236":1}}],["007302467",{"2":{"236":1}}],["007303775",{"2":{"236":1}}],["007354409",{"2":{"236":1}}],["007363342",{"2":{"236":1}}],["00736",{"2":{"187":1}}],["0073489705",{"2":{"106":2}}],["007278774",{"2":{"236":1}}],["00729",{"2":{"187":1}}],["00724",{"2":{"187":1}}],["007249862",{"2":{"106":1}}],["0072498624",{"2":{"106":1}}],["00709",{"2":{"187":1}}],["00706411",{"2":{"77":1}}],["007827340",{"2":{"236":1}}],["00786",{"2":{"187":1}}],["007841279",{"2":{"236":1}}],["00784",{"2":{"187":2}}],["007981384",{"2":{"236":1}}],["007984845",{"2":{"147":1}}],["007904682",{"2":{"236":1}}],["007905646",{"2":{"236":1}}],["007971080",{"2":{"236":1}}],["007934441",{"2":{"236":1}}],["007926281",{"2":{"236":1}}],["00799",{"2":{"187":1}}],["00791",{"2":{"187":1}}],["007966863",{"2":{"236":1}}],["00796",{"2":{"187":1}}],["009802144",{"2":{"236":1}}],["009817538",{"2":{"236":1}}],["009810397",{"2":{"236":1}}],["009220880951566e",{"2":{"258":1}}],["009200627",{"2":{"236":1}}],["009236576",{"2":{"236":1}}],["009299970",{"2":{"236":1}}],["009428f",{"2":{"256":1}}],["009428588",{"2":{"236":1}}],["009412916",{"2":{"236":1}}],["009447398",{"2":{"236":1}}],["00944",{"2":{"187":1}}],["009382382",{"2":{"236":1}}],["009318714",{"2":{"236":1}}],["009342002",{"2":{"236":1}}],["00936",{"2":{"187":1}}],["009321969454423e",{"2":{"258":1}}],["009327068",{"2":{"236":1}}],["009320782",{"2":{"236":1}}],["00932",{"2":{"187":1}}],["009071054",{"2":{"236":1}}],["009029351",{"2":{"236":1}}],["009067146",{"2":{"236":1}}],["00900",{"2":{"187":1}}],["009084041f0",{"2":{"63":1}}],["0096115f",{"2":{"256":1}}],["009639673",{"2":{"236":1}}],["00962",{"2":{"187":1}}],["00968",{"2":{"187":1}}],["009987737",{"2":{"236":1}}],["00999",{"2":{"187":3}}],["009963844931984",{"2":{"153":2}}],["0095045531156137e",{"2":{"258":1}}],["0095046f",{"2":{"256":1}}],["009546057",{"2":{"236":1}}],["00952",{"2":{"187":1}}],["009580643",{"2":{"236":1}}],["00958",{"2":{"187":1}}],["009578831",{"2":{"77":1}}],["009109061",{"2":{"236":1}}],["009125081",{"2":{"236":1}}],["009115184",{"2":{"236":1}}],["009163789",{"2":{"236":1}}],["009147485",{"2":{"236":1}}],["00914141",{"2":{"179":1}}],["009196415",{"2":{"236":1}}],["00918",{"2":{"187":1}}],["00915",{"2":{"187":1}}],["009766437",{"2":{"241":1}}],["009731424",{"2":{"236":1}}],["00973899",{"2":{"131":2}}],["009787671",{"2":{"236":1}}],["009795113",{"2":{"236":1}}],["00977",{"2":{"187":1}}],["005813450",{"2":{"236":1}}],["005897780",{"2":{"236":1}}],["005892518",{"2":{"236":1}}],["005147196",{"2":{"236":1}}],["005167301",{"2":{"236":1}}],["005460986",{"2":{"236":1}}],["005422304",{"2":{"236":1}}],["005454330",{"2":{"236":1}}],["005476930",{"2":{"236":1}}],["005641153",{"2":{"236":1}}],["00564f",{"2":{"77":1}}],["005623828",{"2":{"236":1}}],["005613507",{"2":{"236":1}}],["005612177",{"2":{"236":1}}],["005995554",{"2":{"236":1}}],["005907979",{"2":{"236":1}}],["005927891",{"2":{"236":1}}],["005325366",{"2":{"236":1}}],["005398072",{"2":{"236":1}}],["0053856913",{"2":{"77":1}}],["005276884",{"2":{"236":1}}],["005288779",{"2":{"236":1}}],["005280917",{"2":{"236":1}}],["005226072",{"2":{"236":1}}],["005251979",{"2":{"236":1}}],["005576532",{"2":{"236":1}}],["005502027",{"2":{"236":1}}],["005502339",{"2":{"236":1}}],["005568597",{"2":{"236":1}}],["005f0",{"2":{"236":1}}],["005",{"2":{"202":1}}],["0050086139984667e",{"2":{"258":1}}],["005000000000000009",{"2":{"63":1}}],["005085900",{"2":{"236":1}}],["005051813",{"2":{"236":1}}],["005034053",{"2":{"236":1}}],["005014097",{"2":{"77":1}}],["006367408",{"2":{"236":1}}],["006347454",{"2":{"236":1}}],["006822325",{"2":{"236":1}}],["006823588",{"2":{"236":1}}],["006853437",{"2":{"236":1}}],["006440687544458e",{"2":{"258":1}}],["006449282",{"2":{"236":1}}],["006485965",{"2":{"236":1}}],["006431336",{"2":{"236":1}}],["006432486",{"2":{"77":1}}],["006459999",{"2":{"236":1}}],["006491158",{"2":{"236":1}}],["006239149",{"2":{"236":1}}],["006252684",{"2":{"236":1}}],["006212323",{"2":{"236":1}}],["006269322",{"2":{"236":1}}],["006280850",{"2":{"236":1}}],["006288857",{"2":{"106":1}}],["0062888456",{"2":{"106":1}}],["006679880",{"2":{"236":1}}],["0066712513",{"2":{"179":1}}],["006058069",{"2":{"236":1}}],["006001258",{"2":{"236":1}}],["006012429",{"2":{"236":1}}],["006018532",{"2":{"236":1}}],["006085245",{"2":{"236":1}}],["006999766",{"2":{"236":1}}],["0069942693",{"2":{"106":1}}],["00691",{"2":{"187":1}}],["006506131",{"2":{"236":1}}],["00653",{"2":{"187":1}}],["0065859724",{"2":{"77":1}}],["006178515",{"2":{"236":1}}],["006166900",{"2":{"236":1}}],["0061484f",{"2":{"256":1}}],["006147437",{"2":{"236":1}}],["006143386",{"2":{"236":1}}],["006185614",{"2":{"236":1}}],["006188713",{"2":{"236":1}}],["006184267",{"2":{"77":1}}],["006151655",{"2":{"236":1}}],["006136029",{"2":{"236":1}}],["0061099795",{"2":{"77":1}}],["006799675",{"2":{"236":1}}],["006721330",{"2":{"236":1}}],["00672",{"2":{"187":1}}],["006736353",{"2":{"77":1}}],["0067729075",{"2":{"77":1}}],["004910624",{"2":{"236":1}}],["004987885",{"2":{"236":1}}],["004581080",{"2":{"236":1}}],["004513775",{"2":{"236":1}}],["004619529",{"2":{"236":1}}],["004690929",{"2":{"236":1}}],["004643182",{"2":{"236":1}}],["004668837",{"2":{"236":1}}],["004604227",{"2":{"236":1}}],["004791833",{"2":{"236":1}}],["004799493",{"2":{"236":1}}],["004776356",{"2":{"236":1}}],["004326439",{"2":{"236":1}}],["004315960",{"2":{"236":1}}],["004308904",{"2":{"236":1}}],["004206825",{"2":{"236":1}}],["004206762",{"2":{"236":1}}],["004265270",{"2":{"236":1}}],["004265617",{"2":{"236":1}}],["004287243",{"2":{"106":2}}],["004431641",{"2":{"236":1}}],["004497696",{"2":{"236":1}}],["0044189203",{"2":{"77":1}}],["004814619",{"2":{"236":1}}],["004839203",{"2":{"236":1}}],["004839269",{"2":{"236":1}}],["004838886",{"2":{"236":1}}],["004847473",{"2":{"236":1}}],["00488f",{"2":{"77":1}}],["004065084",{"2":{"236":1}}],["004062099",{"2":{"236":1}}],["004083481",{"2":{"236":1}}],["004099627",{"2":{"236":1}}],["004045234",{"2":{"236":1}}],["004045264",{"2":{"236":1}}],["004047669",{"2":{"77":1}}],["00407581",{"2":{"179":1}}],["004123454",{"2":{"236":1}}],["004102683",{"2":{"236":1}}],["0041674743",{"2":{"106":1}}],["0041674753",{"2":{"106":1}}],["004169049",{"2":{"77":1}}],["004144425",{"2":{"77":1}}],["001524955",{"2":{"236":1}}],["001588614",{"2":{"236":1}}],["001589268",{"2":{"236":1}}],["001558672",{"2":{"236":1}}],["001557242",{"2":{"236":1}}],["001599119",{"2":{"236":1}}],["001596168",{"2":{"236":1}}],["001518356",{"2":{"236":1}}],["001500407",{"2":{"236":1}}],["001562893",{"2":{"236":1}}],["001562393",{"2":{"236":1}}],["001568149",{"2":{"236":1}}],["001571323",{"2":{"236":1}}],["0015313211",{"2":{"77":1}}],["001678471",{"2":{"236":1}}],["001679290",{"2":{"236":1}}],["001626283",{"2":{"236":1}}],["001621564",{"2":{"236":1}}],["001654499",{"2":{"236":1}}],["001652764",{"2":{"236":1}}],["001658047",{"2":{"236":1}}],["001656469",{"2":{"236":1}}],["001651582",{"2":{"236":1}}],["001686519",{"2":{"236":1}}],["001683053",{"2":{"236":1}}],["001680354",{"2":{"236":1}}],["001687118",{"2":{"236":1}}],["001698582",{"2":{"236":1}}],["001609863",{"2":{"236":1}}],["001665982",{"2":{"236":1}}],["001385620",{"2":{"236":1}}],["001385545",{"2":{"236":1}}],["001363050",{"2":{"236":1}}],["001318014",{"2":{"236":1}}],["001311332",{"2":{"236":1}}],["001373227",{"2":{"236":1}}],["001372868",{"2":{"236":1}}],["001379599",{"2":{"236":1}}],["001390965",{"2":{"236":1}}],["001391491",{"2":{"236":1}}],["00139126",{"2":{"77":1}}],["001350533",{"2":{"236":1}}],["001350272",{"2":{"236":1}}],["001353682",{"2":{"236":1}}],["001322284",{"2":{"236":1}}],["001339099",{"2":{"236":1}}],["001330249",{"2":{"236":1}}],["001",{"2":{"202":1}}],["0010350421",{"2":{"241":1}}],["001034181",{"2":{"236":1}}],["001049295",{"2":{"236":1}}],["001028772",{"2":{"236":1}}],["001086145",{"2":{"236":1}}],["001052209",{"2":{"236":1}}],["001058186",{"2":{"236":1}}],["001058255",{"2":{"236":1}}],["001095256",{"2":{"236":1}}],["001092387",{"2":{"236":1}}],["001015767",{"2":{"236":1}}],["001014665",{"2":{"236":1}}],["001019756",{"2":{"236":1}}],["00101147",{"2":{"179":1}}],["001069946",{"2":{"236":1}}],["001069190",{"2":{"236":1}}],["001063806",{"2":{"236":1}}],["001060150",{"2":{"236":1}}],["00106574",{"2":{"77":1}}],["001073368",{"2":{"236":1}}],["001009776",{"2":{"236":1}}],["001f0",{"2":{"78":1,"107":1,"217":1,"228":1}}],["0012",{"2":{"249":1}}],["001238901",{"2":{"236":1}}],["001233365",{"2":{"236":1}}],["001213058",{"2":{"236":1}}],["001213073",{"2":{"236":1}}],["001206274",{"2":{"236":1}}],["001204579",{"2":{"236":1}}],["001263972",{"2":{"236":1}}],["001266953",{"2":{"236":1}}],["00126273",{"2":{"77":1}}],["001278718",{"2":{"236":1}}],["001278843",{"2":{"236":1}}],["001272737",{"2":{"236":1}}],["001279285",{"2":{"236":1}}],["001256422",{"2":{"236":1}}],["001256183",{"2":{"236":1}}],["001255589",{"2":{"236":1}}],["0012237771997909e",{"2":{"258":1}}],["001222603",{"2":{"236":1}}],["001229342",{"2":{"236":1}}],["001229187",{"2":{"236":1}}],["001229975",{"2":{"236":1}}],["001243522",{"2":{"236":1}}],["001291234",{"2":{"236":1}}],["001292446",{"2":{"236":1}}],["001292421",{"2":{"236":1}}],["001297660",{"2":{"236":1}}],["001293960",{"2":{"236":1}}],["001286207",{"2":{"236":1}}],["001282274",{"2":{"236":1}}],["001287611",{"2":{"236":1}}],["001280978",{"2":{"236":1}}],["00128162",{"2":{"77":1}}],["0012843215",{"2":{"77":1}}],["001812232",{"2":{"236":1}}],["001814278",{"2":{"236":1}}],["001842899",{"2":{"236":1}}],["001848977",{"2":{"236":1}}],["001820831",{"2":{"236":1}}],["001821541",{"2":{"236":1}}],["001828601",{"2":{"236":1}}],["001869006",{"2":{"236":1}}],["001866840",{"2":{"236":1}}],["001868680",{"2":{"236":1}}],["001853661",{"2":{"236":1}}],["0018510937",{"2":{"77":1}}],["001895187",{"2":{"236":1}}],["001893832",{"2":{"236":1}}],["001888746",{"2":{"236":1}}],["001882976",{"2":{"236":1}}],["001882480",{"2":{"236":1}}],["001885105",{"2":{"236":1}}],["001871673",{"2":{"236":1}}],["001872860",{"2":{"236":1}}],["001872000",{"2":{"236":1}}],["00187748",{"2":{"77":1}}],["0018058",{"2":{"77":1}}],["001497735",{"2":{"236":1}}],["001440010",{"2":{"236":1}}],["001460486",{"2":{"236":1}}],["001467459",{"2":{"236":1}}],["001467187",{"2":{"236":1}}],["001465238",{"2":{"236":1}}],["001435974",{"2":{"236":1}}],["001431392",{"2":{"236":1}}],["001436205",{"2":{"236":1}}],["001436674",{"2":{"236":1}}],["001432847",{"2":{"236":1}}],["0014795f",{"2":{"256":1}}],["001471427",{"2":{"236":1}}],["001473510",{"2":{"236":1}}],["001476229",{"2":{"236":1}}],["001407278",{"2":{"236":1}}],["00140763051",{"2":{"78":1}}],["001401214",{"2":{"236":1}}],["001400374",{"2":{"236":1}}],["001453047",{"2":{"236":1}}],["001458879",{"2":{"236":1}}],["00145586",{"2":{"77":1}}],["001415687",{"2":{"236":1}}],["001415296",{"2":{"236":1}}],["001411696",{"2":{"236":1}}],["0014175134",{"2":{"77":1}}],["001142784",{"2":{"236":1}}],["001149042",{"2":{"236":1}}],["001104411",{"2":{"236":1}}],["001176956",{"2":{"236":1}}],["001172049",{"2":{"236":1}}],["00117228",{"2":{"77":1}}],["001128200",{"2":{"236":1}}],["001126713",{"2":{"236":1}}],["001126977",{"2":{"236":1}}],["001124672",{"2":{"236":1}}],["001180587",{"2":{"236":1}}],["001184456",{"2":{"236":1}}],["001199473",{"2":{"236":1}}],["001192615",{"2":{"236":1}}],["001193460",{"2":{"236":1}}],["001151307",{"2":{"236":1}}],["001156326",{"2":{"236":1}}],["00115296",{"2":{"77":1}}],["001166611",{"2":{"236":1}}],["001132011",{"2":{"236":1}}],["001741661",{"2":{"236":1}}],["001768059",{"2":{"236":1}}],["001760620",{"2":{"236":1}}],["001760687",{"2":{"236":1}}],["0017147640174854e",{"2":{"258":1}}],["001718084",{"2":{"236":1}}],["001715263",{"2":{"236":1}}],["001757617",{"2":{"236":1}}],["00175152",{"2":{"77":1}}],["001738493",{"2":{"236":1}}],["001730394",{"2":{"236":1}}],["0017369908",{"2":{"77":1}}],["001706076",{"2":{"236":1}}],["001702445",{"2":{"236":1}}],["001704919",{"2":{"236":1}}],["001701293",{"2":{"236":1}}],["001703513",{"2":{"236":1}}],["00170922",{"2":{"77":1}}],["001785598",{"2":{"236":1}}],["001797515",{"2":{"236":1}}],["0017900232",{"2":{"77":1}}],["001965609",{"2":{"236":1}}],["001955235",{"2":{"236":1}}],["0019504696",{"2":{"77":1}}],["001948264",{"2":{"236":1}}],["001945538",{"2":{"236":1}}],["001921760",{"2":{"236":1}}],["001928137",{"2":{"236":1}}],["001929016",{"2":{"236":1}}],["0019262917",{"2":{"77":1}}],["001975889",{"2":{"236":1}}],["001976204",{"2":{"236":1}}],["001973851",{"2":{"236":1}}],["001973281",{"2":{"77":1}}],["001970439",{"2":{"236":1}}],["001939833",{"2":{"236":1}}],["001939239",{"2":{"236":1}}],["001933446",{"2":{"236":1}}],["001933840",{"2":{"236":1}}],["001909967",{"2":{"236":1}}],["001985326",{"2":{"236":1}}],["001985265",{"2":{"236":1}}],["001983398",{"2":{"236":1}}],["00198415",{"2":{"179":1}}],["00198767",{"2":{"77":1}}],["0019153744",{"2":{"77":1}}],["002562924",{"2":{"236":1}}],["002570425",{"2":{"236":1}}],["002595518",{"2":{"236":1}}],["002537781",{"2":{"236":1}}],["002533023",{"2":{"236":1}}],["002517481",{"2":{"236":1}}],["002514324",{"2":{"236":1}}],["002514284",{"2":{"236":1}}],["002516578",{"2":{"236":1}}],["002502190",{"2":{"236":1}}],["002971325",{"2":{"236":1}}],["002906828",{"2":{"236":1}}],["002901576",{"2":{"236":1}}],["002910380",{"2":{"236":1}}],["002995419",{"2":{"236":1}}],["002888216",{"2":{"236":1}}],["002885855",{"2":{"236":1}}],["002853266",{"2":{"236":1}}],["002875937",{"2":{"236":1}}],["002827825",{"2":{"236":1}}],["002826607",{"2":{"236":1}}],["0028297466",{"2":{"106":1}}],["0028297475",{"2":{"106":1}}],["002695851",{"2":{"236":1}}],["002697282",{"2":{"236":1}}],["002625272",{"2":{"236":1}}],["0026271285",{"2":{"110":1}}],["002606902",{"2":{"236":1}}],["002229830",{"2":{"236":1}}],["002220259",{"2":{"236":1}}],["002289784",{"2":{"236":1}}],["0022497",{"2":{"241":1}}],["002249273",{"2":{"236":1}}],["002242853",{"2":{"236":1}}],["002276014",{"2":{"236":1}}],["002234953",{"2":{"236":1}}],["002235665",{"2":{"236":1}}],["0022308934",{"2":{"77":1}}],["002208042",{"2":{"236":1}}],["002357919",{"2":{"236":1}}],["002372291",{"2":{"236":1}}],["002375467",{"2":{"236":1}}],["002387334",{"2":{"236":1}}],["002393598",{"2":{"236":1}}],["002316626",{"2":{"236":1}}],["002310942",{"2":{"236":1}}],["002343330",{"2":{"236":1}}],["002349454",{"2":{"236":1}}],["002334290",{"2":{"236":1}}],["002361344",{"2":{"236":1}}],["002366644",{"2":{"236":1}}],["0023s\\ttraining",{"2":{"217":1}}],["002083311",{"2":{"236":1}}],["002075948",{"2":{"236":1}}],["002079992",{"2":{"236":1}}],["002064294",{"2":{"236":1}}],["002061526",{"2":{"236":1}}],["002025729",{"2":{"236":1}}],["002027335",{"2":{"236":1}}],["002009441",{"2":{"236":1}}],["002004704",{"2":{"236":1}}],["002058655",{"2":{"236":1}}],["002055192",{"2":{"236":1}}],["0020579377",{"2":{"106":1}}],["002057937",{"2":{"106":1}}],["00209504",{"2":{"77":1}}],["002704483",{"2":{"236":1}}],["00270753",{"2":{"77":1}}],["002716297",{"2":{"236":1}}],["002771435",{"2":{"236":1}}],["002757893",{"2":{"236":1}}],["002758041",{"2":{"236":1}}],["002723655",{"2":{"236":1}}],["002738171",{"2":{"236":1}}],["0027386106",{"2":{"77":1}}],["002797e",{"2":{"202":1}}],["0027955552",{"2":{"77":1}}],["002161729",{"2":{"236":1}}],["002160525",{"2":{"236":1}}],["002166245",{"2":{"236":1}}],["002133374",{"2":{"236":1}}],["002157764",{"2":{"236":1}}],["002191632",{"2":{"236":1}}],["002193680",{"2":{"236":1}}],["00219681",{"2":{"77":1}}],["002185026",{"2":{"236":1}}],["002187656",{"2":{"236":1}}],["00214946",{"2":{"77":1}}],["0021203319",{"2":{"77":1}}],["002106787",{"2":{"77":1}}],["002414002",{"2":{"236":1}}],["002425219",{"2":{"236":1}}],["002423544",{"2":{"236":1}}],["002475326",{"2":{"236":1}}],["002436990",{"2":{"236":1}}],["002438489",{"2":{"236":1}}],["0024323382",{"2":{"77":1}}],["0024971329",{"2":{"77":1}}],["003309689947565e",{"2":{"258":1}}],["003345200",{"2":{"236":1}}],["003361913",{"2":{"236":1}}],["003964073079977e",{"2":{"258":1}}],["003964f",{"2":{"256":1}}],["003918901",{"2":{"236":1}}],["003986657",{"2":{"236":1}}],["003927392",{"2":{"236":1}}],["00390526722",{"2":{"78":1}}],["003857500",{"2":{"236":1}}],["003873884",{"2":{"236":1}}],["003878577",{"2":{"236":1}}],["0038745",{"2":{"77":1}}],["003810477",{"2":{"236":1}}],["003807114",{"2":{"236":1}}],["003880639",{"2":{"236":1}}],["0032486f",{"2":{"256":1}}],["003226456",{"2":{"236":1}}],["003218761",{"2":{"236":1}}],["003282005",{"2":{"236":1}}],["003250526",{"2":{"236":1}}],["003200589",{"2":{"236":1}}],["003109351",{"2":{"236":1}}],["003105919",{"2":{"236":1}}],["003145006",{"2":{"236":1}}],["003114783",{"2":{"236":1}}],["003181246",{"2":{"236":1}}],["003124564",{"2":{"236":1}}],["003192307",{"2":{"236":1}}],["0031376407",{"2":{"77":1}}],["003527584",{"2":{"236":1}}],["003555858",{"2":{"236":1}}],["003588729",{"2":{"236":1}}],["00351666",{"2":{"77":1}}],["003618096",{"2":{"236":1}}],["003635545",{"2":{"236":1}}],["003681621",{"2":{"236":1}}],["0036810287",{"2":{"106":1}}],["0036810301",{"2":{"106":1}}],["003646539",{"2":{"236":1}}],["00364828",{"2":{"77":1}}],["003666145",{"2":{"236":1}}],["003025806",{"2":{"236":1}}],["003027094",{"2":{"77":1}}],["003097285",{"2":{"236":1}}],["003044792",{"2":{"236":1}}],["003036225",{"2":{"236":1}}],["003003312",{"2":{"236":1}}],["003051734",{"2":{"236":1}}],["003050850",{"2":{"236":1}}],["003089970",{"2":{"236":1}}],["003071634",{"2":{"236":1}}],["00306169",{"2":{"77":1}}],["0034101289",{"2":{"77":1}}],["003796777",{"2":{"236":1}}],["0037725805",{"2":{"77":1}}],["003750000000000005",{"2":{"63":1}}],["0008188282810970212",{"2":{"258":1}}],["00081659508330541",{"2":{"258":1}}],["000845213",{"2":{"236":1}}],["000848341",{"2":{"236":1}}],["000833410",{"2":{"236":1}}],["000837591",{"2":{"236":1}}],["000830096",{"2":{"236":1}}],["0008090121770929926",{"2":{"258":1}}],["000809125",{"2":{"236":1}}],["000800843",{"2":{"236":1}}],["000870643",{"2":{"236":1}}],["000871024327",{"2":{"78":1}}],["000895934",{"2":{"236":1}}],["000894471",{"2":{"236":1}}],["000899577",{"2":{"236":1}}],["000829655",{"2":{"236":1}}],["000824967",{"2":{"236":1}}],["000827484",{"2":{"236":1}}],["000821749",{"2":{"236":1}}],["000820911",{"2":{"236":1}}],["000825630",{"2":{"236":1}}],["000852256",{"2":{"236":1}}],["000854950",{"2":{"236":1}}],["000071671",{"2":{"236":1}}],["00000",{"2":{"187":100,"217":18,"219":3}}],["000",{"2":{"229":2,"238":2,"250":1}}],["00024344589819855688",{"2":{"258":1}}],["00024238950764526578",{"2":{"258":1}}],["0002445260700302267",{"2":{"258":1}}],["00024452427",{"2":{"256":1}}],["00024000306265029188",{"2":{"258":1}}],["00024000471",{"2":{"256":1}}],["00024938021991749594",{"2":{"258":1}}],["00024938013",{"2":{"256":1}}],["0002386690866586987",{"2":{"258":1}}],["0002326689146992544",{"2":{"258":1}}],["00023266925",{"2":{"256":1}}],["00023529045644837644",{"2":{"258":1}}],["0002352912",{"2":{"256":1}}],["0002375480325360175",{"2":{"258":1}}],["00023754647",{"2":{"256":1}}],["00023443684296223387",{"2":{"258":1}}],["00023443296",{"2":{"256":1}}],["000239130808",{"2":{"78":1}}],["00022644538420962983",{"2":{"258":1}}],["00022637474534430537",{"2":{"258":1}}],["00022637536",{"2":{"256":1}}],["00022938817682048194",{"2":{"258":1}}],["00022938936",{"2":{"256":1}}],["00022549747933323905",{"2":{"258":1}}],["0002254977",{"2":{"256":1}}],["00022372100621379767",{"2":{"258":1}}],["00022372045",{"2":{"256":1}}],["00022457709849473415",{"2":{"258":1}}],["00022457454",{"2":{"256":1}}],["00022448884948184845",{"2":{"258":1}}],["00022448925",{"2":{"256":1}}],["00022290339151415838",{"2":{"258":1}}],["00022290568",{"2":{"256":1}}],["0002224535393759746",{"2":{"258":1}}],["00022245743",{"2":{"256":1}}],["0002067543200442797",{"2":{"258":1}}],["00020295179048113686",{"2":{"258":1}}],["0002029514",{"2":{"256":1}}],["0002093120931842273",{"2":{"258":1}}],["00020931265",{"2":{"256":1}}],["00020594343826402825",{"2":{"258":1}}],["00020594428",{"2":{"256":1}}],["0002056306433941474",{"2":{"258":1}}],["00020563042",{"2":{"256":1}}],["00020731906530684744",{"2":{"258":1}}],["00020731823",{"2":{"256":1}}],["0002077890839374628",{"2":{"258":1}}],["00020778744",{"2":{"256":1}}],["00020768779579119",{"2":{"258":1}}],["00020769515921099894",{"2":{"258":1}}],["0002076928",{"2":{"256":1}}],["00020763810451676645",{"2":{"258":1}}],["00020764141",{"2":{"256":1}}],["0002083845939179537",{"2":{"258":1}}],["00020838533",{"2":{"256":1}}],["00020849705",{"2":{"146":1}}],["00020451560202494075",{"2":{"258":1}}],["00020451599",{"2":{"256":1}}],["0002042418339070592",{"2":{"258":1}}],["00020424122",{"2":{"256":1}}],["00020058507343278912",{"2":{"258":1}}],["0002005834",{"2":{"256":1}}],["0002003185273243402",{"2":{"258":1}}],["00020031813",{"2":{"256":1}}],["00021962347088651664",{"2":{"258":1}}],["00021320531260383206",{"2":{"258":1}}],["00021320413",{"2":{"256":1}}],["00021536490777118708",{"2":{"258":1}}],["00021536481",{"2":{"256":1}}],["00021805007",{"2":{"256":1}}],["00021246919855926692",{"2":{"258":1}}],["00021247026",{"2":{"256":1}}],["00021277014738130048",{"2":{"258":1}}],["00021276702",{"2":{"256":1}}],["00021121956770491288",{"2":{"258":1}}],["00021121927",{"2":{"256":1}}],["000211085549540022",{"2":{"258":1}}],["00021108559",{"2":{"256":1}}],["0002110207715302368",{"2":{"258":1}}],["00021102077",{"2":{"256":1}}],["00021643845326449829",{"2":{"258":1}}],["00021643919",{"2":{"256":1}}],["00021610962220441316",{"2":{"258":1}}],["00021611001",{"2":{"256":1}}],["000214067404",{"2":{"78":1}}],["00029064249278658256",{"2":{"258":1}}],["00029299364541612773",{"2":{"258":1}}],["00029299365",{"2":{"256":1}}],["00029249687",{"2":{"256":1}}],["00029203",{"2":{"179":1}}],["000298400",{"2":{"236":1}}],["00028225686738572416",{"2":{"258":1}}],["00028123813089167386",{"2":{"258":1}}],["000281235",{"2":{"256":1}}],["0002861855390168684",{"2":{"258":1}}],["00028618306",{"2":{"256":1}}],["0002875950521490153",{"2":{"258":1}}],["0002875935",{"2":{"256":1}}],["000287049",{"2":{"236":1}}],["00028501027004306206",{"2":{"258":1}}],["00028501067",{"2":{"256":1}}],["000285411",{"2":{"236":1}}],["00025862815255747214",{"2":{"258":1}}],["000256707887242131",{"2":{"258":1}}],["00025670705",{"2":{"256":1}}],["0002575727303903201",{"2":{"258":1}}],["00025757347",{"2":{"256":1}}],["000255029",{"2":{"236":1}}],["00025326014",{"2":{"146":1}}],["00026801874025249095",{"2":{"258":1}}],["00026887345040255226",{"2":{"258":1}}],["00026887178",{"2":{"256":1}}],["000262824",{"2":{"236":1}}],["000269293",{"2":{"236":1}}],["00027279834140029273",{"2":{"258":1}}],["000272142",{"2":{"236":1}}],["00027573378158736456",{"2":{"258":1}}],["000276926",{"2":{"236":1}}],["000271947",{"2":{"236":1}}],["00042179008933688464",{"2":{"258":1}}],["000451327",{"2":{"236":1}}],["000454922556",{"2":{"78":1}}],["000409009",{"2":{"236":1}}],["000403142",{"2":{"236":1}}],["000492043",{"2":{"236":1}}],["000499937",{"2":{"236":1}}],["000470345",{"2":{"236":1}}],["000481077",{"2":{"236":1}}],["000481490",{"2":{"236":1}}],["000432294",{"2":{"236":1}}],["000438132",{"2":{"77":1}}],["000412366",{"2":{"236":1}}],["000464835",{"2":{"236":1}}],["000461346",{"2":{"236":1}}],["00046014786",{"2":{"145":1}}],["000441076729",{"2":{"78":1}}],["0006384380101355744",{"2":{"258":1}}],["000638054666204854",{"2":{"258":1}}],["0006780007670647525",{"2":{"258":1}}],["000672969",{"2":{"236":1}}],["000620521836429977",{"2":{"258":1}}],["000620552",{"2":{"236":1}}],["000620883",{"2":{"236":1}}],["0006685830389952254",{"2":{"258":1}}],["000660004",{"2":{"236":1}}],["000667028",{"2":{"77":1}}],["0006129880701127028",{"2":{"258":1}}],["000612726",{"2":{"77":1}}],["0006106590559730796",{"2":{"258":1}}],["0006149025421776231",{"2":{"258":1}}],["000614744",{"2":{"236":1}}],["000619964",{"2":{"236":1}}],["000616715",{"2":{"236":1}}],["0006054688376952504",{"2":{"258":1}}],["000600878",{"2":{"236":1}}],["000609521",{"2":{"77":1}}],["0006478916014476879",{"2":{"258":1}}],["000645818",{"2":{"236":1}}],["000648862",{"2":{"236":1}}],["000640416",{"2":{"236":1}}],["000641468",{"2":{"236":1}}],["0006549193992018953",{"2":{"258":1}}],["0006542395072238465",{"2":{"258":1}}],["0006594741368423112",{"2":{"258":1}}],["000653678",{"2":{"236":1}}],["000658372",{"2":{"236":1}}],["0006903661698708907",{"2":{"258":1}}],["0006994921614618403",{"2":{"257":1}}],["000699265",{"2":{"77":1}}],["000691837",{"2":{"236":1}}],["000697236",{"2":{"236":1}}],["000685165866837486",{"2":{"258":1}}],["0006822095209065945",{"2":{"258":1}}],["000688716987191034",{"2":{"258":1}}],["0006894606397778792",{"2":{"258":1}}],["000689341",{"2":{"236":1}}],["000689046",{"2":{"77":1}}],["00068473816",{"2":{"145":1}}],["00035529054072876593",{"2":{"258":1}}],["0003376355658037225",{"2":{"258":1}}],["00031691102393929387",{"2":{"258":1}}],["0003152906837482448",{"2":{"258":1}}],["00031529382",{"2":{"256":1}}],["0003211215798211186",{"2":{"258":1}}],["0003245993159796126",{"2":{"258":1}}],["000325338",{"2":{"236":1}}],["000365198",{"2":{"236":1}}],["00036629336",{"2":{"78":1}}],["000302386",{"2":{"236":1}}],["000346806",{"2":{"236":1}}],["000374875",{"2":{"77":1}}],["0003904691802563027",{"2":{"258":1}}],["00039923915367428507",{"2":{"258":1}}],["000394875",{"2":{"236":1}}],["000395792",{"2":{"236":1}}],["000391871",{"2":{"236":1}}],["000391076",{"2":{"77":1}}],["0003920444",{"2":{"77":1}}],["000565240734261215",{"2":{"258":1}}],["000563284",{"2":{"77":1}}],["000543235",{"2":{"236":1}}],["000542785",{"2":{"236":1}}],["000555111",{"2":{"236":1}}],["000554309",{"2":{"236":1}}],["000554629",{"2":{"236":1}}],["000554123",{"2":{"236":1}}],["000554918",{"2":{"77":1}}],["000508584",{"2":{"236":1}}],["000500864",{"2":{"236":1}}],["000576096",{"2":{"236":1}}],["000578991",{"2":{"236":1}}],["000574894",{"2":{"77":1}}],["000532408",{"2":{"236":1}}],["000537340",{"2":{"236":1}}],["000531116",{"2":{"236":1}}],["00053332",{"2":{"179":1}}],["000510654",{"2":{"236":1}}],["000510198",{"2":{"77":1}}],["000595867",{"2":{"236":1}}],["000597971",{"2":{"236":1}}],["000597963",{"2":{"236":1}}],["000598925",{"2":{"236":1}}],["0005922087",{"2":{"106":1}}],["00059220614",{"2":{"106":1}}],["000525414",{"2":{"236":1}}],["000523516",{"2":{"236":1}}],["000528065",{"2":{"236":1}}],["000527887",{"2":{"236":1}}],["0005892286392886316",{"2":{"258":1}}],["000589236",{"2":{"236":1}}],["000583803",{"2":{"236":1}}],["000582542",{"2":{"236":1}}],["000586666",{"2":{"236":1}}],["000580705",{"2":{"77":1}}],["0005f0",{"2":{"236":1}}],["0007676011839588111",{"2":{"258":1}}],["000766442",{"2":{"236":1}}],["0007969439236190279",{"2":{"258":1}}],["000796935",{"2":{"236":1}}],["0007522449748653104",{"2":{"258":1}}],["0007500193367682273",{"2":{"258":1}}],["000757549",{"2":{"236":1}}],["000749967054858272",{"2":{"258":1}}],["000741202",{"2":{"236":1}}],["0007119907802648645",{"2":{"258":1}}],["000711415",{"2":{"236":1}}],["000715418",{"2":{"236":1}}],["000781609",{"2":{"236":1}}],["000789657",{"2":{"236":1}}],["000782973",{"2":{"236":1}}],["000728400",{"2":{"236":1}}],["000728280",{"2":{"236":1}}],["0007330832427160187",{"2":{"258":1}}],["0007318573278050192",{"2":{"258":1}}],["000731462",{"2":{"77":1}}],["000736817",{"2":{"236":1}}],["000702699",{"2":{"236":1}}],["0007797070160926858",{"2":{"258":1}}],["000779939",{"2":{"236":1}}],["0007788534505924469",{"2":{"258":1}}],["000776913",{"2":{"236":1}}],["00077649",{"2":{"77":1}}],["0007736237",{"2":{"179":1}}],["00077338",{"2":{"77":1}}],["0009757f",{"2":{"256":1}}],["000975115",{"2":{"236":1}}],["000979689",{"2":{"236":1}}],["000994589",{"2":{"236":1}}],["000997462",{"2":{"236":1}}],["000992391663909964",{"2":{"63":2}}],["000920903",{"2":{"236":1}}],["000985127",{"2":{"236":1}}],["000951834",{"2":{"236":1}}],["000957004",{"2":{"236":1}}],["000956681",{"2":{"236":1}}],["000935075",{"2":{"236":1}}],["000936727",{"2":{"77":1}}],["000946846",{"2":{"236":1}}],["000942026",{"2":{"77":1}}],["000966280",{"2":{"236":1}}],["000912286",{"2":{"236":1}}],["000917391",{"2":{"236":1}}],["000919413",{"2":{"236":1}}],["000910802",{"2":{"77":1}}],["0001752321965248829",{"2":{"258":1}}],["00017523114",{"2":{"256":1}}],["0001799537939011677",{"2":{"258":1}}],["00017994971",{"2":{"256":1}}],["00017095396861927688",{"2":{"258":1}}],["00017095437",{"2":{"256":1}}],["0001772373517577329",{"2":{"258":1}}],["00017723769",{"2":{"256":1}}],["00017798736817951032",{"2":{"258":1}}],["0001779905",{"2":{"256":1}}],["00017406216862597297",{"2":{"258":1}}],["00017406323",{"2":{"256":1}}],["00017457359528103169",{"2":{"258":1}}],["00017457195",{"2":{"256":1}}],["0001734228207430405",{"2":{"258":1}}],["00017342126",{"2":{"256":1}}],["0001736656521987529",{"2":{"258":1}}],["00017366491",{"2":{"256":1}}],["00017155234894268115",{"2":{"258":1}}],["0001715258825111625",{"2":{"258":1}}],["00017152919",{"2":{"256":1}}],["00017146696110282325",{"2":{"258":1}}],["00017146657",{"2":{"256":1}}],["00017197156813943473",{"2":{"258":1}}],["00017196768",{"2":{"256":1}}],["00017647810610872474",{"2":{"258":1}}],["0001764748",{"2":{"256":1}}],["00017649083434760035",{"2":{"258":1}}],["00017649106",{"2":{"256":1}}],["0001780792737459982",{"2":{"258":1}}],["00017808175",{"2":{"256":1}}],["00017834424",{"2":{"256":1}}],["00017859623023947177",{"2":{"258":1}}],["0001785962",{"2":{"256":1}}],["00018809201505049565",{"2":{"258":1}}],["00018809095",{"2":{"256":1}}],["00018289921795869367",{"2":{"258":1}}],["00018289514",{"2":{"256":1}}],["0001807862603881634",{"2":{"258":1}}],["00018078604",{"2":{"256":1}}],["00018936240653637186",{"2":{"258":1}}],["00018935993",{"2":{"256":1}}],["0001856869127639691",{"2":{"258":1}}],["00018568635",{"2":{"256":1}}],["00018457219015582057",{"2":{"258":1}}],["00018409355376616799",{"2":{"258":1}}],["00018409271",{"2":{"256":1}}],["00018497418274792557",{"2":{"258":1}}],["00018497492",{"2":{"256":1}}],["00018340926731325743",{"2":{"258":1}}],["00018340634",{"2":{"256":1}}],["00018310234185919255",{"2":{"258":1}}],["00018310527",{"2":{"256":1}}],["00018384945802312768",{"2":{"258":1}}],["00018384538",{"2":{"256":1}}],["0001397386634331515",{"2":{"258":1}}],["00013973896",{"2":{"256":1}}],["00013366244",{"2":{"256":1}}],["00013304138255015307",{"2":{"258":1}}],["00013302088783404502",{"2":{"258":1}}],["00013302402",{"2":{"256":1}}],["00013303825",{"2":{"256":1}}],["00013098878629520027",{"2":{"258":1}}],["00013098773",{"2":{"256":1}}],["00013054524148586346",{"2":{"258":1}}],["0001305444",{"2":{"256":1}}],["0001305295083282009",{"2":{"258":1}}],["00013052954",{"2":{"256":1}}],["0001354552542483205",{"2":{"258":1}}],["00013545212",{"2":{"256":1}}],["00013506328774604167",{"2":{"258":1}}],["00013506036",{"2":{"256":1}}],["0001369163767111125",{"2":{"258":1}}],["00013691728",{"2":{"256":1}}],["00013679834888453683",{"2":{"258":1}}],["00013679774",{"2":{"256":1}}],["00013815512385723008",{"2":{"258":1}}],["00013815345",{"2":{"256":1}}],["00013888742406782632",{"2":{"258":1}}],["00013888412",{"2":{"256":1}}],["0001321192202927102",{"2":{"258":1}}],["00013211838",{"2":{"256":1}}],["0001325855737942889",{"2":{"258":1}}],["00013258631",{"2":{"256":1}}],["00013262470123104556",{"2":{"258":1}}],["00013262503",{"2":{"256":1}}],["00013266518041143833",{"2":{"258":1}}],["00013266527",{"2":{"256":1}}],["0001348183158499198",{"2":{"258":1}}],["00013481667",{"2":{"256":1}}],["00013478324137200767",{"2":{"258":1}}],["00013478157",{"2":{"256":1}}],["0001341794085763753",{"2":{"258":1}}],["00013417867",{"2":{"256":1}}],["0001343889356710264",{"2":{"258":1}}],["00013439129",{"2":{"256":1}}],["0001372912513309467",{"2":{"258":1}}],["00013729051",{"2":{"256":1}}],["00013716311928972227",{"2":{"258":1}}],["00013716334",{"2":{"256":1}}],["00013781613220123864",{"2":{"258":1}}],["00013781365",{"2":{"256":1}}],["00013195338005123078",{"2":{"258":1}}],["00013195348",{"2":{"256":1}}],["00013142922",{"2":{"256":1}}],["00013125093676276848",{"2":{"258":1}}],["00013124838",{"2":{"256":1}}],["00013121",{"2":{"77":1}}],["00013104114797366984",{"2":{"258":1}}],["00013107348891889498",{"2":{"258":1}}],["0001310712",{"2":{"256":1}}],["0001310395",{"2":{"256":1}}],["000195794726838513",{"2":{"258":1}}],["00019579084",{"2":{"256":1}}],["000190839455330881",{"2":{"258":1}}],["00019083949",{"2":{"256":1}}],["00019341419813395872",{"2":{"258":1}}],["00019341416",{"2":{"256":1}}],["00019266559581060106",{"2":{"258":1}}],["00019266526",{"2":{"256":1}}],["0001920831369582128",{"2":{"258":1}}],["00019208078",{"2":{"256":1}}],["00019695894545278714",{"2":{"258":1}}],["00019663653905250167",{"2":{"258":1}}],["00019663687",{"2":{"256":1}}],["00019670696187890814",{"2":{"258":1}}],["00019670448",{"2":{"256":1}}],["00019929420041569243",{"2":{"258":1}}],["0001992939",{"2":{"256":1}}],["00019976532171501866",{"2":{"258":1}}],["00019976492",{"2":{"256":1}}],["00019136307791322918",{"2":{"258":1}}],["00019136128",{"2":{"256":1}}],["0001919215207530704",{"2":{"258":1}}],["0001919227",{"2":{"256":1}}],["00019782936787099414",{"2":{"258":1}}],["00019794775219478724",{"2":{"258":1}}],["00019794701",{"2":{"256":1}}],["00019718893657916247",{"2":{"258":1}}],["00019719187",{"2":{"256":1}}],["00019493563987032264",{"2":{"258":1}}],["00019493535",{"2":{"256":1}}],["0001946906153888681",{"2":{"258":1}}],["0001946904",{"2":{"256":1}}],["0001941201843012629",{"2":{"258":1}}],["00019412015",{"2":{"256":1}}],["0001989364128252271",{"2":{"258":1}}],["00019893328",{"2":{"256":1}}],["00019875751220131186",{"2":{"258":1}}],["0001987566",{"2":{"256":1}}],["0001984639426133598",{"2":{"258":1}}],["00019846226",{"2":{"256":1}}],["0001983177565875168",{"2":{"258":1}}],["00019831482",{"2":{"256":1}}],["00019833619418006238",{"2":{"258":1}}],["00019833563",{"2":{"256":1}}],["00015858274434813938",{"2":{"258":1}}],["00015829563809111817",{"2":{"258":1}}],["00015829744",{"2":{"256":1}}],["0001505381595568376",{"2":{"258":1}}],["00015053408",{"2":{"256":1}}],["0001575392605961893",{"2":{"258":1}}],["00015753518",{"2":{"256":1}}],["0001550530385393652",{"2":{"258":1}}],["00015502569352754357",{"2":{"258":1}}],["00015502863",{"2":{"256":1}}],["00015579326",{"2":{"256":1}}],["00015620606250447454",{"2":{"258":1}}],["00015620314",{"2":{"256":1}}],["00015603313",{"2":{"256":1}}],["0001561896855150079",{"2":{"258":1}}],["00015618675",{"2":{"256":1}}],["00015616323380740252",{"2":{"258":1}}],["00015616491",{"2":{"256":1}}],["00015202011155589635",{"2":{"258":1}}],["00015201893",{"2":{"256":1}}],["0001525932871226935",{"2":{"258":1}}],["0001525929",{"2":{"256":1}}],["0001527821803622454",{"2":{"258":1}}],["00015278214",{"2":{"256":1}}],["00015427663882797108",{"2":{"258":1}}],["00015427994",{"2":{"256":1}}],["00015459353811902082",{"2":{"258":1}}],["00015459415",{"2":{"256":1}}],["00015447046943757196",{"2":{"258":1}}],["00015447076",{"2":{"256":1}}],["00015441404606759293",{"2":{"258":1}}],["00015441813",{"2":{"256":1}}],["00015336449331575854",{"2":{"258":1}}],["00015336409",{"2":{"256":1}}],["00015310373445589243",{"2":{"258":1}}],["0001531043",{"2":{"256":1}}],["00015120252181272583",{"2":{"258":1}}],["00015120419",{"2":{"256":1}}],["00015189590495942777",{"2":{"258":1}}],["0001518926",{"2":{"256":1}}],["00015194269521963324",{"2":{"258":1}}],["00015194518",{"2":{"256":1}}],["00015113852",{"2":{"78":1}}],["00011217357574982597",{"2":{"258":1}}],["00011217324",{"2":{"256":1}}],["00011283250231843832",{"2":{"258":1}}],["00011283306",{"2":{"256":1}}],["0001167365383431639",{"2":{"258":1}}],["00011673683",{"2":{"256":1}}],["00011693449393821139",{"2":{"258":1}}],["00011693359",{"2":{"256":1}}],["00011654584779188501",{"2":{"258":1}}],["00011654196",{"2":{"256":1}}],["00011657592248007039",{"2":{"258":1}}],["00011657923",{"2":{"256":1}}],["00011736551213525923",{"2":{"258":1}}],["000117363714",{"2":{"256":1}}],["00011733264539424001",{"2":{"258":1}}],["00011733108",{"2":{"256":1}}],["00011716019001168527",{"2":{"258":1}}],["00011710617173955082",{"2":{"258":1}}],["00011710578",{"2":{"256":1}}],["00011715945",{"2":{"256":1}}],["00011032929004850627",{"2":{"258":1}}],["00011032749",{"2":{"256":1}}],["00011015533410263506",{"2":{"258":1}}],["00011015713",{"2":{"256":1}}],["00011062931656654735",{"2":{"258":1}}],["00011062858",{"2":{"256":1}}],["00011094090036560689",{"2":{"258":1}}],["0001109413",{"2":{"256":1}}],["00011476730041488437",{"2":{"258":1}}],["000114767594",{"2":{"256":1}}],["00011435171649936983",{"2":{"258":1}}],["00011435464",{"2":{"256":1}}],["00011420510048795209",{"2":{"258":1}}],["00011420436",{"2":{"256":1}}],["00011499716719120773",{"2":{"258":1}}],["00011499756",{"2":{"256":1}}],["00011454023361457142",{"2":{"258":1}}],["00011454203",{"2":{"256":1}}],["00011457079873682055",{"2":{"258":1}}],["00011457315",{"2":{"256":1}}],["00011469302367867727",{"2":{"258":1}}],["00011469073",{"2":{"256":1}}],["00011463925262351623",{"2":{"258":1}}],["00011463864",{"2":{"256":1}}],["00011970321898796433",{"2":{"258":1}}],["000119700926",{"2":{"256":1}}],["00011963930848304569",{"2":{"258":1}}],["00011963542",{"2":{"256":1}}],["0001190602489939384",{"2":{"258":1}}],["000119060285",{"2":{"256":1}}],["00011991298144013656",{"2":{"258":1}}],["000119915334",{"2":{"256":1}}],["00011992512",{"2":{"256":1}}],["00011532659461697469",{"2":{"258":1}}],["00011532569",{"2":{"256":1}}],["00011590276670945117",{"2":{"258":1}}],["00011590395",{"2":{"256":1}}],["0001155539617663254",{"2":{"258":1}}],["000115552284",{"2":{"256":1}}],["00011512875405347932",{"2":{"258":1}}],["00011512785",{"2":{"256":1}}],["00011510762695759204",{"2":{"258":1}}],["00011511093",{"2":{"256":1}}],["00011151542757454656",{"2":{"258":1}}],["00011151572",{"2":{"256":1}}],["00011144056712728296",{"2":{"258":1}}],["000111437635",{"2":{"256":1}}],["00011187006439073063",{"2":{"258":1}}],["000111844696",{"2":{"256":1}}],["00011186839",{"2":{"256":1}}],["00011177807",{"2":{"256":1}}],["0001116511728616103",{"2":{"258":1}}],["00011167815347230175",{"2":{"258":1}}],["00011167856",{"2":{"256":1}}],["00011168607698830286",{"2":{"258":1}}],["00011168598",{"2":{"256":1}}],["000111647285",{"2":{"256":1}}],["00011325016559036582",{"2":{"258":1}}],["00011324852",{"2":{"256":1}}],["00011310579835788056",{"2":{"258":1}}],["00011310589",{"2":{"256":1}}],["0001130596063834579",{"2":{"258":1}}],["000113057125",{"2":{"256":1}}],["0001137550249019633",{"2":{"258":1}}],["00011375335",{"2":{"256":1}}],["00011832902680259316",{"2":{"258":1}}],["00011833158",{"2":{"256":1}}],["00011875094816018567",{"2":{"258":1}}],["00011874976",{"2":{"256":1}}],["00011816811456800735",{"2":{"258":1}}],["000118168115",{"2":{"256":1}}],["00011848190264556968",{"2":{"258":1}}],["00011848084",{"2":{"256":1}}],["00011842121849019783",{"2":{"258":1}}],["000118420925",{"2":{"256":1}}],["0001184982250289855",{"2":{"258":1}}],["000118498785",{"2":{"256":1}}],["00014663913829266416",{"2":{"258":1}}],["00014663808",{"2":{"256":1}}],["00014609860633152372",{"2":{"258":1}}],["0001460987",{"2":{"256":1}}],["00014357596571532865",{"2":{"258":1}}],["00014357506",{"2":{"256":1}}],["00014333560567853233",{"2":{"258":1}}],["00014333393",{"2":{"256":1}}],["00014283905766451167",{"2":{"258":1}}],["00014276082954104035",{"2":{"258":1}}],["0001427624",{"2":{"256":1}}],["0001426893794134766",{"2":{"258":1}}],["00014268934",{"2":{"256":1}}],["00014533625759167532",{"2":{"258":1}}],["00014533396",{"2":{"256":1}}],["00014525730774234935",{"2":{"258":1}}],["00014525898",{"2":{"256":1}}],["0001458235136458298",{"2":{"258":1}}],["00014581943",{"2":{"256":1}}],["0001418282672925401",{"2":{"258":1}}],["0001418291",{"2":{"256":1}}],["00014198544987980186",{"2":{"258":1}}],["00014198378",{"2":{"256":1}}],["00014101640303173694",{"2":{"258":1}}],["0001410187",{"2":{"256":1}}],["0001410796248819983",{"2":{"258":1}}],["00014107962",{"2":{"256":1}}],["00014043715061353668",{"2":{"258":1}}],["00014043871",{"2":{"256":1}}],["00014010041564885663",{"2":{"258":1}}],["00014010115",{"2":{"256":1}}],["0001405512845746231",{"2":{"258":1}}],["00014055088",{"2":{"256":1}}],["00014058569472770083",{"2":{"258":1}}],["0001405853",{"2":{"256":1}}],["00014805938582098773",{"2":{"258":1}}],["0001480582",{"2":{"256":1}}],["00014851721317990345",{"2":{"258":1}}],["00014851333",{"2":{"256":1}}],["0001484312896143118",{"2":{"258":1}}],["0001484272",{"2":{"256":1}}],["00014863137780558722",{"2":{"258":1}}],["00014863386",{"2":{"256":1}}],["00014716079534976827",{"2":{"258":1}}],["00014715974",{"2":{"256":1}}],["0001473491735982382",{"2":{"258":1}}],["00014734587",{"2":{"256":1}}],["00014739854512804722",{"2":{"258":1}}],["00014702333",{"2":{"256":1}}],["00014740185",{"2":{"256":1}}],["0001492228158147876",{"2":{"258":1}}],["00014922114",{"2":{"256":1}}],["00014913768973205503",{"2":{"258":1}}],["00014913769",{"2":{"256":1}}],["00014970471966083986",{"2":{"258":1}}],["00014970472",{"2":{"256":1}}],["00012801898922615992",{"2":{"258":1}}],["00012801825",{"2":{"256":1}}],["00012843253359398103",{"2":{"258":1}}],["0001284341",{"2":{"256":1}}],["00012197666021787196",{"2":{"258":1}}],["000121977566",{"2":{"256":1}}],["00012144021351606074",{"2":{"258":1}}],["00012144251",{"2":{"256":1}}],["00012118766785344962",{"2":{"258":1}}],["000121188066",{"2":{"256":1}}],["00012020967654630444",{"2":{"258":1}}],["00012021001",{"2":{"256":1}}],["00012097718998973196",{"2":{"258":1}}],["00012097406",{"2":{"256":1}}],["00012066667100078717",{"2":{"258":1}}],["00012066419",{"2":{"256":1}}],["00012007268496072491",{"2":{"258":1}}],["00012007089",{"2":{"256":1}}],["00012005508877867815",{"2":{"258":1}}],["00012005584220608799",{"2":{"258":1}}],["000120056735",{"2":{"256":1}}],["00012005623",{"2":{"256":1}}],["0001269027465375235",{"2":{"258":1}}],["0001269051",{"2":{"256":1}}],["00012687753400117012",{"2":{"258":1}}],["0001268768",{"2":{"256":1}}],["00012681691859838386",{"2":{"258":1}}],["00012681524",{"2":{"256":1}}],["00012546289923291126",{"2":{"258":1}}],["00012546312",{"2":{"256":1}}],["000125267199157021",{"2":{"258":1}}],["0001252668",{"2":{"256":1}}],["0001295895628848218",{"2":{"258":1}}],["00012958882",{"2":{"256":1}}],["00012933625577521304",{"2":{"258":1}}],["00012933458",{"2":{"256":1}}],["0001296721467350404",{"2":{"258":1}}],["00012967248",{"2":{"256":1}}],["0001296541588627464",{"2":{"258":1}}],["00012965355",{"2":{"256":1}}],["00012902111012968843",{"2":{"258":1}}],["00012907708486514048",{"2":{"258":1}}],["00012907668",{"2":{"256":1}}],["00012901722",{"2":{"256":1}}],["00012979857587098995",{"2":{"258":1}}],["00012979835",{"2":{"256":1}}],["00012972956014925983",{"2":{"258":1}}],["00012972708",{"2":{"256":1}}],["00012407386727606787",{"2":{"258":1}}],["00012407296",{"2":{"256":1}}],["00012425462599443082",{"2":{"258":1}}],["00012425627",{"2":{"256":1}}],["00012459842342870078",{"2":{"258":1}}],["00012459549",{"2":{"256":1}}],["0001249789874672603",{"2":{"258":1}}],["00012497939",{"2":{"256":1}}],["00012487959205520594",{"2":{"258":1}}],["0001248795",{"2":{"256":1}}],["00012218504236097142",{"2":{"258":1}}],["00012218115",{"2":{"256":1}}],["0001222170070007688",{"2":{"258":1}}],["00012221691",{"2":{"256":1}}],["00012208599946444672",{"2":{"258":1}}],["000122086",{"2":{"256":1}}],["0001229504715719054",{"2":{"258":1}}],["0001229508",{"2":{"256":1}}],["00012296723257604767",{"2":{"258":1}}],["00012296431",{"2":{"256":1}}],["00012294222674462564",{"2":{"258":1}}],["00012293975",{"2":{"256":1}}],["0001277881692178221",{"2":{"258":1}}],["00012779109",{"2":{"256":1}}],["0001275287675673003",{"2":{"258":1}}],["00012752488",{"2":{"256":1}}],["00012795331713263893",{"2":{"258":1}}],["00012795332",{"2":{"256":1}}],["0001274553830541428",{"2":{"258":1}}],["00012745548",{"2":{"256":1}}],["00012745065733708302",{"2":{"258":1}}],["0001274501",{"2":{"256":1}}],["0001235665512811753",{"2":{"258":1}}],["00012356581",{"2":{"256":1}}],["0001232954042145835",{"2":{"258":1}}],["00012329305",{"2":{"256":1}}],["00012366636105791602",{"2":{"258":1}}],["00012366676",{"2":{"256":1}}],["00012370083100183914",{"2":{"258":1}}],["00012370043",{"2":{"256":1}}],["00012380761137260248",{"2":{"258":1}}],["00012380801",{"2":{"256":1}}],["00012384634",{"2":{"256":1}}],["0001233550239041024",{"2":{"258":1}}],["00012335247",{"2":{"256":1}}],["00012334863512049402",{"2":{"258":1}}],["00012334628",{"2":{"256":1}}],["00010346820850842252",{"2":{"258":1}}],["00010346927",{"2":{"256":1}}],["0001030861814281599",{"2":{"258":1}}],["00010302195888875587",{"2":{"258":1}}],["000103090264",{"2":{"256":1}}],["000103018654",{"2":{"256":1}}],["0001001544189665124",{"2":{"258":1}}],["000100155325",{"2":{"256":1}}],["00010043331538857763",{"2":{"258":1}}],["00010043496",{"2":{"256":1}}],["00010022054708018434",{"2":{"258":1}}],["000100216464",{"2":{"256":1}}],["00010060676",{"2":{"256":1}}],["00010917862081385706",{"2":{"258":1}}],["00010917549",{"2":{"256":1}}],["00010969113901973412",{"2":{"258":1}}],["00010968725",{"2":{"256":1}}],["00010992921628405631",{"2":{"258":1}}],["000109929126",{"2":{"256":1}}],["00010980521405130015",{"2":{"258":1}}],["000109804154",{"2":{"256":1}}],["00010983074287845628",{"2":{"258":1}}],["000109832414",{"2":{"256":1}}],["0001086862204166697",{"2":{"258":1}}],["000108683664",{"2":{"256":1}}],["00010843535945721786",{"2":{"258":1}}],["00010843301",{"2":{"256":1}}],["00010800564240403152",{"2":{"258":1}}],["00010800524",{"2":{"256":1}}],["00010800165732878346",{"2":{"258":1}}],["00010800126",{"2":{"256":1}}],["00010682118660925566",{"2":{"258":1}}],["00010682193",{"2":{"256":1}}],["00010628883969116538",{"2":{"258":1}}],["00010629292",{"2":{"256":1}}],["00010638573595719233",{"2":{"258":1}}],["00010638243",{"2":{"256":1}}],["00010605713367899186",{"2":{"258":1}}],["00010605673",{"2":{"256":1}}],["00010691171672865347",{"2":{"258":1}}],["00010691131",{"2":{"256":1}}],["00010649001498171168",{"2":{"258":1}}],["00010649421906202128",{"2":{"258":1}}],["0001064967",{"2":{"256":1}}],["00010648671",{"2":{"256":1}}],["00010485894642092461",{"2":{"258":1}}],["00010485861",{"2":{"256":1}}],["00010457436916671864",{"2":{"258":1}}],["000104574036",{"2":{"256":1}}],["00010429941560398894",{"2":{"258":1}}],["0001047987573741247",{"2":{"258":1}}],["00010479866",{"2":{"256":1}}],["00010432608823005876",{"2":{"258":1}}],["00010432296",{"2":{"256":1}}],["00010437864384371862",{"2":{"258":1}}],["000104376966",{"2":{"256":1}}],["00010431526163952301",{"2":{"258":1}}],["00010431158835610957",{"2":{"258":1}}],["00010431149",{"2":{"256":1}}],["000104312705",{"2":{"256":1}}],["00010430032",{"2":{"256":1}}],["00010707555600869373",{"2":{"258":1}}],["00010707723",{"2":{"256":1}}],["00010749304069541923",{"2":{"258":1}}],["00010749552",{"2":{"256":1}}],["0001071507456946745",{"2":{"258":1}}],["00010715892440416103",{"2":{"258":1}}],["00010714686",{"2":{"256":1}}],["0001071606",{"2":{"256":1}}],["00010132969269982383",{"2":{"258":1}}],["00010132676",{"2":{"256":1}}],["00010139692784269844",{"2":{"258":1}}],["00010192503642008395",{"2":{"258":1}}],["00010192549",{"2":{"256":1}}],["00010192842397594061",{"2":{"258":1}}],["00010192474",{"2":{"256":1}}],["00010117571488492138",{"2":{"258":1}}],["000101174875",{"2":{"256":1}}],["0001011474052819118",{"2":{"258":1}}],["00010114831",{"2":{"256":1}}],["00010122633169966698",{"2":{"258":1}}],["00010122694",{"2":{"256":1}}],["00010122059003134992",{"2":{"258":1}}],["00010122003",{"2":{"256":1}}],["00010144755259777768",{"2":{"258":1}}],["00010144765",{"2":{"256":1}}],["00010143332474399266",{"2":{"258":1}}],["000101433325",{"2":{"256":1}}],["000101400816",{"2":{"256":1}}],["00010580349557116111",{"2":{"258":1}}],["000105802435",{"2":{"256":1}}],["00010522481554910645",{"2":{"258":1}}],["00010522572",{"2":{"256":1}}],["000105296",{"2":{"77":1}}],["00010503480840224687",{"2":{"258":1}}],["000105032515",{"2":{"256":1}}],["00010509097202377471",{"2":{"258":1}}],["00010508767",{"2":{"256":1}}],["00010546855",{"2":{"179":1}}],["00010551",{"2":{"179":1}}],["00016060745007611773",{"2":{"258":1}}],["00016060712",{"2":{"256":1}}],["0001609052343607146",{"2":{"258":1}}],["00016090772",{"2":{"256":1}}],["00016907961370871138",{"2":{"258":1}}],["00016907782",{"2":{"256":1}}],["0001694114662636514",{"2":{"258":1}}],["0001694123",{"2":{"256":1}}],["0001648593724081675",{"2":{"258":1}}],["0001648625",{"2":{"256":1}}],["0001644379824160282",{"2":{"258":1}}],["00016443468",{"2":{"256":1}}],["00016205058459968245",{"2":{"258":1}}],["00016200238480021362",{"2":{"258":1}}],["0001624425913658241",{"2":{"258":1}}],["00016243967",{"2":{"256":1}}],["00016262584862181272",{"2":{"258":1}}],["00016262576",{"2":{"256":1}}],["00016371746818351842",{"2":{"258":1}}],["00016371673",{"2":{"256":1}}],["00016369696978876612",{"2":{"258":1}}],["00016369366",{"2":{"256":1}}],["000163183",{"2":{"236":1}}],["00016842915882558278",{"2":{"258":1}}],["0001684268",{"2":{"256":1}}],["00016835314062394827",{"2":{"258":1}}],["00016835274",{"2":{"256":1}}],["00016878",{"2":{"179":1}}],["00016661754547165592",{"2":{"258":1}}],["00016662048",{"2":{"256":1}}],["00016678708952002404",{"2":{"258":1}}],["00016678765",{"2":{"256":1}}],["00016691081691508447",{"2":{"258":1}}],["00016691082",{"2":{"256":1}}],["00016547872627458",{"2":{"258":1}}],["00016547991",{"2":{"256":1}}],["00016527810813790064",{"2":{"258":1}}],["0001652748",{"2":{"256":1}}],["00016596112564215394",{"2":{"258":1}}],["0001659628",{"2":{"256":1}}],["000165657",{"2":{"77":1}}],["0001",{"2":{"77":7,"78":1}}],["0001f0",{"2":{"77":1}}],["01s",{"2":{"196":1}}],["013727957168783e",{"2":{"258":1}}],["013724705",{"2":{"236":1}}],["01378",{"2":{"187":1}}],["013655247",{"2":{"236":1}}],["013669461",{"2":{"236":1}}],["013054269",{"2":{"236":1}}],["013011657",{"2":{"236":1}}],["01306",{"2":{"187":1}}],["013354947",{"2":{"236":1}}],["01335",{"2":{"187":1}}],["01332",{"2":{"187":1}}],["013891050",{"2":{"236":1}}],["01382",{"2":{"187":1}}],["01386",{"2":{"187":1}}],["01387",{"2":{"187":1}}],["01388",{"2":{"187":1}}],["01349",{"2":{"187":1}}],["013299890",{"2":{"236":1}}],["013241904",{"2":{"236":1}}],["01326",{"2":{"187":1}}],["013274295",{"2":{"236":1}}],["01327",{"2":{"187":2}}],["01325",{"2":{"187":1}}],["01351",{"2":{"187":1}}],["013908f",{"2":{"256":1}}],["013991628",{"2":{"236":1}}],["013992591",{"2":{"106":1}}],["013992601",{"2":{"106":1}}],["013967324",{"2":{"236":1}}],["01395",{"2":{"187":1}}],["013194922",{"2":{"236":1}}],["01312",{"2":{"187":1}}],["01310",{"2":{"187":1}}],["01317",{"2":{"187":1}}],["01314",{"2":{"187":1}}],["017522790",{"2":{"236":1}}],["017566014",{"2":{"236":1}}],["017906483",{"2":{"236":1}}],["01797",{"2":{"187":1}}],["017810564",{"2":{"236":1}}],["01785",{"2":{"187":1}}],["017424010",{"2":{"236":1}}],["017435743",{"2":{"236":1}}],["017432783",{"2":{"106":2}}],["01744",{"2":{"187":1}}],["01721",{"2":{"187":1}}],["01727",{"2":{"187":1}}],["01711",{"2":{"187":1}}],["01717",{"2":{"187":1}}],["0171745",{"2":{"168":1}}],["01765",{"2":{"187":1}}],["01766",{"2":{"187":1}}],["01771",{"2":{"187":1}}],["01770",{"2":{"187":1}}],["017309623",{"2":{"236":1}}],["01735",{"2":{"187":1}}],["01738",{"2":{"187":1}}],["01733",{"2":{"187":1}}],["01665f",{"2":{"256":1}}],["016564976",{"2":{"236":1}}],["016914256",{"2":{"236":1}}],["016295999",{"2":{"236":1}}],["016243555",{"2":{"236":1}}],["01624",{"2":{"187":1}}],["016779371",{"2":{"236":1}}],["016781844",{"2":{"236":1}}],["01676",{"2":{"187":1}}],["016142424",{"2":{"236":1}}],["016118057",{"2":{"236":1}}],["01612",{"2":{"187":1}}],["01613",{"2":{"187":1}}],["0163",{"2":{"249":1}}],["01635",{"2":{"187":1}}],["01639",{"2":{"187":1}}],["01607",{"2":{"187":1}}],["01609",{"2":{"187":1}}],["01641",{"2":{"187":1}}],["01644",{"2":{"187":1}}],["016479947",{"2":{"106":2}}],["01683",{"2":{"187":1}}],["0168695",{"2":{"145":1}}],["01f0",{"2":{"145":1,"146":1,"179":2,"187":1}}],["019357491",{"2":{"236":1}}],["019913949",{"2":{"236":1}}],["01997",{"2":{"187":1}}],["019488331",{"2":{"236":1}}],["019566942",{"2":{"236":1}}],["01955",{"2":{"187":1}}],["01951",{"2":{"187":1}}],["01986",{"2":{"187":1}}],["01962",{"2":{"187":1}}],["01965834",{"2":{"126":1}}],["01925",{"2":{"187":1}}],["01923",{"2":{"187":1}}],["019236464",{"2":{"77":1}}],["01974",{"2":{"187":1}}],["019791335",{"2":{"106":1}}],["019791333",{"2":{"106":1}}],["01911576",{"2":{"106":1}}],["019115759",{"2":{"106":1}}],["0189805",{"2":{"241":1}}],["01891",{"2":{"187":1}}],["018100940",{"2":{"236":1}}],["01816",{"2":{"187":1}}],["018278142",{"2":{"236":1}}],["018228574",{"2":{"236":1}}],["01829",{"2":{"187":1}}],["018052436",{"2":{"236":1}}],["018401809",{"2":{"236":1}}],["018484056",{"2":{"236":1}}],["01841",{"2":{"187":1}}],["01865",{"2":{"187":1}}],["01857",{"2":{"187":1}}],["0183s\\ttraining",{"2":{"217":1}}],["018316794",{"2":{"236":1}}],["01831",{"2":{"187":1}}],["018338893",{"2":{"106":1}}],["018338889",{"2":{"106":1}}],["018811548",{"2":{"236":1}}],["01883",{"2":{"187":1}}],["018884756",{"2":{"106":1}}],["018884761",{"2":{"106":1}}],["01879",{"2":{"187":1}}],["018743665453639813",{"2":{"174":1}}],["01872db4",{"2":{"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["0187668",{"2":{"106":1}}],["018701272",{"2":{"77":1}}],["015367235729255e",{"2":{"258":1}}],["015305851",{"2":{"236":1}}],["015340659",{"2":{"236":1}}],["015074271",{"2":{"236":1}}],["015056589",{"2":{"236":1}}],["01505",{"2":{"187":1}}],["0154013f",{"2":{"256":1}}],["015406f",{"2":{"256":1}}],["015475f",{"2":{"256":1}}],["015478307",{"2":{"244":1}}],["015417031",{"2":{"236":1}}],["015482016",{"2":{"236":1}}],["01542",{"2":{"187":1}}],["01542492",{"2":{"107":1}}],["015850905",{"2":{"236":1}}],["015801162",{"2":{"236":1}}],["015882893",{"2":{"236":1}}],["015886255",{"2":{"236":1}}],["0158726",{"2":{"77":1}}],["015216023",{"2":{"236":1}}],["015703991737016e",{"2":{"258":1}}],["015774786",{"2":{"236":1}}],["015748031",{"2":{"241":2}}],["01574",{"2":{"187":1}}],["015725624",{"2":{"236":1}}],["01572",{"2":{"187":1}}],["015927596",{"2":{"236":1}}],["01598",{"2":{"187":1}}],["015937408",{"2":{"126":1,"129":1}}],["01516",{"2":{"187":1}}],["01515",{"2":{"187":1}}],["015592407",{"2":{"236":1}}],["015559800",{"2":{"236":1}}],["01558",{"2":{"187":1}}],["01552",{"2":{"187":1}}],["0155126",{"2":{"106":1}}],["015693573368616e",{"2":{"258":1}}],["01569",{"2":{"187":1}}],["01568",{"2":{"187":1}}],["010948162",{"2":{"236":1}}],["010941349",{"2":{"236":1}}],["01096",{"2":{"187":1}}],["010911969",{"2":{"236":1}}],["01091",{"2":{"187":1}}],["010460960",{"2":{"236":1}}],["010467471",{"2":{"236":1}}],["010452234",{"2":{"236":1}}],["01047",{"2":{"187":1}}],["010404",{"2":{"145":1}}],["010381337",{"2":{"236":1}}],["01039",{"2":{"187":1}}],["01035",{"2":{"187":1}}],["010597511",{"2":{"236":1}}],["010531792",{"2":{"236":1}}],["010536365",{"2":{"236":1}}],["010573968",{"2":{"236":1}}],["010573865",{"2":{"236":1}}],["010572717",{"2":{"236":1}}],["010586375",{"2":{"236":1}}],["01055",{"2":{"187":1}}],["0105465",{"2":{"77":1}}],["010649174",{"2":{"236":1}}],["01064s\\ttraining",{"2":{"228":8}}],["010623454",{"2":{"236":1}}],["01062s\\ttraining",{"2":{"228":1}}],["01065s\\ttraining",{"2":{"228":6}}],["010637476",{"2":{"236":1}}],["010637204",{"2":{"106":1}}],["010637202",{"2":{"106":1}}],["01063s\\ttraining",{"2":{"228":3}}],["01060s\\ttraining",{"2":{"228":2}}],["01067s\\ttraining",{"2":{"228":9}}],["01066s\\ttraining",{"2":{"228":7}}],["01068s\\ttraining",{"2":{"228":6}}],["01069s\\ttraining",{"2":{"228":6}}],["010617163",{"2":{"236":1}}],["01061s\\ttraining",{"2":{"228":1}}],["01061",{"2":{"187":1}}],["010728635",{"2":{"236":1}}],["010727987",{"2":{"236":1}}],["01072s\\ttraining",{"2":{"228":2}}],["01079s\\ttraining",{"2":{"228":1}}],["01073s\\ttraining",{"2":{"228":2}}],["01071s\\ttraining",{"2":{"228":7}}],["01070s\\ttraining",{"2":{"228":5}}],["01074s\\ttraining",{"2":{"228":2}}],["01075",{"2":{"187":1}}],["01078",{"2":{"187":1}}],["01077s\\ttraining",{"2":{"228":1}}],["01077",{"2":{"187":1}}],["01076",{"2":{"187":2}}],["010039996",{"2":{"236":1}}],["01001",{"2":{"187":1}}],["01008",{"2":{"187":1}}],["0100971125",{"2":{"77":1}}],["010883353",{"2":{"236":1}}],["01088s\\ttraining",{"2":{"228":2}}],["01084s\\ttraining",{"2":{"228":1}}],["010866777",{"2":{"236":1}}],["0108668577150213",{"2":{"171":1}}],["01086s\\ttraining",{"2":{"228":1}}],["01085s\\ttraining",{"2":{"228":1}}],["01082s\\ttraining",{"2":{"228":1}}],["01083s\\ttraining",{"2":{"228":1}}],["010876307",{"2":{"236":1}}],["01087s\\ttraining",{"2":{"228":1}}],["01087",{"2":{"187":1}}],["010811541",{"2":{"236":1}}],["01081s\\ttraining",{"2":{"228":2}}],["01081",{"2":{"187":2}}],["010297f",{"2":{"256":1}}],["010293764",{"2":{"236":1}}],["010204735",{"2":{"236":1}}],["010220452",{"2":{"236":1}}],["01023",{"2":{"187":1}}],["01026",{"2":{"187":1}}],["010257332098586e",{"2":{"258":1}}],["01025",{"2":{"187":1}}],["0102189295",{"2":{"106":1}}],["010218928",{"2":{"106":1}}],["010190512",{"2":{"236":1}}],["010141237",{"2":{"236":1}}],["01015",{"2":{"187":1}}],["01013",{"2":{"187":1}}],["0101",{"2":{"78":1}}],["0122",{"2":{"249":1}}],["012213146",{"2":{"236":1}}],["01223",{"2":{"187":1}}],["012733568",{"2":{"236":1}}],["012724041",{"2":{"236":1}}],["01270",{"2":{"187":1}}],["012929341",{"2":{"236":1}}],["012921538",{"2":{"236":1}}],["01291",{"2":{"187":1}}],["0120984053104614e",{"2":{"258":1}}],["01207s\\ttraining",{"2":{"228":1}}],["012041669",{"2":{"236":1}}],["01204",{"2":{"187":1}}],["01208",{"2":{"187":1}}],["012447415",{"2":{"236":1}}],["012446284",{"2":{"236":1}}],["01242",{"2":{"187":1}}],["01247",{"2":{"187":1}}],["012518023",{"2":{"236":1}}],["01251s\\ttraining",{"2":{"228":1}}],["01258",{"2":{"187":1}}],["012522168",{"2":{"77":1}}],["012820827",{"2":{"236":1}}],["012885604",{"2":{"236":1}}],["01283",{"2":{"187":1}}],["01289",{"2":{"187":1}}],["01262",{"2":{"187":1}}],["01264",{"2":{"187":1}}],["012341642135895e",{"2":{"258":1}}],["012398f",{"2":{"256":1}}],["012390045",{"2":{"236":1}}],["012356437",{"2":{"236":1}}],["012318298",{"2":{"236":1}}],["0123139",{"2":{"77":1}}],["012339489",{"2":{"236":1}}],["01233s\\ttraining",{"2":{"228":1}}],["01233",{"2":{"187":1}}],["01236",{"2":{"187":1}}],["012387147",{"2":{"236":1}}],["01238",{"2":{"187":1}}],["012136834",{"2":{"236":1}}],["012116373",{"2":{"236":1}}],["012141772",{"2":{"236":1}}],["01214s\\ttraining",{"2":{"228":1}}],["01218s\\ttraining",{"2":{"228":1}}],["012183795",{"2":{"106":1}}],["012183794",{"2":{"106":1}}],["01210s\\ttraining",{"2":{"228":1}}],["01219s\\ttraining",{"2":{"228":1}}],["01212s\\ttraining",{"2":{"228":2}}],["01215s\\ttraining",{"2":{"228":1}}],["01217",{"2":{"187":1}}],["01216s\\ttraining",{"2":{"228":1}}],["01216",{"2":{"187":1}}],["0149",{"2":{"249":1}}],["014784457",{"2":{"236":1}}],["014771502",{"2":{"236":1}}],["014708348",{"2":{"236":1}}],["01437",{"2":{"187":1}}],["01433",{"2":{"187":1}}],["014529660",{"2":{"236":1}}],["014503423",{"2":{"236":1}}],["01454",{"2":{"187":1}}],["01451",{"2":{"187":1}}],["01445",{"2":{"187":1}}],["01448",{"2":{"187":1}}],["01444",{"2":{"187":1}}],["014405347",{"2":{"236":1}}],["01440",{"2":{"187":1}}],["01468",{"2":{"187":1}}],["014612451",{"2":{"236":1}}],["01461",{"2":{"187":1}}],["0146587",{"2":{"77":1}}],["014263613",{"2":{"236":1}}],["014241925",{"2":{"236":1}}],["014243502",{"2":{"236":1}}],["01428",{"2":{"187":1}}],["0142814",{"2":{"77":1}}],["01420",{"2":{"187":1}}],["014815168",{"2":{"236":1}}],["014845728",{"2":{"236":1}}],["014840655",{"2":{"236":1}}],["01484",{"2":{"187":1}}],["0148516611103858e",{"2":{"258":1}}],["01485",{"2":{"187":1}}],["014877922",{"2":{"106":1}}],["01405",{"2":{"187":1}}],["01415047",{"2":{"106":1}}],["014150472",{"2":{"106":1}}],["014100174",{"2":{"77":1}}],["011266185",{"2":{"236":1}}],["011235875",{"2":{"236":1}}],["011299610",{"2":{"236":1}}],["01122",{"2":{"187":1}}],["011309550",{"2":{"236":1}}],["011386497",{"2":{"236":1}}],["01136",{"2":{"187":1}}],["011628653",{"2":{"236":1}}],["011624727",{"2":{"236":1}}],["011651803",{"2":{"236":1}}],["011666608",{"2":{"236":1}}],["011611394",{"2":{"106":1}}],["011611392",{"2":{"106":1}}],["011070153",{"2":{"236":1}}],["01107s\\ttraining",{"2":{"228":1}}],["011011195",{"2":{"236":1}}],["01103",{"2":{"187":2}}],["011443286",{"2":{"236":1}}],["011416912",{"2":{"236":1}}],["01143",{"2":{"187":1}}],["01147",{"2":{"187":1}}],["011942f",{"2":{"256":1}}],["011947901",{"2":{"236":1}}],["01192",{"2":{"187":1}}],["011937321",{"2":{"179":1}}],["011868341",{"2":{"236":1}}],["01189s\\ttraining",{"2":{"228":1}}],["01183s\\ttraining",{"2":{"228":1}}],["01183",{"2":{"187":1}}],["01185",{"2":{"187":1}}],["011500452",{"2":{"236":1}}],["011515658",{"2":{"236":1}}],["011523969",{"2":{"236":1}}],["01157",{"2":{"187":1}}],["01158",{"2":{"187":1}}],["011797804",{"2":{"236":1}}],["01177s\\ttraining",{"2":{"228":1}}],["01177",{"2":{"187":1}}],["01178",{"2":{"187":1}}],["011734666",{"2":{"77":1}}],["011739168",{"2":{"77":1}}],["011197914",{"2":{"236":1}}],["011115490",{"2":{"236":1}}],["01118",{"2":{"187":1}}],["01110",{"2":{"187":1}}],["011100831f0",{"2":{"63":1}}],["01",{"2":{"28":3,"63":2,"179":1,"202":3,"228":1,"258":1}}],["0",{"2":{"2":1,"4":1,"7":3,"11":1,"19":3,"28":44,"41":2,"48":3,"50":3,"53":2,"54":10,"55":1,"59":2,"63":122,"64":4,"69":16,"74":1,"77":204,"78":14,"106":292,"107":11,"109":5,"110":37,"115":1,"116":1,"118":2,"120":1,"126":54,"129":25,"131":8,"135":6,"136":1,"137":7,"145":18,"146":5,"147":35,"148":23,"152":7,"153":358,"155":1,"168":55,"170":1,"171":155,"172":15,"173":3,"174":62,"176":12,"177":9,"178":4,"179":72,"180":11,"183":1,"186":1,"187":452,"189":11,"192":1,"193":2,"194":2,"195":7,"197":3,"200":4,"202":2,"204":11,"213":1,"214":2,"215":1,"216":2,"217":26,"218":1,"219":8,"220":19,"221":11,"224":2,"227":2,"228":105,"229":11,"235":7,"236":800,"237":6,"238":11,"241":139,"242":1,"243":4,"244":40,"245":11,"246":11,"248":4,"249":66,"251":3,"254":4,"255":9,"256":472,"257":1,"258":389,"260":3}}],["1e",{"2":{"202":1,"254":1}}],["1g",{"2":{"180":1,"189":1,"204":1,"221":1,"246":1}}],["1×16",{"2":{"147":1,"148":1}}],["1×32",{"2":{"69":1,"78":1}}],["1`",{"2":{"136":3}}],["1st",{"2":{"127":1}}],["1116973448797237e",{"2":{"258":1}}],["1114492f",{"2":{"256":1}}],["11111",{"2":{"219":1}}],["111552626",{"2":{"106":1}}],["1149924f",{"2":{"256":1}}],["114975",{"2":{"147":1}}],["1143",{"2":{"249":1}}],["114",{"2":{"196":4}}],["11007148",{"2":{"241":1}}],["11001",{"2":{"187":1,"236":1}}],["11023622",{"2":{"241":2}}],["11023762",{"2":{"179":1}}],["110",{"2":{"196":1,"220":6}}],["11010",{"2":{"187":1}}],["119263521317027e",{"2":{"258":1}}],["119031297957508e",{"2":{"258":1}}],["1190964f",{"2":{"256":1}}],["11942f",{"2":{"256":1}}],["119",{"2":{"217":1}}],["11993",{"2":{"187":1}}],["11957",{"2":{"187":1}}],["11984776",{"2":{"77":1}}],["116989",{"2":{"171":1}}],["116796836",{"2":{"106":1}}],["115053432664885e",{"2":{"258":1}}],["11501",{"2":{"236":1}}],["115342f",{"2":{"256":1}}],["11577im",{"2":{"168":1}}],["11515346",{"2":{"106":1}}],["11515345",{"2":{"106":1}}],["1181102",{"2":{"241":2}}],["1188200246206075",{"2":{"153":2}}],["1187928",{"2":{"106":2}}],["1121",{"2":{"249":1}}],["112",{"2":{"196":1}}],["1126356",{"2":{"148":1}}],["1124609",{"2":{"106":1}}],["113",{"2":{"196":2}}],["11373",{"2":{"187":1}}],["1137304",{"2":{"147":1}}],["11365668",{"2":{"106":2}}],["117236",{"2":{"241":1}}],["1173706",{"2":{"147":1}}],["1175871f",{"2":{"106":1}}],["11",{"2":{"109":1,"145":2,"146":2,"147":2,"148":1,"180":2,"187":14,"189":2,"204":2,"220":3,"221":2,"228":3,"229":2,"238":2,"245":1,"246":2,"249":3,"258":3}}],["146448875127744e",{"2":{"258":1}}],["146762f",{"2":{"256":1}}],["14681",{"2":{"187":1}}],["140935461817854e",{"2":{"258":1}}],["1409915f",{"2":{"256":1}}],["1406",{"2":{"249":2}}],["1405",{"2":{"249":1}}],["14001",{"2":{"236":1}}],["1438525558973062e",{"2":{"258":1}}],["1434443f",{"2":{"256":1}}],["14340512",{"2":{"107":1}}],["1439",{"2":{"249":1}}],["1437841",{"2":{"179":1}}],["1497044230215683e",{"2":{"258":1}}],["1491996992752195e",{"2":{"258":1}}],["1493156f",{"2":{"256":1}}],["149032f",{"2":{"256":1}}],["1490868",{"2":{"244":1}}],["1496063",{"2":{"241":2}}],["149648",{"2":{"168":1}}],["14941669",{"2":{"77":1}}],["1458",{"2":{"249":1}}],["14501",{"2":{"236":1}}],["145296",{"2":{"168":1}}],["14568081",{"2":{"106":1}}],["147705370924899",{"2":{"153":1}}],["1477053709248992",{"2":{"153":1}}],["142470881033475",{"2":{"153":2}}],["142251",{"2":{"106":1}}],["148804818537372e",{"2":{"258":1}}],["1488954f",{"2":{"256":1}}],["1488175",{"2":{"117":1}}],["14815",{"2":{"217":2}}],["14825",{"2":{"171":1}}],["148248",{"2":{"171":2}}],["1489007",{"2":{"147":1}}],["14",{"2":{"145":1,"187":14,"228":3,"249":7}}],["144347f",{"2":{"256":1}}],["14418",{"2":{"187":1}}],["14445141",{"2":{"106":1}}],["14488737",{"2":{"106":1}}],["141839913481628e",{"2":{"258":1}}],["14132261",{"2":{"241":1}}],["14173229",{"2":{"241":2}}],["14118214",{"2":{"106":1}}],["1415927",{"2":{"63":1}}],["173052860755846e",{"2":{"258":1}}],["17322835",{"2":{"241":2}}],["178766f",{"2":{"256":1}}],["17816184",{"2":{"241":1}}],["179059263073716e",{"2":{"258":1}}],["1790565f",{"2":{"256":1}}],["179291785548102e",{"2":{"258":1}}],["179899351797133e",{"2":{"258":1}}],["1791",{"2":{"249":1}}],["179584495291061",{"2":{"153":2}}],["171800841637475e",{"2":{"258":1}}],["1718045f",{"2":{"256":1}}],["17111921",{"2":{"244":1}}],["171316e",{"2":{"202":1}}],["1759023f",{"2":{"256":1}}],["17501",{"2":{"236":1}}],["17571",{"2":{"187":1}}],["1728236f",{"2":{"256":1}}],["17258",{"2":{"187":1}}],["172325e",{"2":{"148":1}}],["17239982",{"2":{"77":1}}],["17706361",{"2":{"179":1}}],["1706",{"2":{"249":1}}],["17001",{"2":{"236":1}}],["170342",{"2":{"171":1}}],["17048413",{"2":{"106":1}}],["1761580142006772e",{"2":{"258":1}}],["176260519083579e",{"2":{"258":1}}],["17642799519663e",{"2":{"258":1}}],["176428f",{"2":{"256":1}}],["1764085",{"2":{"126":1}}],["1763",{"2":{"249":1}}],["176",{"2":{"202":1}}],["1769385",{"2":{"147":1}}],["17",{"2":{"110":1,"148":1,"187":14,"196":10,"228":3,"242":1,"249":4,"258":14}}],["174910936900174e",{"2":{"258":1}}],["1741965f",{"2":{"256":1}}],["174792f",{"2":{"256":1}}],["1744522063627435e",{"2":{"258":1}}],["17441",{"2":{"171":1}}],["17449912",{"2":{"77":1}}],["17435724",{"2":{"106":2}}],["1746128",{"2":{"77":1}}],["184531804194882e",{"2":{"258":1}}],["1844984f",{"2":{"256":1}}],["1846",{"2":{"249":1}}],["1846486482317626",{"2":{"153":2}}],["18481395",{"2":{"244":1}}],["1822244448951675e",{"2":{"258":1}}],["1826226626589792e",{"2":{"258":1}}],["182381f",{"2":{"256":1}}],["1825823f",{"2":{"256":1}}],["1828",{"2":{"249":1}}],["1828456",{"2":{"241":1}}],["1820s\\ttraining",{"2":{"219":1}}],["18550416894993e",{"2":{"258":1}}],["1855652f",{"2":{"256":1}}],["185817229987542e",{"2":{"258":1}}],["185795f",{"2":{"256":1}}],["18535946",{"2":{"244":1}}],["18564",{"2":{"241":1}}],["18501",{"2":{"236":1}}],["18519",{"2":{"217":2}}],["185124",{"2":{"147":1}}],["189394359787317e",{"2":{"258":1}}],["189355f",{"2":{"256":1}}],["18932f",{"2":{"256":1}}],["1898917f",{"2":{"256":1}}],["18915",{"2":{"187":1}}],["18941",{"2":{"168":1}}],["180583505736319e",{"2":{"258":1}}],["1805106f",{"2":{"256":1}}],["18056087",{"2":{"179":1}}],["180067f",{"2":{"256":1}}],["18001",{"2":{"236":1}}],["180992f",{"2":{"256":1}}],["18099",{"2":{"187":1}}],["18072",{"2":{"187":1}}],["18089005",{"2":{"106":1}}],["187904699619065e",{"2":{"258":1}}],["18794",{"2":{"168":1}}],["18741",{"2":{"187":1}}],["1871622",{"2":{"176":4}}],["187646",{"2":{"168":1}}],["1862644435950507e",{"2":{"258":1}}],["186",{"2":{"256":1}}],["186746e",{"2":{"202":1}}],["18695377",{"2":{"179":1}}],["18652338",{"2":{"148":1}}],["186451",{"2":{"148":1}}],["1837590217856658e",{"2":{"258":1}}],["1830252216760097e",{"2":{"258":1}}],["18308182",{"2":{"126":1}}],["1834459f",{"2":{"256":1}}],["1832733f",{"2":{"256":1}}],["1832",{"2":{"249":1}}],["18365",{"2":{"168":1}}],["1836515766351254",{"2":{"153":2}}],["183307",{"2":{"147":1}}],["188485720674376e",{"2":{"258":1}}],["18841726",{"2":{"179":1}}],["1887441f",{"2":{"256":1}}],["188779f",{"2":{"256":1}}],["1887817",{"2":{"241":1}}],["18860114",{"2":{"126":1}}],["18807876",{"2":{"106":2}}],["18811704",{"2":{"77":1}}],["181862689971448e",{"2":{"258":1}}],["181843536374034e",{"2":{"258":1}}],["1815495f",{"2":{"256":1}}],["1817697f",{"2":{"256":1}}],["18173",{"2":{"187":1}}],["1811024",{"2":{"241":2}}],["18102339",{"2":{"77":1}}],["181208",{"2":{"77":1}}],["18",{"2":{"69":1,"187":14,"196":1,"217":1,"228":3,"249":5,"258":8}}],["1−2∑yy^+α∑y2+∑y^2+α",{"2":{"63":1}}],["1−α",{"2":{"63":2}}],["1−y",{"2":{"63":1}}],["1−yy^",{"2":{"63":2}}],["1−y^+ϵ",{"2":{"63":1}}],["1−y~",{"2":{"63":2}}],["1−z",{"2":{"52":1}}],["1th",{"2":{"55":3}}],["134042725249435e",{"2":{"258":1}}],["134072f",{"2":{"256":1}}],["13475561",{"2":{"241":1}}],["1330720333450102e",{"2":{"258":1}}],["133156f",{"2":{"256":1}}],["13343152",{"2":{"106":1}}],["136344793206388e",{"2":{"258":1}}],["136356",{"2":{"168":1}}],["1364634f",{"2":{"256":1}}],["13682",{"2":{"187":1}}],["13",{"2":{"187":14,"228":2,"249":3}}],["13234",{"2":{"187":1}}],["1304326276892678e",{"2":{"258":1}}],["1304417f",{"2":{"256":1}}],["1303",{"2":{"249":1}}],["13001",{"2":{"236":1}}],["13057",{"2":{"187":1}}],["13073668",{"2":{"179":1}}],["1306755",{"2":{"148":1}}],["13024",{"2":{"187":1}}],["1302",{"2":{"54":1}}],["13811",{"2":{"187":1}}],["13819042",{"2":{"126":1}}],["13864",{"2":{"106":1}}],["13725637",{"2":{"106":1}}],["13725635",{"2":{"106":1}}],["13760868",{"2":{"106":1}}],["13760866",{"2":{"106":1}}],["1377298",{"2":{"106":2}}],["1356313369812806e",{"2":{"258":1}}],["1355753f",{"2":{"256":1}}],["13559",{"2":{"171":1}}],["13501",{"2":{"236":1}}],["1350521",{"2":{"147":1}}],["13504265",{"2":{"106":1}}],["13518013",{"2":{"106":1}}],["13518015",{"2":{"106":1}}],["1392578643109326e",{"2":{"258":1}}],["1399819895018836e",{"2":{"258":1}}],["139529f",{"2":{"256":1}}],["1395766",{"2":{"241":1}}],["1393765f",{"2":{"256":1}}],["139314+1",{"2":{"168":1}}],["139689f",{"2":{"256":1}}],["139433",{"2":{"148":1}}],["13913931",{"2":{"106":1}}],["13988f",{"2":{"77":1}}],["1314715527015514e",{"2":{"258":1}}],["1310",{"2":{"249":1}}],["13193",{"2":{"187":1}}],["131726",{"2":{"147":1,"148":1}}],["131583631",{"2":{"78":1}}],["13168f",{"2":{"77":1}}],["1312",{"2":{"28":1}}],["1c",{"2":{"52":1}}],["1b",{"2":{"52":3}}],["1a",{"2":{"52":3}}],["168168734739605e",{"2":{"258":1}}],["168563587758528e",{"2":{"258":1}}],["168399f",{"2":{"256":1}}],["166704533135075e",{"2":{"258":1}}],["1667444f",{"2":{"256":1}}],["165037f",{"2":{"256":1}}],["16501",{"2":{"236":1}}],["1652",{"2":{"249":1}}],["16566901",{"2":{"145":1}}],["167098083456447e",{"2":{"258":1}}],["167989f",{"2":{"256":1}}],["167108f",{"2":{"256":1}}],["1671s\\ttraining",{"2":{"217":1}}],["16786",{"2":{"187":1}}],["162200848409247e",{"2":{"258":1}}],["1623195f",{"2":{"256":1}}],["162357",{"2":{"168":1}}],["16241",{"2":{"220":12}}],["16240",{"2":{"220":12}}],["16261",{"2":{"187":1}}],["161735212402272e",{"2":{"258":1}}],["1617228180412864",{"2":{"153":2}}],["161007f",{"2":{"256":1}}],["161695f",{"2":{"256":1}}],["16187939",{"2":{"106":1}}],["16450",{"2":{"220":6}}],["16454",{"2":{"187":1}}],["164786",{"2":{"171":1}}],["164379",{"2":{"168":1}}],["1642133",{"2":{"147":1}}],["1641532f",{"2":{"106":1}}],["16407",{"2":{"106":1}}],["16409892",{"2":{"106":1}}],["169893693848465e",{"2":{"258":1}}],["1696584f",{"2":{"256":1}}],["16946f",{"2":{"77":1}}],["16909766",{"2":{"77":1}}],["160398391589894e",{"2":{"258":1}}],["160932848385687e",{"2":{"258":1}}],["16094846",{"2":{"145":1}}],["16001",{"2":{"236":1}}],["160876",{"2":{"168":1}}],["16083185f0",{"2":{"63":1}}],["160832f0",{"2":{"63":1}}],["1607115f",{"2":{"256":1}}],["1607",{"2":{"19":2,"55":1}}],["16",{"2":{"46":1,"69":3,"109":13,"110":14,"115":1,"147":1,"148":1,"171":1,"187":14,"193":3,"217":2,"228":3,"229":2,"235":1,"236":12,"238":2,"242":4,"245":2,"249":4,"251":4,"258":8,"260":4}}],["10f",{"2":{"257":1}}],["104966340033266e",{"2":{"258":1}}],["104951f",{"2":{"256":1}}],["10458f",{"2":{"77":1}}],["108145",{"2":{"241":1}}],["108",{"2":{"196":1}}],["105066009507857e",{"2":{"258":1}}],["10501",{"2":{"236":1}}],["105244334542321e",{"2":{"258":1}}],["1052587f",{"2":{"256":1}}],["10527",{"2":{"187":1}}],["1058",{"2":{"249":1}}],["10552457",{"2":{"106":1}}],["105524555",{"2":{"106":1}}],["101",{"2":{"202":1,"220":6,"245":1}}],["10197",{"2":{"187":1}}],["10140238162746013",{"2":{"153":2}}],["107888",{"2":{"241":1}}],["10795",{"2":{"187":1}}],["1079326",{"2":{"147":1}}],["107044+0",{"2":{"168":1}}],["10754459",{"2":{"106":1}}],["102609950833082e",{"2":{"258":1}}],["1023929082758117e",{"2":{"258":1}}],["102839f",{"2":{"256":1}}],["10227",{"2":{"187":1}}],["102432f",{"2":{"256":1}}],["1024",{"2":{"153":7}}],["102",{"2":{"109":1}}],["109687316864863e",{"2":{"258":1}}],["109513202642436e",{"2":{"258":1}}],["1095227f",{"2":{"256":1}}],["1095082",{"2":{"106":1}}],["10950818",{"2":{"106":1}}],["1090",{"2":{"254":2}}],["10904",{"2":{"187":1}}],["109261915",{"2":{"126":1}}],["109203495",{"2":{"106":1}}],["106187531039724e",{"2":{"258":1}}],["1060815f",{"2":{"256":1}}],["106069",{"2":{"145":1}}],["10666",{"2":{"187":1}}],["10688411",{"2":{"106":1}}],["106884085",{"2":{"106":1}}],["106985640034847e",{"2":{"258":1}}],["106946f",{"2":{"256":1}}],["1069",{"2":{"106":1}}],["1032465",{"2":{"106":1}}],["10",{"2":{"41":2,"46":2,"48":1,"55":12,"59":1,"74":2,"76":1,"77":5,"78":1,"106":5,"117":2,"118":1,"145":2,"146":2,"148":2,"171":4,"174":2,"179":3,"180":5,"187":14,"189":5,"193":5,"195":1,"196":22,"197":2,"204":5,"215":5,"220":30,"221":5,"226":1,"228":2,"229":5,"238":5,"246":5,"249":4,"250":1,"251":2,"254":2,"258":12,"260":2}}],["1001",{"2":{"179":1,"236":1}}],["1003911",{"2":{"137":1}}],["10001",{"2":{"236":1}}],["10000",{"2":{"179":2,"236":1}}],["1000",{"2":{"69":1,"78":3,"107":14,"179":1}}],["100",{"2":{"28":1,"49":6,"59":2,"78":1,"107":2,"180":2,"189":2,"195":2,"196":1,"197":1,"204":2,"217":2,"220":6,"221":2,"228":35,"229":3,"238":2,"246":2,"251":1,"255":2,"260":1}}],["12207874832416e",{"2":{"258":1}}],["1220454f",{"2":{"256":1}}],["1229820843349717e",{"2":{"258":1}}],["1216831364397432e",{"2":{"258":1}}],["1216441f",{"2":{"256":1}}],["12181",{"2":{"187":1}}],["12096125",{"2":{"244":1}}],["12001",{"2":{"236":1}}],["1204217935313214",{"2":{"153":2}}],["12s",{"2":{"228":2}}],["1265",{"2":{"249":1}}],["126",{"2":{"202":1}}],["1261990389976131",{"2":{"174":1}}],["127101300288545e",{"2":{"258":1}}],["1270174f",{"2":{"256":1}}],["12709",{"2":{"187":1}}],["127726f",{"2":{"256":1}}],["12772",{"2":{"187":1}}],["1277556f0",{"2":{"63":1}}],["12501",{"2":{"236":1}}],["12589",{"2":{"179":1}}],["12556516",{"2":{"106":2}}],["12477568",{"2":{"179":1}}],["12442",{"2":{"168":1}}],["1245008",{"2":{"147":1}}],["12abac4f24f6",{"2":{"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["129337445068244e",{"2":{"258":1}}],["1293236",{"2":{"106":1}}],["129173f",{"2":{"256":1}}],["12902719",{"2":{"106":2}}],["12992674",{"2":{"106":1}}],["129",{"2":{"55":1,"69":1}}],["128056570344334e",{"2":{"258":1}}],["1286439",{"2":{"241":1}}],["12827216",{"2":{"106":1}}],["128",{"2":{"46":2,"55":3,"69":12,"77":4,"193":6,"195":1,"217":1,"241":4,"251":1,"260":1}}],["12",{"2":{"41":2,"135":1,"147":6,"148":7,"152":9,"171":2,"180":6,"187":14,"189":6,"197":1,"204":6,"221":6,"228":14,"229":7,"238":6,"246":6,"249":5,"251":1,"254":1,"260":1}}],["123839607928261e",{"2":{"258":1}}],["1238396f",{"2":{"256":1}}],["1234",{"2":{"248":1}}],["12345",{"2":{"241":1}}],["1239749674027375",{"2":{"153":2}}],["123",{"2":{"28":3,"48":1,"153":1}}],["1=dense",{"2":{"39":1}}],["1d",{"2":{"28":1,"52":1}}],["1f0",{"2":{"19":3,"55":2,"63":2,"69":1,"78":1,"241":1}}],["1977007881848314e",{"2":{"258":1}}],["1976448f",{"2":{"256":1}}],["197045",{"2":{"171":1}}],["1988",{"2":{"254":2}}],["1984668",{"2":{"145":1}}],["1943",{"2":{"249":1}}],["1949s\\ttraining",{"2":{"217":1}}],["19s",{"2":{"196":1}}],["19679",{"2":{"187":1}}],["196703",{"2":{"131":2}}],["19666",{"2":{"187":1}}],["1954633811668815e",{"2":{"258":1}}],["195234f",{"2":{"256":1}}],["195746",{"2":{"241":1}}],["19501",{"2":{"236":1}}],["1953009",{"2":{"177":1,"178":1}}],["19597391412112541",{"2":{"174":1}}],["1958431",{"2":{"179":1}}],["1958",{"2":{"17":1,"249":1}}],["19131751478338e",{"2":{"258":1}}],["1912615f",{"2":{"256":1}}],["19124654",{"2":{"145":1}}],["1914",{"2":{"249":1}}],["191956",{"2":{"172":1}}],["19381",{"2":{"187":1}}],["19397983",{"2":{"179":1}}],["193291",{"2":{"171":1}}],["1936812",{"2":{"145":1}}],["1900587960646143e",{"2":{"258":1}}],["19001",{"2":{"236":1}}],["1904755",{"2":{"148":1}}],["19084926",{"2":{"106":1}}],["199854510901844e",{"2":{"258":1}}],["1990",{"2":{"149":1}}],["199387",{"2":{"145":1}}],["19975941",{"2":{"77":1}}],["19",{"2":{"116":1,"187":14,"196":1,"217":4,"228":3,"249":5,"258":2}}],["192567684357778e",{"2":{"258":1}}],["1926393f",{"2":{"256":1}}],["192604f",{"2":{"256":1}}],["192836",{"2":{"168":1}}],["1921258",{"2":{"147":1}}],["19209f",{"2":{"106":2}}],["19296975380428e",{"2":{"258":1}}],["1929",{"2":{"17":1}}],["1531335089642805e",{"2":{"258":1}}],["152652555161868e",{"2":{"258":1}}],["1529537f",{"2":{"256":1}}],["152264f",{"2":{"256":1}}],["152749525733976e",{"2":{"258":1}}],["1527586f",{"2":{"256":1}}],["15276335",{"2":{"135":1}}],["159364078864327e",{"2":{"258":1}}],["1593237f",{"2":{"256":1}}],["1591883255837038e",{"2":{"258":1}}],["159159f",{"2":{"256":1}}],["15911",{"2":{"187":1}}],["159042358",{"2":{"236":2}}],["1565",{"2":{"249":1}}],["15681",{"2":{"220":6}}],["15680",{"2":{"220":6}}],["15686738",{"2":{"106":1}}],["156",{"2":{"193":2}}],["15615",{"2":{"187":1}}],["1550",{"2":{"249":2}}],["15501",{"2":{"236":1}}],["15561",{"2":{"187":1}}],["1557574",{"2":{"148":1}}],["15544322",{"2":{"106":1}}],["15542f",{"2":{"77":1}}],["15701",{"2":{"220":6}}],["15700",{"2":{"220":12}}],["15707001",{"2":{"147":1}}],["15738",{"2":{"187":1}}],["15733",{"2":{"187":1}}],["1577715",{"2":{"77":1}}],["151576646692004e",{"2":{"258":1}}],["1515363f",{"2":{"256":1}}],["151280825108199e",{"2":{"258":1}}],["1511",{"2":{"249":1}}],["151",{"2":{"202":1,"245":1}}],["15102",{"2":{"187":1}}],["15107603",{"2":{"106":1}}],["15107606",{"2":{"106":1}}],["15133555",{"2":{"106":1}}],["158470438758418e",{"2":{"258":1}}],["1584s\\ttraining",{"2":{"217":1}}],["1583008022675615e",{"2":{"258":1}}],["158323f",{"2":{"256":1}}],["15873",{"2":{"168":1}}],["1586047939092094",{"2":{"153":2}}],["1585735",{"2":{"106":1}}],["15802455",{"2":{"106":1}}],["154109318293463e",{"2":{"258":1}}],["154289f",{"2":{"256":1}}],["15424",{"2":{"187":1}}],["154388f",{"2":{"256":1}}],["154041",{"2":{"171":1}}],["1545162",{"2":{"148":1}}],["15451166",{"2":{"77":1}}],["154",{"2":{"116":1,"193":2}}],["15493166",{"2":{"106":1}}],["1549292",{"2":{"106":1}}],["150532385385703e",{"2":{"258":1}}],["1503678f",{"2":{"256":1}}],["1501",{"2":{"236":1}}],["1500018045876406e",{"2":{"258":1}}],["1500239f",{"2":{"256":1}}],["15001",{"2":{"236":1}}],["1500",{"2":{"213":2,"250":1}}],["150",{"2":{"55":1}}],["15",{"2":{"17":1,"63":1,"147":1,"148":3,"171":4,"180":3,"187":14,"189":3,"197":1,"204":3,"221":3,"228":2,"229":4,"238":5,"246":3,"249":5,"251":1,"260":1}}],["1",{"2":{"2":1,"4":1,"5":2,"7":1,"11":4,"17":8,"19":10,"28":48,"39":6,"46":5,"48":27,"49":9,"50":1,"51":24,"53":7,"54":12,"55":13,"56":3,"58":6,"59":1,"62":1,"63":110,"64":1,"69":24,"77":57,"78":13,"88":1,"97":1,"102":2,"106":36,"107":5,"109":35,"110":47,"115":6,"117":1,"118":2,"126":4,"127":2,"128":2,"129":9,"131":4,"135":3,"136":11,"137":1,"140":2,"145":13,"147":23,"148":19,"150":3,"151":3,"152":3,"153":88,"168":5,"171":45,"173":4,"174":5,"176":4,"177":6,"178":1,"179":6,"180":4,"183":11,"185":1,"186":1,"187":116,"189":4,"192":5,"193":8,"194":1,"195":5,"196":2,"197":1,"200":12,"202":17,"203":7,"204":4,"213":5,"214":3,"216":1,"217":6,"218":1,"219":1,"220":118,"221":4,"224":4,"225":1,"226":1,"227":1,"228":7,"229":4,"233":1,"234":6,"235":3,"236":6,"237":5,"238":4,"241":90,"242":5,"243":1,"244":4,"245":17,"246":4,"248":6,"249":68,"250":9,"251":1,"254":37,"255":7,"256":111,"257":2,"258":108,"259":5,"260":1}}],["e^2",{"2":{"255":1}}],["err",{"2":{"237":3}}],["errs",{"2":{"237":3}}],["errored",{"2":{"108":1}}],["errors",{"2":{"40":3,"111":1,"202":1}}],["error",{"2":{"2":3,"3":1,"38":1,"40":6,"55":1,"60":2,"62":1,"63":3,"65":1,"67":2,"70":1,"90":2,"97":1,"109":3,"145":1,"179":1,"237":1,"249":2}}],["ess",{"2":{"249":3}}],["essentially",{"2":{"48":1,"69":1,"136":1,"166":1,"245":1}}],["estimated",{"2":{"149":1}}],["estimate",{"2":{"149":1,"152":1}}],["estimation",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":1,"202":4}}],["established",{"2":{"81":1,"174":1}}],["especially",{"2":{"104":1,"108":1}}],["epyc",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["epochs",{"2":{"202":2,"245":3}}],["epoch",{"2":{"69":1,"187":353,"195":2,"217":2,"228":2,"245":11}}],["eps",{"2":{"19":8,"63":1}}],["epsilon=0",{"2":{"63":2}}],["epsilon=nothing",{"2":{"63":1}}],["epsilon=1f",{"2":{"55":4}}],["epsilon",{"2":{"19":9,"55":4,"63":9}}],["evolved",{"2":{"250":1}}],["evalpoly",{"2":{"241":2,"245":1}}],["eval=32",{"2":{"224":1}}],["eval",{"2":{"153":4,"224":5}}],["evaluate",{"2":{"60":1}}],["eventually",{"2":{"153":1}}],["even",{"2":{"3":1,"7":1,"8":1,"65":1,"82":1,"83":1,"84":1,"108":1,"110":1,"156":1,"171":1,"175":1,"196":1}}],["everystep",{"2":{"220":13}}],["everystep=false",{"2":{"215":1}}],["everything",{"2":{"83":2,"140":1}}],["every",{"2":{"50":1,"58":1,"113":1,"114":1,"160":1,"171":1}}],["ever",{"2":{"3":1,"83":1}}],["emp",{"2":{"145":2,"146":2}}],["empting",{"2":{"49":1}}],["empty",{"2":{"7":1,"8":2,"49":2,"51":9,"53":4,"54":5,"55":5,"56":1,"62":2,"63":1,"135":1,"179":1,"256":1}}],["embed",{"2":{"69":8,"78":8,"153":1}}],["embeddings",{"2":{"53":2}}],["embedding",{"2":{"53":2,"99":1,"153":1,"226":1}}],["either",{"2":{"49":4,"51":6,"63":1,"254":1}}],["eagerly",{"2":{"160":1}}],["easily",{"2":{"82":1,"145":1,"157":1}}],["easier",{"2":{"54":1,"62":1,"250":1}}],["easy",{"2":{"48":1,"71":1,"83":1,"108":1,"155":1}}],["eachindex",{"2":{"249":1}}],["eachslice",{"2":{"64":2,"184":1,"185":1}}],["each",{"2":{"5":1,"19":2,"28":1,"41":4,"48":15,"49":8,"51":12,"52":4,"53":2,"54":4,"55":2,"146":3,"171":2,"201":1,"249":1,"255":1}}],["ecosystem",{"2":{"44":1,"78":1,"101":1}}],["eccv",{"2":{"19":1}}],["edges",{"2":{"144":1}}],["edge",{"2":{"37":1,"83":1,"104":1,"105":1,"111":1,"153":1}}],["effort",{"2":{"106":1}}],["efficient",{"2":{"63":1,"65":1,"74":1,"100":1,"138":1,"145":1,"146":1,"175":2,"232":1}}],["efficiently",{"2":{"22":2,"63":1}}],["effectively",{"2":{"202":2}}],["effect",{"2":{"1":1,"56":1,"70":1}}],["european",{"2":{"19":1}}],["equilibrium",{"2":{"83":1}}],["equivalent",{"2":{"8":1,"46":1,"54":1,"63":2,"65":4,"69":1,"91":1,"171":1}}],["equations",{"2":{"74":1,"257":1}}],["equals",{"2":{"54":1}}],["equal",{"2":{"28":1,"41":1,"49":2,"51":3,"63":1}}],["equally",{"2":{"19":1,"64":1}}],["et",{"2":{"17":2,"28":2,"63":2,"252":1}}],["etc",{"2":{"3":2,"7":1,"20":1,"83":1,"135":1,"138":1,"139":1,"140":1,"142":1,"145":2,"174":1,"198":1}}],["elem",{"2":{"235":2,"237":2}}],["element",{"0":{"67":1},"2":{"17":2,"28":1,"41":1,"48":1,"52":6,"55":1,"63":1,"65":1,"67":10,"69":2,"153":4,"171":4,"176":1,"177":1,"179":1,"184":1,"188":1}}],["elementwise",{"2":{"15":1,"19":1,"54":2,"55":5}}],["elements",{"2":{"8":1,"28":9,"39":1,"49":2,"51":3,"53":1,"81":1,"171":6,"184":1}}],["elman",{"2":{"52":1}}],["eltypes",{"2":{"155":1,"165":1}}],["eltype",{"0":{"165":1},"2":{"19":9,"65":6,"66":3,"67":3,"115":1,"155":5,"165":2,"173":1}}],["elseif",{"2":{"132":1,"137":1,"254":1}}],["else",{"2":{"8":1,"17":1,"23":1,"44":1,"48":1,"50":3,"52":1,"54":1,"60":1,"62":1,"64":1,"65":1,"132":1,"254":3}}],["eg",{"2":{"8":1,"49":2,"51":3}}],["energy",{"2":{"249":2}}],["environment",{"2":{"180":2,"189":2,"197":1,"204":2,"221":2,"229":2,"238":2,"246":2,"251":1,"260":1}}],["engine",{"2":{"173":1}}],["enforce",{"2":{"173":1}}],["enough",{"2":{"139":1,"160":1,"202":1}}],["encountered",{"2":{"110":1}}],["encounter",{"2":{"105":1,"106":1,"144":1}}],["enhance",{"2":{"104":1}}],["entries",{"2":{"146":1}}],["entry",{"2":{"125":1}}],["entropy",{"2":{"63":3}}],["enter",{"2":{"76":1}}],["entire",{"2":{"50":2,"52":5,"55":1,"114":1,"202":1}}],["entirely",{"2":{"6":1,"49":2,"53":3,"218":1,"220":1}}],["enzymemlir",{"2":{"106":1}}],["enzyme",{"2":{"59":3,"62":1,"83":1,"88":2,"102":1,"103":4,"105":1,"106":10,"144":1,"153":1,"173":1,"175":1}}],["enable",{"2":{"157":1}}],["enables",{"2":{"18":1,"83":1,"247":1}}],["enabled",{"2":{"7":1,"40":2,"97":1,"247":1}}],["ensuring",{"2":{"17":1}}],["ensures",{"2":{"120":1}}],["ensure",{"2":{"2":1,"8":1,"28":1,"83":1,"105":1,"120":1,"149":1,"184":1,"244":1}}],["endpoints",{"2":{"254":2}}],["end",{"2":{"5":2,"11":1,"28":2,"39":2,"48":4,"49":2,"51":3,"69":14,"77":1,"78":7,"106":2,"107":4,"109":2,"110":4,"115":3,"116":1,"117":3,"126":1,"127":2,"128":2,"132":1,"135":4,"136":2,"137":3,"140":3,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"153":1,"160":2,"172":1,"174":2,"179":3,"180":3,"181":1,"183":3,"184":4,"185":3,"186":1,"187":4,"189":3,"192":1,"194":2,"195":3,"197":3,"200":2,"201":1,"202":4,"203":1,"204":3,"213":1,"214":7,"215":1,"216":2,"217":3,"218":3,"221":3,"224":2,"225":3,"226":1,"227":2,"228":4,"229":3,"233":3,"234":3,"235":1,"236":3,"237":2,"238":3,"241":2,"245":4,"246":3,"248":1,"249":4,"250":3,"251":3,"254":26,"255":2,"256":2,"257":2,"259":2,"260":3}}],["enumerate",{"2":{"5":2}}],["execution",{"2":{"174":1}}],["executables",{"2":{"106":1}}],["exhaustive",{"2":{"161":1}}],["existent",{"2":{"123":1}}],["exists",{"2":{"122":1,"214":1}}],["exist",{"2":{"69":2}}],["external",{"2":{"249":1}}],["extensively",{"2":{"83":1,"104":2,"105":1,"175":1}}],["extensive",{"2":{"82":1,"83":1}}],["extensible",{"2":{"82":1}}],["extensions",{"0":{"60":1}}],["extending",{"2":{"173":1,"184":1}}],["extended",{"2":{"38":1,"39":1,"40":1,"48":6,"49":2,"51":3,"52":2,"54":1,"55":4,"56":1,"63":2,"67":1,"69":1}}],["extend",{"2":{"7":1,"38":1,"83":1}}],["ext",{"2":{"116":2}}],["extrema",{"2":{"235":2}}],["extremely",{"2":{"40":1,"62":1,"130":1,"140":1}}],["extract",{"2":{"249":2,"250":1}}],["extra",{"2":{"28":1}}],["exact",{"2":{"28":1,"136":1,"187":1}}],["exactly",{"2":{"16":1,"56":1,"64":1,"91":1,"135":1,"136":1,"142":1}}],["examples",{"0":{"169":1},"2":{"5":1,"28":2,"63":2,"69":4,"84":1,"169":1,"209":1}}],["example",{"0":{"114":1},"1":{"115":1,"116":1,"117":1,"118":1},"2":{"2":1,"3":2,"4":1,"8":1,"19":1,"23":1,"28":1,"38":3,"39":2,"41":3,"44":1,"46":1,"48":3,"54":5,"55":3,"58":1,"59":1,"62":1,"63":14,"68":1,"84":1,"91":1,"94":1,"108":1,"119":2,"126":1,"127":1,"137":1,"138":1,"139":1,"140":1,"146":4,"147":1,"148":1,"155":1,"166":1,"176":1,"177":1,"256":1}}],["excellent",{"2":{"190":1}}],["except",{"2":{"7":1,"52":2,"54":1,"156":1}}],["exceptions",{"2":{"28":1}}],["exception",{"2":{"3":2,"60":1}}],["exciting",{"2":{"85":1}}],["exclusively",{"2":{"39":1,"57":1}}],["excluding",{"2":{"38":1,"46":1,"54":1,"55":1}}],["exclude",{"2":{"10":2,"87":1}}],["exclude=internal",{"2":{"10":1}}],["exp",{"2":{"235":1}}],["exposing",{"2":{"220":1}}],["exported",{"2":{"3":1,"6":1,"37":1,"172":1}}],["exploiting",{"2":{"160":1}}],["explore",{"2":{"144":2}}],["explicit",{"2":{"74":1,"83":1,"168":2,"247":1}}],["explicitly",{"2":{"7":1,"69":2}}],["express",{"2":{"249":1}}],["expressed",{"2":{"107":1}}],["expression",{"2":{"58":1,"69":1}}],["expr",{"2":{"60":4,"69":1}}],["expected",{"2":{"48":1,"49":2,"53":1,"63":4,"83":1,"110":1,"123":1,"174":1}}],["expect",{"2":{"37":1,"83":1,"106":1,"123":1,"153":1,"247":1}}],["expects",{"2":{"23":1,"46":1,"49":1,"51":3,"56":1,"58":1,"137":1}}],["experimental",{"0":{"37":1},"1":{"38":1,"39":1,"40":1,"41":1},"2":{"2":1,"4":2,"7":1,"37":2,"38":10,"39":2,"40":6,"41":2,"97":5,"98":5,"106":2,"108":1,"109":2,"110":2,"111":1,"124":1,"125":1,"126":3,"127":2,"128":2,"129":1,"130":1,"153":2,"155":1}}],["e",{"2":{"2":1,"11":1,"19":1,"28":1,"38":1,"44":1,"46":1,"49":2,"50":2,"51":9,"55":3,"63":3,"67":1,"68":1,"90":2,"106":1,"109":4,"110":4,"116":2,"120":1,"138":1,"149":1,"254":4,"255":6,"256":3}}],["gw",{"2":{"255":1}}],["gc",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["gnu",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["guess",{"2":{"258":1}}],["guide",{"0":{"120":1,"121":1},"1":{"122":1,"123":1},"2":{"121":1,"135":1}}],["guarantee",{"2":{"98":1,"166":1}}],["guarantees",{"2":{"37":1}}],["guaranteed",{"2":{"16":1,"18":1,"145":2}}],["gdev",{"2":{"95":1,"131":1,"132":4,"153":2,"160":6,"199":1,"201":3,"202":3,"203":1,"231":1,"236":3}}],["gs",{"2":{"69":2,"77":3,"195":1}}],["gif",{"2":{"237":1,"250":2}}],["gib",{"2":{"180":2,"189":2,"204":2,"221":2,"229":4,"238":4,"246":2}}],["gigantic",{"2":{"108":1}}],["give",{"2":{"183":1,"258":2}}],["gives",{"2":{"64":1,"69":1,"135":1}}],["given",{"2":{"2":1,"4":2,"19":1,"20":1,"28":6,"29":24,"32":6,"33":2,"34":5,"44":1,"48":2,"53":3,"55":5,"63":3,"82":1,"83":1,"202":1}}],["github",{"2":{"52":1,"71":2,"84":2,"111":1,"144":1,"153":1}}],["g=tanh",{"2":{"52":1}}],["globals",{"2":{"256":1}}],["globally",{"2":{"247":1}}],["globallppool",{"2":{"51":1,"100":1}}],["globalmeanpool",{"2":{"51":1}}],["globalmaxpool",{"2":{"51":1}}],["global",{"2":{"51":3,"58":1,"153":1,"155":1,"201":1,"247":1}}],["glorot",{"2":{"28":4,"135":2}}],["gaussian",{"2":{"249":1}}],["gaussadjoint",{"2":{"217":1}}],["gamma=0",{"2":{"63":1}}],["gamma",{"2":{"63":2}}],["gated",{"2":{"52":1}}],["ganguli",{"2":{"28":1}}],["gain=1",{"2":{"28":1,"168":2}}],["gain",{"2":{"28":15,"49":4,"53":2,"99":2}}],["green",{"2":{"245":1}}],["great",{"2":{"198":1}}],["grid",{"2":{"235":5,"237":8}}],["ground",{"2":{"179":3}}],["group",{"2":{"19":3,"55":1,"249":1}}],["groups=1",{"2":{"49":2}}],["groups",{"2":{"19":4,"49":8,"55":4}}],["groupnorm",{"2":{"19":1,"55":7,"142":1}}],["gravitational",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["gravitate",{"2":{"65":1}}],["graph",{"2":{"248":1,"250":1}}],["grads",{"2":{"62":11,"107":1,"179":1}}],["gradient|jacobian",{"2":{"144":2}}],["gradient",{"0":{"59":1,"147":1},"2":{"22":2,"57":1,"59":1,"62":1,"106":8,"110":4,"139":2,"140":2,"145":6,"146":3,"147":6,"148":6,"152":3,"176":9,"179":1,"198":1,"234":3}}],["gradients",{"0":{"176":1},"2":{"22":2,"35":1,"48":1,"59":10,"62":29,"63":2,"64":1,"77":6,"122":1,"144":1,"145":1,"152":1,"176":1,"195":1,"232":1}}],["grucell",{"2":{"52":6,"99":1}}],["gru",{"2":{"7":1,"52":1}}],["günter",{"2":{"17":1}}],["gt",{"2":{"5":1,"11":1,"28":1,"49":2,"51":3,"54":2,"55":2,"56":3,"63":1,"64":1,"69":2,"79":1,"90":2,"95":1,"131":1,"136":2,"144":12,"184":1}}],["geared",{"2":{"198":1}}],["gelu",{"2":{"106":6,"107":2,"155":1}}],["geoffrey",{"2":{"19":1}}],["genericlossfunction",{"2":{"63":2}}],["genericbroadcastop",{"2":{"20":1}}],["generic",{"2":{"13":1,"16":2,"18":2,"78":1,"97":1,"106":1,"110":1,"131":1,"150":1,"151":1,"152":1,"153":1,"183":1,"185":1,"186":1,"192":1,"194":1,"195":1,"199":1,"213":1,"214":2,"215":1,"216":1,"224":1,"225":1,"226":1,"227":1,"231":1,"234":2,"241":1,"249":2,"250":1,"254":5,"256":1,"257":2}}],["generating",{"0":{"248":1},"2":{"56":1}}],["generation",{"2":{"52":3}}],["generator=lux",{"2":{"225":1}}],["generator",{"2":{"17":4,"28":3,"50":3,"135":1,"170":1,"174":2,"225":4,"226":2,"241":1}}],["generates",{"2":{"248":1}}],["generated",{"2":{"17":1,"28":1,"50":1,"69":1,"144":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"239":1,"246":1,"250":1,"251":1,"260":1}}],["generate",{"0":{"200":1,"235":1},"2":{"3":1,"9":1,"10":1,"52":1,"69":1,"138":1,"168":2,"179":2,"183":1,"225":1,"235":1,"241":4,"248":2}}],["generalization",{"2":{"95":1}}],["generalized",{"2":{"5":1}}],["general",{"0":{"8":1},"2":{"48":1,"71":1,"79":1,"154":1,"256":1}}],["generally",{"2":{"5":1}}],["getaxes",{"2":{"225":1}}],["getindex",{"2":{"201":1}}],["getproperty",{"2":{"64":3,"137":2,"220":3}}],["getting",{"0":{"75":1},"1":{"76":1,"77":1,"78":1,"79":1,"80":1},"2":{"8":1}}],["getfield",{"2":{"7":1,"115":3,"137":3}}],["get",{"0":{"84":1},"2":{"3":5,"13":1,"14":1,"20":1,"31":2,"32":2,"33":2,"64":1,"77":1,"120":2,"137":1,"139":1,"149":2,"153":2,"165":1,"169":1,"171":2,"174":2,"183":3,"184":1,"187":2,"210":1,"249":2,"254":1}}],["g",{"2":{"2":1,"48":4,"50":2,"55":4,"106":1,"254":1}}],["goal",{"2":{"248":1}}],["goodies",{"2":{"177":1}}],["good",{"2":{"133":1,"145":1,"202":1,"249":1,"258":2}}],["goodfellow",{"2":{"54":1}}],["going",{"2":{"44":1,"147":1}}],["go",{"2":{"2":1,"84":2,"110":1,"114":1,"125":1,"136":1,"137":1,"154":1,"181":1,"230":1}}],["gpuci",{"2":{"109":1,"220":3,"236":4}}],["gpuarray",{"2":{"64":1}}],["gpuarrayscore",{"2":{"156":2}}],["gpuarrays",{"2":{"26":7,"64":1}}],["gpubroadcastop",{"2":{"20":1}}],["gpusintel",{"2":{"72":2}}],["gpusmetal",{"2":{"72":2}}],["gpusamd",{"2":{"72":2}}],["gpus",{"2":{"13":1,"32":1,"72":2,"130":1,"155":1,"156":1,"160":1}}],["gpu",{"0":{"4":1,"72":1,"80":1,"130":1,"153":1,"156":1,"163":1,"164":1},"1":{"131":1,"132":1},"2":{"1":8,"2":7,"3":10,"5":2,"16":1,"20":1,"51":3,"59":1,"72":4,"73":1,"77":2,"78":2,"80":1,"98":2,"102":1,"103":3,"106":2,"115":1,"123":4,"130":4,"131":9,"132":5,"140":1,"153":15,"160":4,"164":3,"168":1,"172":3,"187":1,"199":1,"201":3,"217":3,"220":1,"228":1,"231":1,"244":3,"245":2}}],["nparameters",{"2":{"249":3}}],["nvml",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["nvidia",{"2":{"3":1,"5":1,"72":2,"180":4,"189":4,"197":2,"204":4,"221":4,"229":3,"238":3,"246":4}}],["ndrange=length",{"2":{"153":2}}],["ndims",{"2":{"11":1,"23":4,"49":4,"51":9,"54":4,"55":2,"56":2,"115":2,"150":2,"151":2}}],["nicely",{"2":{"145":1}}],["nilarray",{"2":{"90":1}}],["nitish",{"2":{"17":1}}],["nlayers",{"2":{"69":2,"78":2}}],["n=tanh⁡",{"2":{"52":1}}],["nn",{"2":{"49":2,"53":1,"153":7,"249":12,"250":11,"256":14,"257":1,"259":9}}],["nnlib",{"2":{"13":2,"14":2,"49":1,"56":6,"97":3,"139":2,"140":2,"158":3,"245":1}}],["ntuple",{"2":{"48":1,"49":2,"55":2,"115":1,"220":13,"236":1}}],["nabla",{"2":{"176":1}}],["native",{"2":{"143":1,"198":1}}],["navab",{"2":{"63":1}}],["nassir",{"2":{"63":1}}],["naming",{"2":{"48":9,"54":2}}],["name>=",{"2":{"161":3}}],["namefreezing",{"2":{"127":1}}],["name=",{"2":{"48":1,"69":1}}],["name=nothing",{"2":{"48":11,"69":1}}],["names",{"2":{"38":1,"55":1,"69":1}}],["named",{"2":{"19":2,"38":2,"63":1,"137":1,"215":2,"217":1,"249":1}}],["namedtuples",{"2":{"137":1}}],["namedtuple",{"2":{"7":4,"8":1,"10":3,"38":5,"39":1,"48":10,"49":2,"51":9,"52":2,"53":4,"54":9,"55":5,"56":1,"62":1,"64":1,"65":1,"69":3,"77":24,"78":1,"90":1,"106":7,"109":3,"110":6,"113":3,"115":3,"116":7,"117":3,"126":3,"129":4,"135":4,"136":3,"138":1,"153":2,"184":2,"220":147,"244":2,"245":1,"249":1,"256":6}}],["name",{"0":{"127":1},"2":{"7":2,"8":5,"38":1,"39":1,"54":1,"69":3,"79":1,"97":1,"115":5,"127":5,"128":6,"131":1,"137":1,"228":4}}],["nans",{"0":{"110":1},"2":{"40":3,"110":7,"111":1}}],["nan",{"2":{"40":5,"110":31,"236":1}}],["nccl",{"2":{"31":1,"32":4,"122":1}}],["ncclbackend",{"2":{"31":2,"32":2,"120":4}}],["nₙ",{"2":{"23":2}}],["n₂",{"2":{"23":2}}],["n₁",{"2":{"23":2}}],["n",{"2":{"19":5,"48":23,"49":27,"51":54,"53":2,"54":11,"55":11,"56":8,"64":2,"69":5,"78":5,"107":1,"179":3,"187":2,"192":3,"195":1,"202":1,"213":3,"214":5,"217":1,"218":5,"224":12,"228":2,"236":1,"245":1,"248":2,"249":3,"250":2,"257":1}}],["numer",{"2":{"255":3,"256":3}}],["numeric",{"2":{"28":2}}],["numerically",{"2":{"186":1}}],["numerical",{"2":{"19":4,"55":4,"63":1,"155":1,"249":1}}],["num",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"250":2,"251":1,"260":1}}],["number=0",{"2":{"28":2}}],["number=1",{"2":{"28":2}}],["numbers",{"2":{"28":6,"29":12,"168":1,"171":3}}],["number",{"2":{"9":1,"10":1,"17":4,"19":1,"28":8,"33":1,"48":3,"49":10,"50":3,"51":3,"53":7,"54":2,"55":3,"56":3,"62":1,"63":4,"64":3,"65":1,"69":3,"135":1,"170":1,"171":1,"174":2,"175":1,"241":1,"248":1,"249":4}}],["nepochs",{"2":{"179":3,"195":3,"217":3,"228":3}}],["nest",{"2":{"145":1}}],["nested",{"0":{"24":1,"144":1,"162":1},"1":{"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1},"2":{"3":1,"24":2,"41":2,"65":5,"68":2,"102":1,"144":1,"147":2,"162":3,"230":1,"232":1,"237":1}}],["next",{"2":{"120":1,"135":1,"249":1,"250":1,"254":1,"256":1,"257":1}}],["negative",{"2":{"63":1,"254":1}}],["net",{"2":{"63":1,"234":11,"249":1,"256":1,"259":2}}],["network",{"0":{"242":1,"247":1,"249":1,"256":1,"257":1,"258":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"19":1,"28":1,"78":1,"81":2,"83":1,"114":1,"145":1,"146":2,"153":2,"170":1,"181":2,"225":5,"226":3,"242":1,"248":1,"249":6,"250":4,"256":6}}],["networks",{"0":{"153":1,"233":1},"2":{"17":3,"28":3,"46":1,"54":1,"63":2,"74":1,"83":3,"152":1,"153":1,"190":1,"198":2,"233":2,"249":1}}],["nearest",{"2":{"56":7}}],["necessary",{"2":{"51":3,"135":1}}],["never",{"2":{"37":1,"44":1,"54":1,"256":1}}],["neuralpde",{"2":{"230":2}}],["neuralode",{"2":{"214":7,"217":4,"220":5}}],["neuralodecompact",{"2":{"214":2,"217":1,"220":1}}],["neural",{"0":{"153":1,"211":1,"214":1,"215":1,"219":1,"233":1,"242":1,"247":1,"249":1,"252":1,"256":1,"257":1,"258":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"248":1,"249":1,"250":1,"251":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"17":4,"28":4,"46":1,"63":1,"68":1,"74":2,"78":1,"81":2,"83":4,"114":3,"145":3,"146":2,"152":1,"153":3,"170":1,"181":2,"190":1,"198":5,"211":2,"214":1,"215":1,"242":1,"248":1,"249":7,"250":2,"256":7,"257":1,"259":2}}],["needed",{"2":{"13":2,"22":2,"23":1,"62":1,"97":1,"122":1,"140":1,"144":1,"168":2}}],["need",{"2":{"6":1,"8":1,"38":1,"41":1,"44":1,"54":1,"69":2,"77":1,"85":1,"97":1,"106":3,"107":1,"109":1,"113":1,"120":1,"121":2,"135":3,"136":2,"140":1,"144":1,"153":1,"166":1,"169":1,"171":1,"183":1,"184":1,"193":2,"202":1,"214":1,"226":1,"234":1,"244":1,"254":1,"256":1}}],["needs",{"2":{"3":1,"7":1,"10":1,"13":2,"49":1,"50":1,"115":1,"121":1,"148":1,"164":1}}],["newtonian",{"2":{"254":2,"256":1,"258":1}}],["new",{"0":{"88":1,"91":1,"95":1,"100":1,"219":1},"2":{"1":1,"39":9,"41":5,"50":1,"52":4,"54":5,"62":1,"69":1,"85":1,"87":1,"98":1,"106":1,"113":1,"121":1,"130":1,"133":1,"135":1,"136":1,"144":1,"153":1,"225":2,"249":3}}],["nom",{"2":{"249":1}}],["noisy",{"2":{"183":2}}],["noise",{"2":{"179":1}}],["now",{"2":{"88":1,"95":1,"98":1,"99":4,"100":2,"106":4,"107":1,"109":2,"110":2,"121":1,"123":3,"135":1,"136":1,"140":1,"145":1,"146":2,"147":1,"148":1,"152":2,"153":2,"155":1,"170":2,"175":1,"179":1,"184":2,"186":1,"193":1,"196":1,"202":1,"245":1,"247":1,"249":2,"250":1,"254":1,"256":2,"257":1,"259":1}}],["norm",{"2":{"55":1,"145":7,"146":7,"147":6,"148":6,"152":12}}],["normally",{"2":{"106":1}}],["normal",{"2":{"26":4,"28":12,"29":6,"168":6,"249":1,"256":3}}],["normalises",{"2":{"19":2,"55":2}}],["normalize",{"2":{"55":2}}],["normalized",{"2":{"19":8,"55":7,"63":3,"237":2}}],["normalization",{"0":{"19":1,"55":1},"2":{"19":12,"23":1,"28":1,"55":11,"146":1}}],["normalizing",{"2":{"17":1}}],["nooplayer",{"2":{"48":6,"50":3,"54":3}}],["nonlinear",{"2":{"28":1}}],["non",{"0":{"118":1},"2":{"16":1,"18":1,"28":2,"32":2,"38":1,"40":1,"53":1,"62":2,"63":1,"69":9,"90":1,"93":1,"108":1,"123":1,"140":2,"210":1,"214":2,"254":1}}],["nonetheless",{"2":{"2":1}}],["none",{"2":{"2":1,"37":1,"38":1,"40":1,"67":1,"110":1}}],["node",{"2":{"3":1}}],["notice",{"2":{"135":1,"136":1,"146":2,"217":1}}],["notion",{"2":{"65":1}}],["notangent",{"2":{"59":1,"110":1}}],["not",{"2":{"2":1,"3":4,"4":2,"6":1,"7":1,"8":1,"15":1,"16":1,"17":1,"19":3,"23":1,"28":2,"31":2,"40":1,"41":1,"46":3,"48":1,"52":11,"54":1,"55":1,"57":1,"60":1,"62":1,"64":1,"67":1,"68":1,"69":1,"77":1,"83":1,"90":2,"97":1,"101":1,"104":1,"105":2,"106":1,"109":1,"110":1,"120":1,"123":1,"131":1,"133":1,"140":3,"144":1,"145":5,"149":1,"152":1,"153":1,"160":1,"167":1,"169":1,"196":2,"202":1,"210":1,"214":1,"217":2,"219":1,"220":2,"242":1,"249":1,"250":1,"254":1}}],["notes",{"2":{"16":1,"18":1,"59":1,"177":1,"211":1}}],["note",{"2":{"2":1,"3":4,"13":2,"28":1,"30":1,"38":1,"54":1,"62":1,"63":1,"65":2,"68":1,"69":1,"74":1,"105":1,"106":1,"107":1,"109":1,"120":1,"135":1,"136":1,"143":1,"144":1,"147":1,"156":1,"173":1,"176":1,"188":1,"214":1,"220":1,"247":1}}],["nothing",{"2":{"2":4,"3":5,"4":1,"8":2,"15":1,"16":1,"17":2,"18":1,"19":15,"31":3,"32":1,"38":5,"48":4,"49":4,"50":1,"52":11,"53":4,"54":10,"55":5,"59":1,"63":12,"64":1,"65":2,"68":1,"77":27,"78":1,"106":9,"110":1,"116":4,"145":2,"147":2,"148":2,"200":1,"201":4,"202":2,"220":117,"245":6,"249":1,"256":1}}],["no",{"2":{"2":2,"4":2,"5":1,"37":1,"38":3,"46":1,"54":1,"58":1,"62":1,"63":2,"67":1,"70":1,"77":1,"83":1,"92":1,"95":1,"97":1,"109":1,"110":1,"116":1,"122":3,"131":2,"132":1,"135":1}}],["u=u",{"2":{"234":1}}],["u0=res",{"2":{"202":1}}],["u0",{"2":{"200":2,"202":2,"203":1,"255":2,"256":2,"257":1,"259":2}}],["utc",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["utility",{"0":{"216":1,"227":1,"254":1}}],["utilities",{"0":{"61":1},"1":{"62":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"70":1},"2":{"57":1,"140":1}}],["utils",{"0":{"30":1},"1":{"31":1,"32":1,"33":1,"34":1,"35":1,"36":1},"2":{"38":2,"65":1,"145":4}}],["url",{"2":{"74":1}}],["url=",{"2":{"71":1}}],["u",{"2":{"22":7,"200":4,"202":11,"203":6,"214":7,"218":2,"233":4,"234":11,"236":4,"237":7,"255":3,"256":4,"258":1,"259":2}}],["ulyanov",{"2":{"19":1,"55":1}}],["upto",{"2":{"232":1}}],["up",{"0":{"257":1},"2":{"64":1,"149":1,"152":1,"210":2,"211":1}}],["upsampled",{"2":{"56":1}}],["upsample",{"2":{"56":5,"99":1}}],["upsampling",{"0":{"56":1},"2":{"56":2}}],["upscaling",{"2":{"56":1}}],["upscale",{"2":{"56":2}}],["updating",{"0":{"66":1,"85":1},"1":{"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1},"2":{"19":2,"35":1,"76":1,"179":1}}],["updates",{"2":{"41":1,"55":1,"62":3,"93":1,"120":1}}],["updated",{"2":{"15":1,"17":2,"19":2,"41":1,"48":6,"50":3,"52":7,"54":1,"55":1,"62":5,"63":1,"78":1,"82":1,"83":1,"99":1,"123":1,"135":1,"184":1,"210":1,"244":1}}],["update",{"2":{"10":2,"15":1,"17":2,"50":3,"52":5,"55":3,"62":2,"66":1,"69":1,"76":2,"85":1,"173":1}}],["upon",{"2":{"5":1}}],["untrained",{"2":{"256":2,"259":1}}],["until",{"2":{"50":1}}],["uncertain",{"2":{"250":1}}],["unchanged",{"2":{"48":1}}],["unable",{"2":{"173":1}}],["unnecessary",{"2":{"146":1}}],["unnormalized",{"2":{"55":1}}],["un",{"2":{"97":1}}],["unexpected",{"2":{"83":1}}],["unwrapped",{"2":{"65":1}}],["unwrap",{"2":{"65":2}}],["unwraps",{"2":{"38":1}}],["unreleased",{"2":{"71":1}}],["unreasonably",{"2":{"48":1}}],["unrolls",{"2":{"48":1}}],["unroll",{"2":{"13":1}}],["undone",{"2":{"41":1}}],["undef",{"2":{"174":1}}],["undefined",{"2":{"8":1,"52":1}}],["underlying",{"2":{"214":1}}],["under",{"2":{"140":1}}],["understand",{"2":{"69":1,"133":1,"184":1,"211":1,"214":1}}],["understanding",{"2":{"28":2,"225":1}}],["undesirable",{"2":{"11":1}}],["unfreezes",{"2":{"38":1}}],["unfreeze",{"2":{"38":2}}],["unlike",{"2":{"29":1,"69":1}}],["unless",{"2":{"2":1,"40":1,"225":1}}],["unified",{"2":{"198":1}}],["uniformly",{"2":{"120":1}}],["uniform",{"2":{"28":6,"29":6,"49":5,"52":6,"53":4,"77":1,"135":3}}],["uninitiated",{"0":{"170":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1}}],["universal",{"2":{"54":1}}],["unity",{"2":{"254":1}}],["unitrange",{"2":{"173":1}}],["unit",{"2":{"52":1}}],["union",{"2":{"2":3,"8":1,"17":1,"28":2,"40":2,"46":1,"52":2,"55":1,"63":3}}],["unpack",{"2":{"8":1}}],["unsupported",{"2":{"8":1,"26":1}}],["unsafe",{"2":{"5":1}}],["unknown",{"2":{"3":2}}],["unknowndevice",{"2":{"3":2}}],["usr",{"2":{"180":2,"189":2,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2}}],["usacases",{"2":{"103":2}}],["usage",{"2":{"38":1,"84":1}}],["usually",{"2":{"56":1,"63":2,"69":1,"171":1,"202":1}}],["usual",{"2":{"55":3,"148":1,"153":1}}],["us",{"2":{"22":2,"37":1,"108":1,"109":1,"110":6,"136":2,"137":3,"140":1,"144":1,"170":1,"174":1,"177":1,"178":1,"179":3,"183":1,"219":1,"256":1,"257":1,"259":2}}],["usecases",{"2":{"65":1,"68":1,"153":1,"230":1}}],["uses",{"2":{"16":1,"18":2,"44":1,"55":1,"65":4,"66":1,"68":1,"90":1,"144":1,"147":1,"155":1,"158":2,"161":1,"174":1,"198":1,"254":2,"255":1,"256":1,"258":1}}],["userbase",{"2":{"44":1}}],["user",{"2":{"7":2,"8":2,"13":2,"48":1,"68":1,"81":1,"113":1,"140":1,"142":2,"147":1}}],["users",{"2":{"6":1,"7":2,"15":1,"31":2,"32":1,"38":1,"62":1,"65":1,"81":1,"83":1,"90":1,"93":1,"155":1,"173":1,"211":2,"214":1,"220":1}}],["useful",{"2":{"3":1,"20":1,"28":3,"40":2,"63":1,"64":1,"67":1,"69":2,"74":1,"108":1,"109":1}}],["use",{"0":{"83":1,"143":1},"2":{"2":1,"4":2,"5":1,"13":3,"15":1,"17":1,"18":2,"20":1,"22":2,"23":1,"28":1,"31":4,"37":1,"38":2,"40":1,"49":11,"52":18,"53":16,"55":3,"57":2,"62":4,"69":1,"71":1,"74":1,"77":2,"78":1,"83":1,"84":2,"93":1,"97":2,"98":4,"99":1,"100":1,"103":3,"105":1,"106":3,"107":2,"108":1,"109":1,"120":2,"124":1,"126":1,"131":1,"143":1,"144":3,"145":1,"146":1,"147":2,"149":2,"153":4,"155":2,"158":1,"159":1,"163":1,"164":2,"167":2,"169":1,"170":1,"171":1,"174":1,"175":1,"176":1,"177":2,"179":3,"183":2,"184":3,"186":2,"190":2,"192":1,"211":1,"213":1,"214":1,"215":2,"217":2,"230":1,"234":1,"236":4,"243":1,"244":1,"245":1,"247":1,"248":1,"249":5,"250":1,"254":1,"256":5}}],["used",{"2":{"2":2,"3":6,"8":2,"17":3,"28":1,"36":1,"40":2,"41":2,"44":1,"46":2,"49":3,"50":3,"52":1,"53":1,"54":1,"55":7,"56":1,"62":2,"63":5,"65":1,"68":1,"69":11,"90":1,"94":1,"97":1,"98":1,"100":1,"106":1,"123":1,"130":1,"131":2,"144":1,"145":2,"155":1,"166":1,"167":1,"169":1,"171":1,"218":1,"252":1}}],["using",{"0":{"45":1,"58":1,"106":1,"107":1,"118":1,"146":1,"150":1,"151":1,"152":1,"185":1,"198":1,"211":1,"218":1,"239":1},"1":{"46":1,"107":1,"199":1,"200":1,"201":1,"202":1,"203":1,"204":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"2":1,"3":1,"4":2,"5":2,"7":2,"20":3,"22":2,"28":2,"34":1,"35":1,"36":1,"38":1,"39":1,"44":3,"46":4,"48":11,"52":6,"53":1,"54":2,"55":1,"58":2,"59":1,"62":7,"63":4,"65":1,"69":9,"71":2,"76":1,"77":3,"78":4,"81":1,"82":1,"83":2,"90":1,"95":1,"100":1,"106":8,"107":2,"109":1,"119":1,"123":1,"130":3,"132":2,"133":2,"137":2,"139":2,"142":1,"143":1,"144":4,"145":6,"146":1,"147":1,"148":2,"149":5,"152":14,"153":3,"155":1,"156":1,"163":1,"170":1,"171":2,"174":1,"177":2,"178":1,"179":1,"180":1,"181":2,"183":1,"185":1,"188":1,"189":1,"190":1,"191":2,"196":1,"197":1,"198":1,"199":1,"202":1,"204":1,"212":2,"214":1,"218":1,"220":2,"221":1,"229":1,"230":3,"231":1,"232":1,"233":1,"234":1,"238":1,"240":1,"242":1,"246":1,"247":2,"249":1,"250":1,"251":1,"253":1,"255":1,"256":1,"260":1}}],["w3",{"2":{"69":3,"78":3}}],["w3=dense",{"2":{"69":1,"78":1}}],["w2",{"2":{"69":3,"78":3}}],["w2=",{"2":{"69":1,"78":1}}],["w1",{"2":{"69":3,"78":3}}],["w1=dense",{"2":{"69":1,"78":1}}],["w=w",{"2":{"234":1}}],["w=",{"2":{"69":1}}],["w=rand",{"2":{"69":1}}],["w=ones",{"2":{"69":3}}],["w=gv∥v∥weight",{"2":{"55":1}}],["w",{"2":{"51":6,"53":1,"56":4,"69":21,"78":4,"137":2,"158":1,"171":1,"179":5,"192":1,"213":1,"233":4,"234":10}}],["write",{"2":{"171":2,"173":1,"179":1,"225":1}}],["writing",{"0":{"117":1},"2":{"97":1,"118":1,"133":1}}],["written",{"2":{"44":1}}],["wrote",{"0":{"81":1},"1":{"82":1,"83":1},"2":{"210":1}}],["wrapping",{"2":{"90":1}}],["wrappers",{"0":{"22":1}}],["wrapper",{"2":{"14":1,"22":2,"40":1,"52":1,"68":1,"136":2,"148":1,"233":1,"245":1}}],["wrappedlayer",{"2":{"50":1}}],["wrappedfunction",{"2":{"49":1,"54":2,"110":3,"220":9,"256":1}}],["wrapped",{"2":{"7":3,"40":1,"48":8,"54":2,"136":1}}],["wrap",{"2":{"35":1,"108":1,"120":1,"136":1,"201":1}}],["wraps",{"2":{"32":2,"46":1,"52":2,"54":1}}],["wrt",{"2":{"22":2,"59":1,"62":3,"63":2,"64":1,"121":1,"148":2,"234":1}}],["wu",{"2":{"19":1}}],["wide",{"2":{"175":1}}],["wider",{"2":{"83":1,"88":1,"92":1,"177":1}}],["width",{"2":{"49":1}}],["wio",{"2":{"52":1}}],["wio×x+who×hprev+bo",{"2":{"52":1}}],["wig",{"2":{"52":1}}],["wig×x+whg×hprev+bg",{"2":{"52":1}}],["wif",{"2":{"52":1}}],["wif×x+whf×hprev+bf",{"2":{"52":1}}],["wii",{"2":{"52":1}}],["wii×x+whi×hprev+bi",{"2":{"52":1}}],["wiz",{"2":{"52":1}}],["wiz×x+biz+whz×hprev+bhz",{"2":{"52":1}}],["wir",{"2":{"52":1}}],["wir×x+bir+whr×hprev+bhr",{"2":{"52":1}}],["win",{"2":{"52":1}}],["win×x+bin+r⋅",{"2":{"52":1}}],["window",{"2":{"51":18}}],["will",{"2":{"3":2,"5":1,"11":1,"19":1,"23":4,"28":3,"34":2,"37":1,"40":2,"41":3,"44":4,"46":1,"48":5,"50":2,"52":3,"54":1,"55":3,"58":2,"59":1,"60":4,"62":1,"65":4,"67":4,"68":1,"69":8,"71":1,"77":1,"78":1,"90":1,"106":5,"108":1,"109":1,"110":2,"114":1,"125":1,"130":1,"133":1,"136":1,"139":1,"140":2,"144":2,"145":3,"147":1,"149":2,"150":1,"153":8,"155":2,"157":1,"167":1,"169":1,"171":1,"172":1,"173":1,"174":2,"175":4,"176":1,"177":1,"179":2,"181":2,"183":3,"184":6,"186":1,"190":2,"196":2,"198":2,"201":4,"202":1,"210":1,"214":1,"217":1,"230":3,"232":1,"233":1,"234":2,"235":3,"239":1,"243":1,"244":1,"245":2,"247":2,"249":4,"256":1}}],["withgradient",{"2":{"69":1}}],["within",{"2":{"17":2,"19":2,"69":1,"145":2}}],["without",{"2":{"2":1,"6":1,"69":1,"83":2,"90":2,"106":1,"107":1,"133":1,"145":2,"173":1,"184":1,"202":2}}],["with",{"0":{"190":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1},"2":{"1":1,"2":1,"3":1,"5":2,"6":1,"7":2,"8":1,"10":2,"11":1,"13":7,"15":1,"16":2,"18":1,"20":1,"23":2,"28":5,"32":2,"35":1,"36":2,"38":5,"39":2,"40":3,"41":1,"44":1,"46":1,"48":14,"49":11,"50":6,"51":12,"52":10,"53":12,"54":5,"55":3,"56":5,"58":2,"59":1,"62":2,"63":6,"64":2,"65":2,"66":3,"67":1,"69":3,"78":3,"81":1,"83":7,"87":1,"88":1,"90":2,"97":3,"99":1,"100":1,"101":2,"103":1,"104":2,"105":2,"106":2,"107":1,"108":3,"109":1,"110":1,"118":2,"120":2,"122":2,"123":1,"124":1,"131":2,"133":1,"135":1,"136":1,"137":6,"139":1,"144":3,"145":1,"146":4,"147":1,"150":1,"151":1,"152":1,"153":5,"155":1,"157":1,"158":6,"163":1,"167":2,"168":2,"169":2,"170":1,"171":5,"172":2,"173":1,"174":2,"175":1,"177":2,"179":4,"181":1,"183":1,"185":1,"186":1,"187":1,"192":1,"194":1,"195":1,"198":2,"199":1,"202":1,"210":1,"213":1,"214":3,"215":1,"216":1,"224":1,"225":2,"226":1,"227":1,"231":1,"232":1,"234":2,"241":1,"247":2,"248":1,"249":6,"250":2,"254":5,"256":2,"257":2}}],["what",{"2":{"109":1,"110":1,"135":1,"137":1,"142":1,"148":1,"165":1,"171":2}}],["whatever",{"2":{"54":1,"62":1}}],["why",{"0":{"81":1,"83":1},"1":{"82":1,"83":1},"2":{"135":1}}],["whole",{"2":{"55":2}}],["whose",{"2":{"53":1,"55":1}}],["who",{"2":{"52":1}}],["whg",{"2":{"52":1}}],["whf",{"2":{"52":1}}],["whn",{"2":{"52":1}}],["whn×hprev+bhn",{"2":{"52":1}}],["whz",{"2":{"52":1}}],["whr",{"2":{"52":1}}],["whcn",{"2":{"49":1,"50":2,"55":3}}],["whi",{"2":{"52":1}}],["while",{"2":{"3":1,"39":1,"46":1,"48":1,"56":1,"68":1,"83":1,"107":1,"128":1,"133":1,"135":1,"155":1,"177":1}}],["which",{"0":{"113":1},"2":{"2":1,"3":1,"7":1,"19":1,"28":1,"37":1,"38":14,"41":1,"48":3,"49":1,"51":3,"52":1,"54":3,"55":4,"62":1,"63":5,"68":1,"69":5,"83":2,"84":1,"99":1,"100":1,"108":2,"133":1,"135":1,"136":2,"137":1,"142":1,"149":1,"155":1,"170":1,"171":2,"173":1,"179":1,"185":1,"202":1,"214":1,"218":1,"245":1,"247":1,"249":1,"252":1,"255":1,"256":1}}],["whether",{"2":{"40":2,"46":1,"50":1,"183":1}}],["whereas",{"2":{"49":1}}],["where",{"2":{"3":1,"13":2,"16":1,"18":1,"19":1,"20":1,"28":7,"39":1,"40":1,"49":5,"51":6,"52":6,"53":4,"54":3,"55":3,"56":1,"63":4,"68":1,"105":1,"110":2,"140":1,"159":1,"176":1,"184":1,"185":1,"224":1,"249":2,"250":2,"254":3}}],["when",{"2":{"3":2,"16":1,"28":2,"46":2,"49":1,"53":1,"54":1,"56":1,"58":1,"63":2,"64":1,"69":2,"104":1,"122":1,"127":1,"135":2,"144":1,"156":1,"165":1,"175":2,"184":1,"196":1,"214":1,"254":1,"257":1}}],["world",{"2":{"202":1}}],["worthwhile",{"2":{"83":1}}],["word",{"2":{"53":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["words",{"2":{"49":1}}],["workaround",{"2":{"153":1}}],["worker",{"2":{"160":1}}],["workers",{"2":{"33":3,"34":5}}],["worked",{"2":{"88":1,"147":1}}],["working",{"2":{"40":1,"101":1,"103":2,"105":1,"106":1,"123":1,"124":1,"131":1,"169":1,"248":1}}],["work",{"2":{"2":1,"5":1,"38":1,"54":1,"56":1,"74":1,"104":1,"105":1,"111":1,"113":1,"120":1,"123":1,"133":1,"140":1,"144":1,"167":1,"171":2,"176":1}}],["works",{"2":{"2":1,"40":1,"44":1,"56":2,"65":1,"68":1,"83":1,"95":1,"105":1,"118":1,"122":1,"135":2,"137":1,"144":1,"145":1,"153":1,"160":1,"171":1,"214":1}}],["would",{"2":{"7":2,"8":1,"49":2,"54":3,"81":1,"83":1,"90":1,"106":2,"109":1,"127":1,"140":1}}],["wondered",{"2":{"83":1}}],["won",{"2":{"2":1,"8":1,"69":1,"93":1,"114":1,"184":1,"198":1}}],["waveform",{"2":{"254":5,"255":7,"256":9,"257":4,"259":12}}],["waveforms",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["wall",{"2":{"249":1}}],["wan",{"2":{"144":1}}],["wants",{"2":{"142":1}}],["wanted",{"2":{"83":1}}],["want",{"0":{"72":1,"73":1},"2":{"8":2,"62":2,"71":1,"78":1,"107":1,"114":1,"127":1,"133":1,"145":1,"148":1,"171":1,"177":1,"198":1,"201":1,"235":1}}],["warmup",{"2":{"195":1,"257":1}}],["warntype",{"2":{"220":3}}],["warn",{"2":{"67":3,"70":1}}],["warning",{"2":{"2":3,"3":1,"4":2,"8":1,"16":1,"22":2,"44":3,"46":2,"49":1,"55":1,"57":1,"62":1,"63":1,"67":5,"68":1,"125":1,"131":1,"144":1,"145":2,"153":1,"209":1,"210":1,"236":4}}],["warde",{"2":{"54":1}}],["way",{"2":{"17":2,"44":3,"46":2,"48":1,"83":1,"108":1,"133":2,"150":1,"161":1,"185":2,"249":1}}],["wasteful",{"2":{"135":1}}],["was",{"2":{"2":1,"5":3,"7":1,"11":1,"28":1,"41":1,"44":1,"49":1,"90":3,"92":1,"93":1,"99":1,"106":1,"122":2,"155":2,"180":1,"189":1,"197":1,"202":1,"204":1,"214":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["welcome",{"2":{"144":1}}],["well",{"2":{"46":1,"63":1,"81":1,"101":1,"103":1,"106":1,"145":1,"146":1,"170":1}}],["weird",{"2":{"135":1,"137":2}}],["weightnorm",{"2":{"55":4,"83":1,"135":1}}],["weighting",{"2":{"63":1}}],["weightinitializers",{"0":{"25":1,"92":1},"1":{"26":1,"27":1,"28":1,"29":1},"2":{"3":1,"28":8,"29":24,"135":3,"168":2,"256":1}}],["weightih×x+biasih+weighthh×hprev+biashh",{"2":{"52":1}}],["weights",{"0":{"168":1},"1":{"169":1},"2":{"52":4,"63":1,"168":10,"169":5,"225":1,"249":2,"250":3}}],["weight=truncated",{"2":{"256":3}}],["weight=l",{"2":{"135":1}}],["weight=randn32",{"2":{"99":1}}],["weight=rand32",{"2":{"53":1,"99":1}}],["weight=ps",{"2":{"69":1}}],["weight=ones32",{"2":{"53":1}}],["weight=glorot",{"2":{"49":1,"135":1}}],["weight=nothing",{"2":{"49":1,"52":3,"53":3}}],["weight=zero",{"2":{"39":1}}],["weight",{"2":{"16":4,"18":4,"25":1,"28":3,"38":3,"39":4,"41":4,"49":6,"52":12,"53":24,"55":5,"77":15,"106":12,"110":4,"116":2,"126":4,"127":2,"128":4,"135":6,"137":4,"145":2,"147":4,"148":4,"168":1,"179":1,"220":30,"225":5,"226":2,"244":2,"249":1,"256":3,"258":3}}],["weak",{"2":{"57":1}}],["weren",{"2":{"97":1}}],["were",{"2":{"11":1,"85":1,"92":1,"97":2,"144":1}}],["we",{"0":{"81":1,"143":1},"1":{"82":1,"83":1},"2":{"2":4,"3":3,"7":2,"8":4,"13":2,"16":2,"18":3,"32":1,"37":2,"44":1,"48":3,"49":4,"52":6,"53":4,"57":1,"62":2,"63":2,"65":5,"67":1,"69":3,"77":4,"78":2,"83":4,"85":2,"90":1,"92":1,"93":2,"97":1,"98":1,"99":1,"101":1,"103":2,"104":3,"105":3,"106":13,"107":2,"109":3,"110":6,"111":2,"113":1,"114":3,"118":1,"121":1,"123":3,"124":2,"125":1,"126":1,"128":1,"133":1,"135":5,"136":4,"137":4,"138":1,"139":2,"140":2,"143":1,"144":3,"145":2,"146":3,"147":3,"148":3,"149":3,"152":2,"153":11,"155":3,"159":1,"160":1,"170":1,"171":11,"173":4,"174":9,"175":6,"177":2,"179":3,"181":1,"183":3,"184":11,"185":2,"186":1,"187":1,"188":3,"190":2,"193":2,"196":3,"198":4,"201":4,"202":5,"210":1,"211":1,"214":3,"217":1,"219":1,"220":2,"225":2,"230":3,"232":2,"233":2,"234":3,"235":3,"239":1,"243":1,"244":2,"245":2,"247":3,"248":1,"249":8,"250":7,"254":3,"256":9,"257":1}}],[">randn32",{"2":{"69":2}}],[">",{"2":{"2":3,"3":8,"8":2,"28":8,"29":24,"52":1,"63":3,"69":2,"106":1,"110":2,"115":1,"139":2,"140":4,"145":2,"146":2,"147":2,"148":2,"152":1,"186":1,"236":2,"241":1,"245":1,"254":1,"258":1}}],["cycle",{"2":{"145":2,"236":2}}],["cdev",{"2":{"131":1,"132":1,"199":1,"203":1,"231":1,"236":2}}],["cdims",{"2":{"16":3}}],["circle",{"2":{"255":1,"256":2,"259":4}}],["circumvented",{"2":{"132":1}}],["circular",{"2":{"28":1,"49":1}}],["ci",{"2":{"83":1,"209":1}}],["cite",{"2":{"74":2}}],["citation",{"0":{"74":1}}],["cvpr",{"2":{"63":1}}],["cnew",{"2":{"52":2}}],["cnew=f⋅cprev+i⋅ghnew=o⋅tanh",{"2":{"52":1}}],["cluster",{"2":{"250":1}}],["clockwise",{"2":{"181":1,"183":4}}],["closest",{"2":{"116":1}}],["closures",{"2":{"69":1}}],["cl",{"2":{"136":3,"168":4}}],["classifying",{"2":{"250":1}}],["classify",{"2":{"181":1,"248":1}}],["classifier=st",{"2":{"184":1}}],["classifier",{"0":{"184":1},"2":{"184":11,"185":3,"249":1}}],["classifiers",{"2":{"63":2}}],["classified",{"2":{"63":1}}],["classification",{"0":{"190":1,"211":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1},"2":{"28":2,"63":2,"247":1}}],["classes",{"2":{"63":1}}],["class",{"0":{"104":1},"2":{"63":1,"102":1,"105":1,"194":5,"216":5,"227":5,"247":1}}],["clear",{"2":{"41":1,"110":1}}],["c",{"2":{"39":3,"49":4,"51":21,"52":1,"56":5,"184":2,"192":1,"213":1,"220":9,"250":2,"254":4}}],["cell=st",{"2":{"184":1}}],["cell`",{"2":{"184":1}}],["cells",{"2":{"52":1,"99":1}}],["cell",{"2":{"52":45,"184":10,"185":4}}],["central",{"2":{"28":1,"220":13}}],["certain",{"0":{"141":1},"1":{"142":1},"2":{"7":1,"8":2,"13":1,"50":2,"64":1,"81":1,"90":1,"97":1,"106":1,"120":1,"124":1,"125":1,"155":1,"167":1,"214":1}}],["ca",{"2":{"202":2,"225":3}}],["cairomakie",{"2":{"199":1,"200":1,"203":1,"231":1,"237":2,"240":1,"241":1,"245":1,"247":1,"248":1,"253":1,"255":1,"256":1,"259":2}}],["capabilities",{"2":{"172":1,"230":1}}],["capture",{"2":{"144":1}}],["causing",{"2":{"155":1}}],["causes",{"2":{"124":1}}],["cause",{"2":{"8":1,"13":1,"41":1,"155":1}}],["cat",{"2":{"115":1,"183":1}}],["catch",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"62":1,"109":2,"110":2,"116":1,"157":2}}],["caching",{"2":{"62":1}}],["cached",{"2":{"62":2}}],["cache",{"2":{"62":2,"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["care",{"2":{"174":1}}],["cartesian",{"2":{"53":1}}],["carry",{"2":{"52":4,"184":3,"185":3}}],["case",{"2":{"38":1,"52":19,"67":1,"122":1,"135":1,"143":1,"145":1,"179":1,"202":1,"225":1,"250":1,"256":1}}],["cases",{"2":{"8":3,"28":1,"37":1,"44":1,"62":2,"65":6,"67":1,"68":1,"69":1,"71":1,"83":1,"104":1,"105":1,"111":2,"139":1,"146":1,"153":1,"159":1,"173":1}}],["caveats",{"2":{"28":1}}],["calculation",{"2":{"145":1,"146":3}}],["calculations",{"2":{"46":1}}],["calculates",{"2":{"51":3}}],["calculate",{"2":{"49":4,"51":3,"53":1}}],["calculated",{"2":{"7":2,"17":1,"63":7}}],["caller",{"2":{"236":4}}],["called",{"2":{"3":1,"17":2,"19":2,"32":1,"48":2,"50":3,"127":1,"135":1,"140":2,"142":1,"169":1,"171":2,"184":1}}],["callback",{"2":{"202":4,"257":3,"258":1}}],["callable",{"2":{"48":2}}],["calls",{"2":{"7":1,"8":2,"144":2}}],["call",{"2":{"7":1,"8":1,"17":1,"39":1,"50":5,"55":3,"58":6,"68":1,"77":1,"83":1,"144":1,"145":2,"153":1,"168":4,"174":2,"184":1}}],["calling",{"2":{"5":1,"7":1,"8":1,"32":1,"58":1,"107":1,"120":1,"123":1,"249":1}}],["candidates",{"2":{"116":1}}],["cannot",{"2":{"28":1,"69":1,"82":1,"135":1,"136":1,"140":1,"153":1,"209":1}}],["can",{"0":{"143":1},"2":{"2":2,"3":4,"4":2,"7":3,"8":3,"11":1,"13":1,"15":1,"16":1,"17":2,"18":1,"19":14,"20":1,"32":3,"38":1,"40":1,"48":6,"49":6,"51":3,"52":7,"53":3,"54":3,"58":1,"59":1,"62":1,"63":2,"65":1,"68":1,"69":5,"70":1,"71":2,"76":2,"77":3,"78":1,"79":1,"98":1,"106":2,"108":2,"109":1,"110":2,"118":1,"120":1,"126":1,"128":1,"130":1,"132":1,"136":1,"137":3,"140":1,"142":1,"143":1,"144":2,"145":2,"146":3,"148":1,"149":2,"153":1,"155":2,"156":1,"157":1,"160":1,"163":1,"166":1,"167":1,"168":1,"169":1,"171":11,"172":2,"174":1,"176":1,"179":1,"184":1,"185":1,"187":1,"188":1,"202":1,"214":2,"217":1,"218":1,"226":1,"232":1,"233":1,"249":2,"250":3,"254":1,"256":1}}],["coordinates",{"2":{"250":1}}],["cos",{"2":{"147":1,"235":1,"254":2,"255":3,"256":8}}],["cover",{"2":{"139":1}}],["covered",{"2":{"111":1}}],["covariate",{"2":{"19":1}}],["could",{"2":{"135":1,"136":1,"249":1}}],["course",{"2":{"118":1,"145":1}}],["courville",{"2":{"54":1}}],["coupled",{"2":{"81":1,"167":1}}],["counterpart",{"2":{"143":1}}],["count",{"2":{"69":1}}],["coefficient",{"2":{"63":2}}],["colormap=",{"2":{"250":3}}],["colorbar",{"2":{"237":1}}],["color=",{"2":{"200":2,"203":4,"241":2,"245":2,"248":2}}],["collect",{"2":{"69":1,"78":1,"183":2,"192":2,"213":2,"235":4,"237":1,"241":1,"250":4}}],["collects",{"2":{"48":1}}],["columns",{"2":{"171":1}}],["column",{"2":{"53":1,"171":4}}],["cols",{"2":{"28":1}}],["corner",{"2":{"56":1}}],["corners",{"2":{"56":1,"99":1}}],["corners=false",{"2":{"56":1}}],["cores",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["core",{"2":{"44":1,"58":3,"83":1,"139":1,"173":1,"180":1,"189":1,"197":1,"204":1,"220":2,"221":1,"225":5,"226":3,"229":1,"236":4,"238":1,"246":1,"251":1,"260":1}}],["correlation=true",{"2":{"49":2,"98":1,"100":1}}],["correlation=false",{"2":{"49":2}}],["correlation",{"2":{"49":7}}],["corresponding",{"2":{"4":3,"39":2,"41":2,"53":1,"56":2,"63":3,"94":1,"97":1,"131":1,"146":1}}],["corresponds",{"2":{"2":1,"202":1}}],["corrections",{"2":{"118":1}}],["correctness",{"0":{"59":1},"2":{"57":1,"65":1}}],["correct",{"0":{"117":1},"2":{"3":2,"59":1,"155":2,"194":3,"216":3,"227":3}}],["correctly",{"2":{"2":1,"40":1,"111":1,"120":1}}],["codebases",{"2":{"68":1}}],["code",{"0":{"120":1},"2":{"37":2,"44":1,"46":1,"65":4,"69":4,"78":1,"85":1,"103":1,"106":1,"120":1,"130":1,"138":1,"142":1,"143":1,"145":2,"155":1,"156":1,"157":1,"161":1,"170":1,"185":1,"187":1,"220":3,"248":1,"252":2}}],["combination",{"2":{"144":1}}],["combined",{"2":{"52":2,"77":1}}],["combines",{"2":{"48":1}}],["come",{"2":{"109":1,"155":1,"177":1}}],["comes",{"2":{"48":1,"81":1,"83":2,"145":1,"147":1}}],["coming",{"2":{"78":1,"135":1,"155":1}}],["com",{"2":{"52":1,"71":1}}],["community",{"2":{"210":1}}],["communication",{"0":{"34":1},"2":{"122":1,"123":1}}],["communications",{"2":{"32":2}}],["commit",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["command",{"2":{"71":2,"161":2}}],["comm",{"2":{"31":2}}],["common",{"2":{"25":1,"41":1,"65":4,"136":1,"160":1,"166":1,"168":1}}],["comprises",{"2":{"136":1}}],["compilable",{"2":{"106":1}}],["compilation",{"2":{"106":1}}],["compiling",{"0":{"106":1},"1":{"107":1},"2":{"83":1}}],["compiled",{"2":{"103":1,"106":7}}],["compiler",{"2":{"81":2,"106":1}}],["compile=true",{"2":{"62":1}}],["compile",{"2":{"3":1,"48":1,"58":1,"62":1,"106":8}}],["completeness",{"2":{"152":1,"217":1}}],["completely",{"2":{"52":1,"81":1,"97":1}}],["complete",{"2":{"51":3,"52":1,"168":1}}],["complexity",{"2":{"84":1,"175":1}}],["complexf64",{"2":{"29":8}}],["complexf32",{"2":{"29":8,"168":2}}],["complexf16",{"2":{"29":8}}],["complex",{"2":{"5":1,"62":1,"69":1,"168":1}}],["compare",{"2":{"121":1,"152":1}}],["comparison",{"2":{"48":1}}],["compactmacroimpl",{"2":{"220":3}}],["compactluxlayer",{"2":{"69":2,"220":3,"225":2}}],["compact",{"0":{"69":1,"185":1},"2":{"68":1,"69":20,"78":3,"98":2,"133":2,"144":2,"185":2,"187":1,"214":4,"218":1,"220":7,"225":1}}],["compatibility",{"2":{"44":1,"105":1,"133":1}}],["compatible",{"2":{"6":1,"16":1,"18":1,"35":1,"36":2,"40":2,"63":2,"133":1,"144":1,"153":2,"217":1}}],["components",{"2":{"254":4}}],["component",{"2":{"254":7}}],["componentvector",{"2":{"179":1,"220":6,"258":1}}],["componentarray",{"2":{"41":1,"145":2,"146":2,"147":2,"148":3,"152":4,"179":1,"202":2,"215":1,"217":1,"225":2,"256":1}}],["componentarrays",{"2":{"41":2,"137":1,"144":1,"175":1,"199":1,"212":1,"220":12,"223":1,"249":1,"253":1}}],["composability",{"2":{"247":1}}],["composes",{"2":{"101":1}}],["composedlinear",{"2":{"136":3}}],["composed",{"2":{"52":1,"62":1}}],["compose",{"2":{"46":1,"249":1}}],["composition",{"2":{"19":1,"136":1}}],["computing",{"0":{"148":1,"150":1,"151":1,"152":1},"2":{"22":2,"23":1,"146":1,"150":1,"176":1}}],["computed",{"2":{"17":1,"19":1,"48":4,"49":2,"53":1,"54":1,"55":2,"62":3,"63":3,"176":4,"244":1}}],["computes",{"2":{"14":1,"16":1,"19":3,"23":1,"28":1,"55":4,"62":2,"140":1,"149":1}}],["compute",{"2":{"13":2,"18":1,"22":6,"23":1,"62":10,"63":1,"77":3,"90":1,"99":2,"106":1,"110":1,"148":3,"149":5,"152":1,"175":1,"176":1,"177":1,"178":1,"186":1,"195":1,"202":1,"232":1,"234":2,"249":1,"254":4,"255":1,"256":1,"257":1,"259":1}}],["computer",{"2":{"5":1,"19":1,"28":2,"63":4}}],["computationally",{"2":{"175":1}}],["computational",{"2":{"175":1}}],["computation",{"0":{"145":1,"147":1},"1":{"146":1},"2":{"3":1,"13":1,"48":2,"55":2,"59":1,"63":1,"90":1,"160":2}}],["copying",{"2":{"184":1}}],["copyto",{"2":{"65":3}}],["copy",{"2":{"8":1,"62":2,"65":1,"140":2,"173":5}}],["copied",{"2":{"5":1,"52":4}}],["concise",{"2":{"133":1,"185":1}}],["concatenate",{"2":{"115":1}}],["concatenated",{"2":{"52":7}}],["conclusion",{"0":{"111":1}}],["concrete",{"0":{"114":1},"1":{"115":1,"116":1,"117":1,"118":1},"2":{"85":1,"137":1,"140":1}}],["conjunction",{"2":{"83":1}}],["connection",{"2":{"48":26,"52":4}}],["connected",{"0":{"18":1},"2":{"28":1,"53":3}}],["confusing",{"2":{"202":1}}],["confusion",{"2":{"90":1}}],["conform",{"2":{"48":1}}],["conference",{"2":{"19":2,"28":6,"63":4}}],["convtranspose",{"2":{"49":1,"99":1,"100":2}}],["convolutions",{"2":{"49":5}}],["convolution",{"2":{"28":1,"49":9}}],["convolutional",{"0":{"16":1,"49":1},"2":{"28":2,"49":4,"63":1}}],["convdims",{"2":{"16":2}}],["conv",{"2":{"16":3,"46":2,"49":4,"98":1,"99":1,"139":1,"158":2,"193":6}}],["convention",{"2":{"173":1}}],["conveniently",{"2":{"171":1}}],["convenience",{"0":{"29":1},"2":{"7":1,"68":1,"101":1,"245":1}}],["conversely",{"2":{"175":1}}],["converse",{"2":{"49":1}}],["conversions",{"2":{"67":2}}],["conversion",{"0":{"165":1},"2":{"5":1}}],["converts",{"2":{"56":1,"66":3}}],["converted",{"2":{"46":2,"67":1}}],["converting",{"2":{"41":1,"46":3}}],["convert",{"2":{"5":1,"8":1,"44":2,"46":4,"67":3,"140":1,"143":1,"153":1,"193":1,"202":1,"203":1,"214":1}}],["cond",{"2":{"8":3}}],["conditions",{"2":{"196":1,"234":1,"255":1,"256":1}}],["condition",{"2":{"8":2,"28":1}}],["continua",{"2":{"254":1}}],["continue",{"2":{"123":1}}],["contour",{"2":{"237":1,"250":6}}],["contrib",{"2":{"109":1}}],["controlled",{"2":{"67":1,"82":1,"100":1}}],["controlling",{"2":{"67":1,"165":1,"166":1}}],["control",{"2":{"56":2,"106":2,"155":1,"170":1}}],["controls",{"2":{"28":1,"49":4,"52":3,"55":8,"63":1}}],["contrastive",{"2":{"63":1}}],["contrast",{"2":{"2":1}}],["context",{"2":{"17":1,"19":2,"64":1}}],["context`",{"2":{"17":1}}],["contained",{"2":{"48":1}}],["containerlayer",{"2":{"214":1}}],["containers",{"0":{"48":1},"2":{"90":2,"220":3}}],["container",{"0":{"136":1},"2":{"7":2,"8":1,"36":1,"40":1,"91":1,"120":1,"136":2,"184":2}}],["contains",{"0":{"147":1},"2":{"8":1,"23":1,"50":1,"54":1,"64":2}}],["contain",{"2":{"7":1,"67":1,"69":1,"83":1,"135":2,"184":1}}],["containing",{"0":{"145":1},"1":{"146":1},"2":{"3":1,"7":3,"19":2,"28":4,"29":24,"52":14,"53":1,"62":2,"63":2,"82":1}}],["consensus",{"2":{"147":1}}],["consecutive",{"2":{"48":1}}],["consequence",{"2":{"39":1}}],["considering",{"2":{"175":1}}],["consider",{"2":{"114":1,"135":1,"137":1,"155":1,"176":1,"179":1}}],["considered",{"2":{"3":1,"4":2,"59":1,"62":1,"125":1,"130":1}}],["consistent",{"2":{"52":1}}],["consistency",{"2":{"28":1}}],["consists",{"2":{"48":1,"146":1}}],["consult",{"2":{"24":1}}],["constrained",{"2":{"138":1}}],["constructured",{"2":{"140":1}}],["constructing",{"2":{"62":1,"120":1}}],["construction",{"2":{"7":1,"56":1,"97":1}}],["constructor",{"2":{"38":1}}],["construct",{"2":{"28":1,"38":1,"40":2,"69":2,"77":2,"82":1,"107":1,"108":1,"136":2,"202":1,"215":1,"249":2}}],["constructed",{"2":{"17":1,"69":1,"135":1}}],["constructs",{"2":{"7":1,"28":2,"38":1}}],["const",{"2":{"72":4,"73":3,"106":5,"110":1,"199":2,"200":2,"220":2,"231":2,"244":2,"254":6,"255":1,"256":2}}],["constants",{"2":{"17":1,"255":1,"256":1}}],["constant",{"2":{"3":1}}],["cupti",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["cusparse",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["cusolver",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["customparamtype",{"2":{"38":2}}],["customabstractluxlayer",{"2":{"7":4}}],["customize",{"2":{"3":1}}],["custom",{"0":{"78":1,"112":1,"140":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1},"2":{"3":1,"7":2,"8":1,"11":1,"20":1,"38":1,"69":1,"78":1,"83":1,"110":2,"117":1,"133":1,"137":1,"144":1,"163":1,"173":1,"181":1,"184":1,"185":1,"214":1,"230":1,"234":1}}],["cufft",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["curand",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["currently",{"2":{"4":4,"18":2,"20":1,"40":1,"51":3,"56":1,"104":2,"105":1,"106":1,"107":1,"124":1,"130":1,"144":1,"153":2,"177":1}}],["current",{"2":{"2":1,"5":1,"7":1,"38":2,"50":1,"62":1,"101":1,"114":1,"173":1,"249":1,"250":1}}],["cublas",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["cublaslt",{"2":{"18":1}}],["cu",{"2":{"172":4}}],["cuarray",{"2":{"5":3,"26":3,"77":18,"131":1,"153":2,"168":2,"172":1,"220":18}}],["cuiterator",{"2":{"5":2,"95":1,"160":1}}],["cudevice",{"2":{"4":1}}],["cudnn",{"2":{"2":2,"3":1,"16":1,"72":1}}],["cudadevice",{"2":{"2":2,"4":2,"5":1,"78":1,"94":1,"123":1,"132":2,"180":1,"189":1,"197":1,"201":2,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["cuda",{"0":{"172":1},"2":{"2":6,"3":2,"4":1,"5":5,"16":1,"18":1,"26":1,"32":7,"72":1,"77":19,"78":1,"80":1,"88":2,"95":1,"122":1,"123":1,"124":1,"130":1,"131":1,"153":2,"160":1,"163":3,"164":1,"168":5,"172":7,"180":10,"189":10,"197":3,"204":10,"212":1,"220":36,"221":10,"223":1,"229":10,"231":1,"238":10,"246":10,"251":3,"260":3}}],["ch",{"2":{"249":2,"250":1}}],["cheat",{"2":{"147":1}}],["checking",{"2":{"135":1,"220":1}}],["check=",{"2":{"110":2}}],["check",{"2":{"8":4,"32":1,"40":11,"50":3,"55":3,"59":1,"69":1,"106":1,"110":3,"152":1,"157":2,"166":3}}],["checked",{"2":{"3":1}}],["checks",{"2":{"3":2,"38":2,"40":1,"69":2}}],["chunks",{"2":{"64":1}}],["choice",{"2":{"188":1}}],["chosen",{"2":{"169":1}}],["chopra",{"2":{"63":1}}],["choose",{"2":{"2":1}}],["chs",{"2":{"49":17,"55":19}}],["christian",{"2":{"19":1}}],["chains",{"0":{"46":1},"2":{"46":4,"193":1,"196":1,"249":2}}],["chain=chain",{"2":{"39":1}}],["chain",{"0":{"115":1,"129":1},"2":{"39":10,"41":3,"44":1,"46":2,"48":7,"52":2,"54":4,"55":6,"69":1,"77":9,"97":1,"106":2,"107":1,"109":8,"110":6,"114":1,"115":2,"118":3,"126":2,"127":1,"129":2,"139":2,"145":1,"146":1,"147":1,"148":1,"152":1,"153":1,"193":6,"202":1,"215":2,"220":18,"226":2,"233":1,"242":2,"245":1,"249":5,"250":2,"256":2}}],["chainrulescore",{"2":{"110":2}}],["chainrules",{"2":{"16":1,"18":1,"40":2,"63":1,"102":1,"105":1,"144":1}}],["channel",{"2":{"51":3,"55":12,"56":3}}],["channels",{"2":{"19":1,"49":6,"50":2,"55":1,"56":1}}],["changed",{"2":{"87":1,"121":1,"142":1}}],["changes",{"0":{"87":1,"90":1,"94":1,"97":1,"98":1,"99":2},"2":{"7":1,"11":1,"48":9,"54":2,"70":1,"81":1,"85":1,"90":1,"92":1,"109":1,"121":1,"139":1,"144":1}}],["change",{"2":{"1":1,"2":1,"68":1,"90":1,"93":1,"120":1,"135":1,"217":1,"254":1}}],["crude",{"2":{"254":1}}],["crc",{"2":{"110":2}}],["critical",{"2":{"104":1,"130":1}}],["criteria",{"2":{"2":1,"40":1}}],["creating",{"0":{"184":1},"2":{"69":1,"214":1}}],["created",{"2":{"69":1}}],["create",{"0":{"215":1,"226":1},"2":{"2":1,"5":1,"31":2,"44":1,"48":2,"53":3,"62":1,"65":2,"69":1,"106":2,"107":1,"110":1,"117":1,"171":1,"174":1,"181":1,"183":3,"184":2,"187":1,"215":2,"217":1,"220":3,"226":2,"228":1,"233":4,"245":1,"249":2}}],["creates",{"2":{"1":1,"8":1,"28":1,"52":5,"69":1,"174":1,"249":1}}],["crossentropyloss",{"2":{"63":6,"194":1,"216":1,"227":1}}],["crosscor",{"2":{"49":1,"98":1}}],["cross",{"2":{"49":11,"63":3,"98":1,"100":1,"133":1}}],["cpu=true",{"2":{"217":2}}],["cpus",{"2":{"13":1,"14":2,"46":1,"155":1,"159":1}}],["cpudevice",{"2":{"2":4,"4":2,"131":3,"132":1,"199":1,"231":1}}],["cpu",{"2":{"2":4,"5":1,"18":1,"20":1,"51":3,"59":1,"73":2,"83":1,"98":2,"102":1,"103":3,"105":1,"106":1,"130":1,"131":6,"132":7,"140":1,"153":3,"167":1,"180":2,"187":1,"188":1,"189":2,"196":1,"197":2,"199":1,"204":2,"217":4,"221":2,"229":2,"231":1,"238":2,"244":2,"245":1,"246":2,"251":2,"260":2}}],["rk4",{"2":{"255":1,"256":1,"257":1,"259":1}}],["r₂",{"2":{"254":2}}],["r₁",{"2":{"254":2}}],["r2",{"2":{"254":1}}],["r1",{"2":{"254":1}}],["r=r1−r2",{"2":{"254":1}}],["r=σ",{"2":{"52":1}}],["rhat",{"2":{"249":1}}],["rtx",{"2":{"229":2,"238":2}}],["risk",{"2":{"124":1}}],["right",{"2":{"54":1}}],["rrule",{"2":{"110":1}}],["rrules",{"2":{"105":1}}],["rᴰ",{"2":{"56":1}}],["r²",{"2":{"56":1}}],["r",{"2":{"56":9,"69":9,"254":3}}],["rnns",{"2":{"52":1}}],["rnn",{"2":{"52":3,"64":1}}],["rnncell",{"2":{"52":9,"99":1,"100":1}}],["rng=xoshiro",{"2":{"195":1}}],["rngs",{"2":{"82":1}}],["rng",{"0":{"26":1},"2":{"3":3,"7":2,"8":6,"9":1,"10":1,"11":2,"17":6,"26":10,"28":11,"29":24,"39":3,"44":1,"46":1,"48":4,"50":6,"52":3,"53":4,"54":18,"69":5,"77":7,"78":3,"92":2,"97":1,"106":1,"107":1,"109":3,"110":2,"116":2,"118":1,"126":4,"129":6,"135":9,"136":2,"137":5,"139":7,"140":9,"155":4,"168":13,"169":6,"170":2,"171":1,"174":5,"176":1,"177":1,"179":8,"187":2,"195":2,"201":1,"202":1,"215":4,"225":3,"228":2,"236":4,"241":4,"244":1,"248":11,"249":1,"256":1}}],["rgb",{"2":{"49":1}}],["row",{"2":{"171":3,"250":1}}],["rows",{"2":{"28":1,"171":1}}],["routines",{"2":{"144":1}}],["route",{"2":{"136":1}}],["rough",{"2":{"144":1}}],["rounded",{"2":{"49":2,"51":3}}],["round",{"2":{"32":1,"228":4,"248":1}}],["root",{"2":{"34":7,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["robin",{"2":{"32":1}}],["rocarray",{"2":{"26":4}}],["rocrand",{"2":{"26":1}}],["rocm",{"2":{"3":1,"72":2,"123":1,"124":1,"163":3,"164":1}}],["ryan",{"2":{"19":1}}],["raw",{"2":{"192":2,"213":2}}],["rademacher",{"2":{"152":1}}],["ra",{"2":{"106":18,"107":9}}],["ratio≥0",{"2":{"254":1}}],["ratio≤1",{"2":{"254":1}}],["ratio",{"2":{"254":6,"255":2,"256":1,"257":1,"259":1}}],["rationale",{"2":{"90":1}}],["rate",{"2":{"179":1,"249":1}}],["rather",{"2":{"3":1,"16":1,"52":1,"82":1,"184":1}}],["raia",{"2":{"63":1}}],["ran",{"2":{"109":1}}],["randc64",{"2":{"29":1}}],["randc32",{"2":{"29":1}}],["randc16",{"2":{"29":1}}],["rand64",{"2":{"29":1}}],["rand32",{"2":{"29":1}}],["rand16",{"2":{"29":1}}],["randnc64",{"2":{"29":1}}],["randnc32",{"2":{"29":1}}],["randnc16",{"2":{"29":1}}],["randn64",{"2":{"29":1}}],["randn32",{"2":{"29":1}}],["randn16",{"2":{"29":1}}],["randn",{"2":{"28":1,"44":1,"46":1,"48":2,"54":2,"69":1,"106":1,"107":1,"109":1,"116":1,"126":1,"129":1,"131":1,"132":1,"135":1,"136":1,"137":1,"139":2,"140":6,"145":2,"146":2,"171":2,"174":1,"176":1,"177":1,"179":4,"195":1,"241":1}}],["randomness",{"0":{"174":1},"2":{"52":3,"77":1,"82":1,"149":1,"170":1,"174":1}}],["randomly",{"2":{"50":2}}],["random",{"2":{"17":4,"26":1,"28":8,"29":12,"39":2,"44":2,"46":2,"48":2,"50":3,"54":8,"69":3,"77":4,"78":3,"106":3,"107":1,"109":1,"115":1,"126":2,"129":4,"135":4,"137":3,"139":4,"140":4,"144":1,"149":1,"153":1,"155":2,"168":3,"170":5,"171":1,"174":8,"179":4,"182":1,"191":1,"199":1,"201":1,"202":1,"212":1,"215":2,"223":1,"225":1,"231":1,"235":1,"236":2,"240":1,"241":2,"247":1,"248":2,"253":1,"256":1}}],["rand",{"2":{"5":1,"69":1,"77":3,"78":1,"147":1,"148":1,"152":2,"153":1,"155":2,"171":5,"172":1,"174":3,"248":8}}],["rank",{"2":{"4":3,"33":3,"120":1,"121":1}}],["ranges",{"2":{"48":1}}],["range",{"2":{"3":2,"92":1,"177":1,"200":1,"235":4,"241":1,"250":23,"255":1}}],["rule",{"2":{"56":1,"245":2}}],["rules",{"0":{"117":1},"2":{"0":1,"65":1,"104":1}}],["runtime",{"2":{"59":2,"180":2,"189":2,"204":2,"221":2,"229":2,"238":2,"246":2}}],["running",{"0":{"116":1},"2":{"17":2,"19":18,"55":16,"90":1,"109":5,"110":11,"126":2,"129":2,"142":1,"156":1,"184":1}}],["run",{"2":{"2":1,"3":1,"5":1,"50":1,"58":1,"71":2,"72":1,"73":1,"77":1,"97":1,"106":3,"110":3,"117":1,"124":1,"135":1,"140":1,"153":3,"155":1,"161":3,"174":1,"184":1,"209":1,"250":1}}],["red",{"2":{"200":1,"203":2,"248":1}}],["reduction",{"2":{"51":3,"63":1}}],["reduce",{"2":{"34":3,"54":1,"57":1,"249":1}}],["reducing",{"2":{"19":1,"48":1}}],["reiterate",{"2":{"140":1}}],["refresher",{"2":{"135":1}}],["referred",{"2":{"171":1}}],["refer",{"2":{"78":1,"198":1,"210":1,"230":1}}],["reference",{"0":{"27":1},"1":{"28":1,"29":1},"2":{"190":1}}],["references",{"2":{"17":2,"19":4,"28":6,"54":1,"55":1,"63":4}}],["reexport",{"2":{"97":2}}],["renamed",{"2":{"90":2,"93":1,"94":2}}],["renormalize",{"2":{"55":2}}],["request",{"2":{"84":1,"90":1}}],["requisites",{"2":{"77":1}}],["requirements",{"2":{"48":1,"245":1}}],["require",{"2":{"19":1,"44":1,"46":1,"69":1,"123":1,"202":1}}],["requires",{"2":{"8":1,"10":1,"32":4,"64":1,"80":1,"113":1,"202":1,"225":1}}],["rev=true",{"2":{"236":4}}],["revising",{"2":{"83":1}}],["reverses",{"2":{"54":1}}],["reversesequence",{"2":{"54":3}}],["reversed",{"2":{"54":1}}],["reversediffadjoint",{"2":{"214":1}}],["reversediff",{"2":{"8":2,"16":1,"18":1,"20":1,"59":1,"62":1,"65":1,"102":1,"103":1,"116":1,"217":1}}],["reverse",{"2":{"22":2,"40":2,"54":2,"59":1,"69":1,"102":6,"106":1,"110":1,"175":2,"176":2}}],["retcode",{"2":{"258":1}}],["retrieve",{"2":{"53":1}}],["retained",{"2":{"50":1}}],["retuened",{"2":{"3":2}}],["returned",{"2":{"17":1,"19":1,"26":1,"28":1,"36":1,"41":1,"44":1,"46":1,"50":4,"52":3,"53":1,"54":1,"62":6,"64":1}}],["returning",{"2":{"8":1,"135":1}}],["returns",{"2":{"3":3,"7":2,"8":3,"13":2,"17":2,"19":4,"20":1,"22":2,"23":1,"28":6,"41":1,"48":7,"49":2,"50":3,"51":9,"52":8,"53":5,"54":6,"55":5,"56":2,"60":1,"62":2,"63":6,"64":2,"67":1,"95":1,"131":3,"146":1,"147":1,"148":1,"202":1,"220":6,"256":1}}],["return",{"2":{"2":3,"3":5,"7":1,"9":1,"10":1,"11":2,"28":6,"29":24,"38":1,"39":2,"48":3,"52":4,"54":1,"62":4,"63":9,"64":2,"65":2,"69":19,"78":3,"82":1,"90":1,"106":2,"107":1,"110":2,"115":1,"117":2,"126":1,"127":3,"128":4,"135":5,"136":2,"137":3,"140":1,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"169":1,"179":1,"183":1,"184":2,"185":2,"186":1,"187":1,"192":1,"194":1,"195":1,"200":1,"202":3,"213":1,"214":5,"215":1,"216":1,"218":2,"220":3,"224":2,"225":3,"226":1,"227":1,"228":1,"233":2,"234":3,"236":1,"237":1,"241":1,"244":1,"245":1,"248":1,"249":2,"250":2,"254":11,"255":1,"256":1,"257":2}}],["remove",{"2":{"174":1}}],["removed",{"0":{"97":1,"122":1},"2":{"49":1,"87":1,"90":2,"94":1,"97":10,"98":4,"106":1,"122":1,"130":1,"145":2,"146":1}}],["remake",{"2":{"202":1}}],["remark",{"2":{"146":1}}],["remains",{"2":{"17":1}}],["remember",{"2":{"107":1,"108":1,"111":1,"135":1,"144":1,"153":1,"184":1,"217":1}}],["regions",{"2":{"250":1}}],["registry",{"2":{"71":1,"79":1}}],["registered",{"2":{"69":1,"71":1}}],["regression",{"0":{"179":1},"2":{"155":1,"179":1}}],["regressions",{"2":{"8":1,"130":1,"196":1}}],["reg",{"2":{"145":2,"146":2}}],["regarding",{"2":{"140":1,"175":1}}],["regularization",{"2":{"145":1,"152":1,"249":1}}],["regular",{"2":{"46":2,"69":2,"153":1}}],["reuse",{"2":{"16":1,"18":1}}],["reusing",{"2":{"16":1,"18":1}}],["reltol",{"2":{"220":13}}],["reltol=1",{"2":{"215":1}}],["reliance",{"2":{"145":2}}],["reliable",{"2":{"103":2}}],["reliability",{"2":{"83":1}}],["relies",{"2":{"140":2,"161":1,"174":1}}],["relevant",{"2":{"106":1,"247":1,"254":1}}],["release",{"2":{"85":2,"92":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["released",{"2":{"71":1}}],["relativisticorbitmodel",{"2":{"255":3}}],["relatively",{"2":{"144":1,"153":1}}],["relation",{"2":{"52":1}}],["related",{"2":{"84":1}}],["relu",{"2":{"16":1,"44":1,"46":4,"48":4,"52":1,"54":3,"55":8,"69":3,"78":1,"109":5,"110":5,"139":2,"153":1,"193":12,"226":2,"242":2,"245":2}}],["rely",{"2":{"15":1,"69":1,"90":1,"174":1}}],["readme",{"2":{"106":1}}],["read",{"2":{"84":1,"173":1}}],["reactant",{"0":{"106":1},"1":{"107":1},"2":{"73":7,"83":1,"106":9,"107":1}}],["reason",{"2":{"103":1,"155":1}}],["reasonable",{"2":{"48":1}}],["reasons",{"2":{"8":1,"214":1}}],["really",{"2":{"46":1,"69":1,"105":1,"135":1,"171":1}}],["reallocations",{"2":{"16":1,"18":1}}],["real",{"2":{"28":1,"50":1,"63":4,"117":1,"237":2}}],["real=eps",{"2":{"19":1}}],["rewrite",{"2":{"13":1,"44":1,"232":1}}],["reconstruction",{"2":{"135":1}}],["recognition",{"2":{"63":1}}],["record",{"2":{"60":1,"237":1,"250":1}}],["recorded",{"2":{"59":1,"60":4}}],["recomputing",{"2":{"17":1}}],["recommendations",{"0":{"103":1},"2":{"154":1,"155":1}}],["recommendation",{"2":{"83":1}}],["recommend",{"2":{"7":1,"83":1,"123":1,"133":1,"135":1,"173":1,"177":1,"185":1,"188":1,"211":1,"214":1,"220":1,"225":1}}],["recommended",{"0":{"131":1},"2":{"3":1,"7":2,"8":1,"15":1,"38":1,"44":1,"57":1,"65":1,"69":2,"83":1,"133":1,"135":2,"140":1,"144":1,"145":2,"150":1,"152":1,"153":1,"156":1,"164":1,"186":1,"256":1}}],["receives",{"2":{"54":1}}],["recvbuf",{"2":{"34":6}}],["rectifiers",{"2":{"28":2}}],["recur",{"2":{"52":3}}],["recurrence",{"2":{"52":4,"184":1}}],["recurrent",{"0":{"52":1},"2":{"28":1,"52":10,"99":1,"181":2}}],["recurse",{"2":{"65":3}}],["recurses",{"2":{"40":1,"66":3}}],["recursion",{"2":{"65":1}}],["recursive",{"0":{"65":1},"2":{"65":9,"153":2,"155":1}}],["recursively",{"2":{"10":1,"65":5}}],["repository",{"2":{"144":1}}],["report",{"2":{"37":1,"84":1,"105":1}}],["reproducer",{"2":{"124":1}}],["represents",{"2":{"249":1}}],["representation",{"2":{"69":1}}],["representing",{"2":{"56":1}}],["represent",{"2":{"28":1}}],["reparameterized",{"2":{"55":2}}],["reparameterization",{"2":{"55":1}}],["repack",{"2":{"8":1}}],["repeat",{"2":{"183":2}}],["repeatedly",{"2":{"48":1}}],["repeatedlayer",{"2":{"48":1}}],["repeats",{"2":{"48":6,"52":5}}],["replacing",{"2":{"139":1,"171":1}}],["replacement",{"2":{"90":1}}],["replaces",{"2":{"40":1,"51":3}}],["replaced",{"2":{"13":1}}],["replace",{"2":{"13":2,"118":1,"158":5}}],["repl",{"2":{"58":1,"71":1,"76":1,"156":2,"161":1}}],["replicated",{"2":{"174":1}}],["replicate",{"2":{"7":1,"8":1,"48":1,"174":3}}],["resolve",{"2":{"148":1}}],["resolution",{"2":{"56":1}}],["resources",{"0":{"84":1}}],["rescale",{"2":{"55":4}}],["res",{"2":{"48":4,"69":2,"202":5,"258":1,"259":1}}],["resnet",{"2":{"48":1}}],["reshape",{"2":{"63":4,"78":1,"150":2,"151":2,"173":2,"183":2,"192":1,"213":1,"214":4,"224":2,"235":2,"237":2,"241":1,"249":1}}],["reshapes",{"2":{"54":1}}],["reshapelayer",{"2":{"54":3}}],["reshaped",{"2":{"16":1,"54":1}}],["reshaping",{"2":{"28":1}}],["restricted",{"2":{"65":1}}],["rest",{"2":{"28":1,"184":2,"185":2}}],["restarted",{"2":{"1":1,"70":1}}],["respective",{"2":{"81":1,"120":2,"140":1,"173":1}}],["respect",{"2":{"23":2,"59":1,"146":2}}],["responsibility",{"2":{"8":1,"137":1}}],["reserved",{"2":{"69":1}}],["research",{"2":{"17":1}}],["resets",{"2":{"3":1}}],["reset",{"2":{"3":1}}],["results",{"0":{"203":1,"237":1,"259":1},"2":{"106":1,"152":2,"245":1,"250":2,"255":1,"256":1,"258":1,"259":1}}],["result",{"2":{"15":1,"23":2,"28":1,"34":4,"55":1,"60":4,"82":1,"137":4}}],["mtensor",{"2":{"254":4}}],["mtlarray",{"2":{"26":3}}],["mcse",{"2":{"249":1}}],["mcmcchains",{"2":{"249":1}}],["mcmc",{"2":{"249":1,"250":1}}],["mcclelland",{"2":{"28":1}}],["mvnormal",{"2":{"249":1}}],["md",{"2":{"236":8}}],["mnist\\ttraining",{"2":{"228":1}}],["mnist\\ttime",{"2":{"228":50}}],["mnist",{"0":{"190":1,"192":1,"211":1,"213":1,"222":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"191":1,"192":2,"212":1,"213":2,"224":1,"228":2}}],["mse",{"2":{"234":3}}],["mseloss",{"2":{"63":4,"77":2,"78":1,"106":1,"107":1,"179":1,"202":1,"234":1,"244":1,"257":3}}],["msleloss",{"2":{"63":2}}],["my",{"2":{"196":1}}],["mybias",{"2":{"137":3}}],["myinputtype",{"2":{"113":3}}],["myfancychain",{"2":{"48":2}}],["myweight",{"2":{"38":1,"137":4}}],["m2",{"2":{"44":3}}],["mpi",{"0":{"163":1},"2":{"31":2,"32":8,"123":2,"124":1,"163":5}}],["mpibackend",{"2":{"31":2,"32":2}}],["mₘ",{"2":{"23":2}}],["m₂",{"2":{"23":2,"254":7}}],["m₁",{"2":{"23":2,"254":6}}],["m",{"2":{"19":2,"44":3,"48":2,"66":6,"72":2,"214":2,"218":2,"248":15,"254":4,"255":5,"256":3}}],["mkl",{"2":{"18":1}}],["mldatasets",{"2":{"191":1,"212":1,"223":1}}],["mldatadevices",{"0":{"0":1,"93":1},"1":{"1":1,"2":1,"3":1,"4":1,"5":1,"94":1,"95":1},"2":{"0":1,"1":1,"2":3,"3":12,"4":1,"5":2,"93":2,"132":2,"164":2,"180":3,"189":3,"197":3,"199":1,"201":3,"204":3,"221":3,"229":3,"231":1,"238":3,"246":3,"251":3,"260":3}}],["ml",{"2":{"171":1,"202":2}}],["mlir",{"2":{"106":1}}],["mlp",{"0":{"239":1},"1":{"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"69":1,"78":1,"106":1,"226":1,"233":4,"239":1}}],["mlutils",{"0":{"36":1},"2":{"5":3,"36":4,"95":1,"120":1,"160":2,"182":1,"183":3,"191":1,"199":1,"201":1,"212":1,"223":1,"231":1}}],["motion",{"2":{"254":1,"255":1,"256":1}}],["motivating",{"2":{"8":1,"68":1}}],["moment",{"2":{"147":1,"254":1}}],["momentum=0",{"2":{"55":2}}],["momentum",{"2":{"19":6,"55":2}}],["mooncake",{"2":{"102":1}}],["monolithic",{"2":{"83":1}}],["month",{"2":{"74":1}}],["mouthful",{"2":{"41":1}}],["moved",{"0":{"98":1},"2":{"90":1,"98":2,"106":1}}],["move",{"2":{"37":1,"106":1,"107":2,"172":1,"173":1,"201":3}}],["movement",{"2":{"3":2}}],["modivations",{"2":{"97":1}}],["modified",{"2":{"65":2,"115":1}}],["modify",{"2":{"55":1}}],["modulating",{"2":{"63":1}}],["modules=",{"2":{"58":2}}],["modules",{"2":{"58":8}}],["module",{"0":{"64":1},"2":{"30":1,"37":1,"58":1,"64":1,"97":1,"98":1}}],["modes",{"2":{"20":1,"56":1,"70":1}}],["mode",{"0":{"142":1},"2":{"17":2,"19":2,"20":5,"22":2,"40":7,"50":6,"52":2,"55":3,"56":5,"59":1,"70":2,"98":1,"102":1,"103":1,"108":3,"109":2,"110":2,"142":4,"145":2,"155":1,"156":1,"161":1,"175":1,"176":4,"177":2,"254":1}}],["model`",{"2":{"214":1}}],["models",{"0":{"44":2,"46":1,"106":1,"108":1,"118":1,"198":1,"209":1},"1":{"107":1,"109":1,"110":1,"111":1,"199":1,"200":1,"201":1,"202":1,"203":1,"204":1},"2":{"8":1,"25":1,"44":2,"46":2,"57":1,"62":2,"69":2,"77":1,"83":6,"106":1,"108":2,"111":1,"114":1,"153":1,"168":1,"171":1,"177":1,"181":1,"185":1,"198":1,"209":2}}],["model",{"0":{"109":1,"125":1,"187":1,"188":1,"193":1,"196":1,"202":1,"252":1,"255":1,"256":1},"1":{"126":1,"127":1,"128":1,"129":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"8":8,"39":1,"40":2,"41":2,"44":6,"46":11,"48":16,"52":3,"54":12,"55":4,"62":5,"63":26,"68":3,"69":14,"77":6,"78":10,"81":1,"106":18,"107":8,"108":3,"109":17,"110":25,"111":1,"114":1,"115":1,"116":2,"117":2,"118":2,"120":2,"125":1,"126":6,"129":7,"135":1,"136":5,"139":9,"140":11,"142":1,"143":1,"145":10,"146":7,"147":7,"148":7,"150":2,"151":2,"152":9,"153":4,"155":6,"179":11,"184":2,"185":1,"186":2,"187":10,"188":6,"190":1,"193":4,"194":2,"195":7,"196":4,"202":9,"203":2,"209":1,"214":11,"215":8,"216":2,"217":10,"218":8,"220":17,"226":4,"227":2,"228":9,"234":4,"235":1,"236":7,"244":2,"245":3,"249":7,"250":1,"254":7,"255":4,"256":11,"257":1,"258":1,"259":2}}],["mostly",{"2":{"14":1,"83":1,"85":1,"97":1,"124":1,"214":1}}],["most",{"2":{"7":1,"8":1,"20":1,"28":1,"40":1,"62":2,"65":1,"71":1,"83":1,"93":1,"101":1,"103":2,"107":1,"133":1,"139":1,"148":1,"150":1,"153":1,"210":1,"256":1}}],["more",{"2":{"5":1,"7":1,"8":1,"11":1,"14":1,"28":2,"44":1,"46":2,"48":1,"56":1,"59":1,"64":1,"65":1,"68":1,"69":3,"83":1,"84":1,"87":1,"90":1,"92":1,"95":1,"97":1,"99":2,"123":1,"145":1,"146":1,"155":1,"157":1,"162":1,"165":1,"166":1,"171":3,"175":1,"184":1,"185":1,"186":1,"198":1,"250":1}}],["mechanics",{"2":{"254":1}}],["mechanism",{"2":{"83":1}}],["message",{"2":{"144":1}}],["mersennetwister",{"2":{"116":1,"168":2,"241":1}}],["merged",{"2":{"52":1}}],["merge",{"2":{"38":1,"39":1,"52":2,"115":1,"184":1}}],["mentioned",{"2":{"79":1,"135":1}}],["measure",{"2":{"63":1}}],["meanpool",{"2":{"51":1}}],["meant",{"2":{"40":1,"62":1,"68":1,"169":1}}],["means",{"2":{"28":1,"37":1,"69":1,"118":1,"135":1,"137":1,"155":1,"171":1,"173":1,"202":2}}],["meaningless",{"2":{"17":1}}],["mean",{"2":{"17":2,"19":13,"28":2,"51":7,"55":13,"63":17,"109":1,"110":2,"126":1,"129":1,"137":1,"179":1,"234":3,"236":12,"249":1,"250":1}}],["medical",{"2":{"63":1}}],["medium",{"2":{"18":1}}],["met",{"2":{"28":1,"40":1}}],["methodinstance",{"2":{"220":3}}],["methoderror",{"2":{"116":1}}],["methods",{"2":{"56":1,"78":1,"131":1,"149":1,"153":1,"169":2,"198":2,"199":1,"202":1,"214":1,"215":1,"224":1,"231":1,"249":1,"254":2}}],["method",{"2":{"28":3,"106":1,"110":1,"113":1,"116":1,"150":1,"151":1,"152":1,"183":1,"185":1,"186":1,"192":1,"194":1,"195":1,"213":1,"214":1,"216":1,"225":1,"226":1,"227":1,"234":2,"241":1,"249":1,"250":2,"254":3,"256":1,"257":2}}],["metaldevice",{"2":{"4":2}}],["metal",{"2":{"2":1,"3":2,"26":1,"72":2,"77":1,"78":1,"80":2,"130":4,"164":1}}],["memory=false",{"2":{"52":1}}],["memory=zeros32",{"2":{"52":1}}],["memory",{"2":{"5":1,"16":1,"18":1,"52":20,"180":2,"184":1,"189":2,"197":1,"204":2,"221":2,"229":2,"238":2,"246":2,"251":1,"260":1}}],["mutability",{"0":{"173":1}}],["mutables",{"2":{"140":1}}],["mutable",{"2":{"135":1,"140":1}}],["mutations",{"2":{"81":1,"83":2,"103":1}}],["mutation",{"2":{"16":1,"18":1,"173":2}}],["mutating",{"2":{"16":1,"18":1}}],["mutate",{"2":{"173":1}}],["mutated",{"2":{"15":1,"65":1,"173":2,"174":1}}],["mutates",{"2":{"8":1}}],["much",{"2":{"63":2,"83":1,"121":1,"250":1}}],["mul",{"2":{"14":2,"158":1}}],["multilayer",{"2":{"239":1}}],["multigate",{"2":{"64":1}}],["multiply",{"2":{"171":1}}],["multiplied",{"2":{"17":1}}],["multiplication",{"2":{"14":3}}],["multiple",{"0":{"146":1},"2":{"16":1,"18":1,"48":1,"52":1,"55":1,"130":1,"153":2,"250":1}}],["multi",{"0":{"4":1},"2":{"53":1,"63":1}}],["must",{"2":{"1":1,"3":2,"4":1,"7":1,"19":1,"23":2,"28":4,"32":1,"36":2,"39":2,"41":2,"44":1,"46":2,"48":3,"49":2,"50":3,"52":7,"53":4,"55":1,"56":1,"62":3,"70":1,"82":4,"83":1,"120":1,"135":1,"138":1,"244":1,"254":4}}],["major",{"0":{"88":1,"91":1,"95":1},"2":{"85":1,"92":1}}],["masses",{"2":{"254":1}}],["mass2",{"2":{"254":5}}],["mass1",{"2":{"254":6}}],["mass",{"2":{"254":14,"255":2,"256":1,"257":1,"259":1}}],["mass=1",{"2":{"254":3}}],["massachusetts",{"2":{"74":1}}],["master",{"2":{"120":1}}],["mask",{"2":{"17":9,"50":10}}],["maeloss",{"2":{"63":2}}],["made",{"2":{"62":1,"85":1,"90":1,"93":1,"135":1,"144":1}}],["making",{"2":{"62":1,"81":1,"83":1,"93":1,"106":1,"153":1}}],["makes",{"2":{"39":1,"52":1,"63":1,"83":1,"100":1,"133":1}}],["make",{"2":{"10":2,"41":1,"52":1,"62":1,"65":6,"81":2,"83":1,"109":1,"121":1,"135":1,"137":1,"139":1,"145":1,"146":1,"153":1,"161":1,"171":1,"175":1,"183":1,"249":1,"256":2}}],["macro",{"2":{"58":1,"59":1,"69":4,"78":2,"106":1,"133":2,"185":1,"214":2}}],["machines",{"2":{"63":2}}],["machine",{"2":{"17":1,"19":1,"28":1,"149":1,"171":1,"196":1,"254":1}}],["magnitude",{"2":{"55":2}}],["maxiters",{"2":{"236":4}}],["maxiters=1000",{"2":{"258":1}}],["maxiters=500",{"2":{"202":1}}],["maxiters=epochs",{"2":{"202":1}}],["maximum",{"2":{"16":1,"18":1,"54":1,"235":2,"237":1}}],["maxout",{"2":{"54":5}}],["max",{"2":{"51":4,"54":1,"63":2,"235":8,"237":1,"250":1}}],["maxpool",{"2":{"46":2,"51":1,"193":6}}],["margin=2",{"2":{"63":1}}],["margin−y^",{"2":{"63":1}}],["margin",{"2":{"63":2}}],["marker=",{"2":{"255":1,"256":2,"259":4}}],["markersize=16",{"2":{"248":2}}],["markersize=12",{"2":{"241":1,"245":2,"255":1,"256":2,"259":4}}],["marked",{"2":{"5":1,"66":3,"69":1}}],["mark",{"2":{"69":2,"256":1}}],["marks",{"2":{"58":2,"93":1}}],["martens",{"2":{"28":1}}],["main",{"0":{"28":1},"2":{"57":1,"117":1,"118":1,"135":1,"160":1,"184":1,"187":3,"201":2,"214":1,"220":18,"233":1,"245":2}}],["maintaining",{"2":{"19":1}}],["matches",{"2":{"186":2}}],["matched",{"2":{"116":5}}],["matching",{"0":{"67":1},"2":{"116":1}}],["match",{"2":{"39":1,"52":5,"67":4,"138":2,"155":1,"165":1}}],["matrices",{"2":{"18":1,"53":1}}],["matrix",{"2":{"5":3,"14":3,"15":1,"18":2,"28":13,"53":6,"54":1,"63":6,"69":4,"78":1,"106":2,"109":5,"110":13,"116":3,"117":1,"118":1,"131":1,"146":1,"147":1,"148":1,"149":5,"153":1,"168":4,"171":13,"175":3,"201":2}}],["matmul",{"2":{"14":2,"150":1,"151":1,"158":1}}],["maps",{"2":{"28":2,"51":3,"52":2,"63":1}}],["mapping",{"2":{"28":7,"63":1}}],["map",{"0":{"39":1},"2":{"7":1,"39":4,"52":4,"65":2,"97":2,"98":1,"126":2,"127":1,"153":2,"250":1}}],["managing",{"0":{"174":1},"2":{"174":1}}],["management",{"0":{"130":1,"131":1,"132":1},"1":{"131":1,"132":1},"2":{"93":1,"130":1,"131":1}}],["manager",{"2":{"76":1}}],["manage",{"2":{"69":1}}],["many",{"2":{"146":1,"171":1,"175":1}}],["mandatorily",{"2":{"135":1}}],["manual",{"0":{"132":1},"2":{"8":1,"24":1,"38":1,"68":1,"111":1,"125":1,"144":1,"162":1,"172":1}}],["manually",{"2":{"2":1,"44":1,"46":1,"120":1,"133":1,"172":1}}],["manipulation",{"2":{"7":2,"83":1}}],["maybe",{"2":{"67":1,"175":1}}],["may",{"2":{"3":1,"56":2,"69":3,"196":1,"210":1}}],["min",{"2":{"235":10,"237":2}}],["minibatching",{"2":{"198":1,"202":1}}],["minibatch",{"2":{"183":1,"192":1,"213":1}}],["mini",{"2":{"177":1}}],["minimum",{"2":{"235":4,"237":2}}],["minimally",{"2":{"252":1}}],["minimal",{"2":{"124":1}}],["minimized",{"2":{"257":1}}],["minimize",{"2":{"18":1}}],["minimizes",{"2":{"16":1,"179":1}}],["minor",{"2":{"144":1,"153":1}}],["mig",{"2":{"180":1,"189":1,"204":1,"221":1,"246":1}}],["migrating",{"0":{"139":1},"1":{"140":1,"141":1,"142":1,"143":1}}],["migration",{"0":{"121":1},"1":{"122":1,"123":1},"2":{"135":1}}],["might",{"2":{"2":1,"8":1,"13":1,"15":1,"44":1,"46":1,"48":1,"54":1,"62":3,"68":2,"69":1,"105":1,"106":1,"107":1,"111":1,"130":1,"135":2,"144":1,"145":2,"155":1,"159":1,"167":2,"171":1,"173":1,"196":1,"202":1,"219":1}}],["miopen",{"2":{"100":1}}],["milletari",{"2":{"63":1}}],["mirza",{"2":{"54":1}}],["mixing",{"2":{"23":1}}],["mixed",{"2":{"16":1,"22":2}}],["mistakes",{"2":{"83":1}}],["mismatch",{"0":{"109":1},"2":{"67":3,"123":1,"155":1,"165":1}}],["misc",{"0":{"54":1}}],["miscellaneous",{"0":{"3":1,"70":1},"2":{"48":1}}],["missing",{"2":{"19":1,"32":1,"55":1}}],["s3",{"2":{"259":2}}],["s0025",{"2":{"254":2}}],["s2",{"2":{"245":2,"256":2,"259":2}}],["s1",{"2":{"245":2,"256":2,"259":2}}],["sgd",{"2":{"179":1}}],["sz",{"2":{"115":2}}],["szegedy",{"2":{"19":1}}],["src",{"2":{"109":1,"116":1,"145":2,"220":3,"236":4}}],["srivastava",{"2":{"17":1}}],["square",{"2":{"171":2,"235":1}}],["squaredhingeloss",{"2":{"63":2}}],["squared",{"2":{"63":3,"179":1}}],["sqrt",{"2":{"28":4,"49":2,"52":6,"53":3,"249":1,"255":2}}],["sm",{"2":{"180":1,"189":1,"204":1,"221":1,"229":2,"238":2,"246":1}}],["smarter",{"2":{"235":1}}],["smatrix",{"2":{"153":4}}],["smaller",{"2":{"83":1}}],["small",{"2":{"18":1,"46":1,"83":1,"106":1,"155":1,"171":1,"190":1,"202":1}}],["smodel",{"2":{"145":3,"146":4,"147":2,"148":2,"150":2,"151":2,"152":2,"202":3}}],["smooth",{"2":{"63":2}}],["smoothing=0",{"2":{"63":3}}],["smoothing",{"2":{"63":16}}],["switch",{"2":{"50":3,"93":1,"144":1,"175":1}}],["switching",{"0":{"43":1},"1":{"44":1},"2":{"144":4,"162":2}}],["skipped",{"2":{"48":1,"254":1}}],["skip",{"2":{"48":1,"59":3,"106":1,"235":1}}],["skipconnection",{"2":{"48":4}}],["speedup",{"2":{"196":2}}],["spectralnorm",{"2":{"83":1}}],["specify",{"2":{"50":2,"56":1,"63":1,"69":1,"128":1,"249":1}}],["specifying",{"2":{"28":1,"49":2,"51":3,"55":2,"58":1,"69":1}}],["specifies",{"2":{"49":4,"51":3,"54":1,"177":1}}],["specified",{"2":{"2":1,"28":2,"48":8,"53":1,"54":3,"56":1,"59":1,"131":1,"169":1}}],["specifically",{"2":{"142":1,"171":1}}],["specifications",{"2":{"111":1}}],["specification",{"0":{"109":1},"2":{"249":1}}],["specific",{"2":{"4":2,"8":1,"53":1,"65":1,"104":1,"164":1}}],["specialized",{"2":{"13":1,"20":1,"69":1,"153":1,"173":1}}],["special",{"2":{"3":2,"18":2,"34":2,"52":1,"63":1,"69":1,"104":1,"220":3}}],["spiralclassifiercompact",{"2":{"185":2,"187":1}}],["spiralclassifier",{"2":{"184":5,"187":1}}],["spiral",{"2":{"183":1}}],["spirals",{"2":{"181":1,"183":6}}],["spiritual",{"2":{"119":1}}],["spurious",{"0":{"155":1}}],["splatted",{"2":{"69":1}}],["splatting",{"2":{"69":1}}],["split=",{"2":{"192":1,"213":1}}],["splitobs",{"2":{"183":1,"192":1,"212":1,"213":1}}],["split",{"2":{"64":1,"120":2,"183":1,"184":1,"192":2,"213":2}}],["spliced",{"2":{"52":2}}],["space",{"2":{"52":6}}],["spatial",{"2":{"49":5,"51":6,"235":1}}],["sparsity=0",{"2":{"28":1}}],["sparsity",{"2":{"28":4}}],["sparsely",{"2":{"28":2,"53":1}}],["sparse",{"2":{"28":2}}],["scientific",{"2":{"254":1}}],["scientificmachinelearning",{"2":{"252":1}}],["scimlsensitivity",{"2":{"199":1,"212":1,"220":26,"253":1}}],["sciml",{"2":{"68":1,"69":1,"78":1,"83":2,"177":1}}],["scatter",{"2":{"241":1,"245":2,"248":2,"255":1,"256":2,"259":4}}],["scalable",{"2":{"150":1}}],["scalar",{"0":{"156":1},"2":{"3":2,"20":1,"107":1,"156":1,"176":1,"233":1}}],["scale=ones32",{"2":{"55":4}}],["scaled",{"2":{"28":3}}],["scale",{"2":{"19":15,"53":2,"55":20,"56":6,"110":1,"129":1,"145":1}}],["scaling",{"2":{"17":2,"28":2,"254":6}}],["scratch",{"2":{"179":1,"184":1,"211":1}}],["schwarzschild",{"2":{"255":1}}],["school",{"2":{"74":1}}],["schemes",{"2":{"25":1,"168":1}}],["score",{"2":{"63":1}}],["slow",{"2":{"145":2}}],["slower",{"2":{"108":1}}],["slices",{"2":{"64":1}}],["slice",{"2":{"19":2,"28":1,"52":2,"55":2}}],["slightly",{"2":{"48":1}}],["slight",{"2":{"13":1,"175":1}}],["sleefpirates",{"2":{"13":2}}],["s",{"2":{"10":1,"20":1,"48":1,"50":2,"53":1,"55":3,"56":1,"67":3,"68":1,"69":1,"72":1,"77":1,"83":1,"106":2,"107":1,"109":1,"110":1,"121":2,"126":1,"135":1,"136":1,"140":1,"144":2,"145":3,"147":1,"148":1,"149":1,"152":3,"153":1,"155":1,"170":4,"171":5,"176":1,"179":2,"184":5,"186":1,"188":1,"198":1,"202":2,"217":1,"241":3,"242":1,"245":1,"249":2,"255":3}}],["syntax",{"2":{"48":1,"69":1,"171":2,"172":1}}],["synchronized",{"2":{"120":2}}],["synchronize",{"2":{"34":2,"120":6,"121":2,"123":2,"153":2}}],["symbol",{"2":{"10":1,"40":1,"55":1,"69":1,"106":1,"137":1,"188":1,"220":29,"249":2}}],["systems",{"2":{"17":1,"77":1,"83":2,"167":1}}],["system",{"2":{"2":1,"3":1,"106":1,"130":2,"137":1,"175":1,"230":1,"255":1,"256":1}}],["safely",{"2":{"256":1}}],["saveat=tsteps",{"2":{"255":1,"256":1,"257":1,"259":1}}],["saveat=t",{"2":{"200":1,"202":1,"203":1}}],["save",{"2":{"188":4,"215":2,"220":26}}],["saving",{"0":{"188":1},"2":{"188":1}}],["sake",{"2":{"184":1,"186":1,"235":1}}],["sarray",{"2":{"153":7}}],["sarrays",{"2":{"153":1}}],["sanity",{"2":{"152":1}}],["sampling",{"2":{"235":1,"247":1}}],["sampler",{"2":{"249":1}}],["sampled",{"2":{"249":2}}],["samples",{"2":{"179":15,"249":2,"250":2}}],["sample",{"2":{"149":3,"152":1,"249":3,"250":1}}],["samepad",{"2":{"49":2,"51":3}}],["same",{"0":{"118":1},"2":{"3":1,"5":2,"7":3,"13":2,"15":1,"17":1,"19":3,"22":2,"28":1,"39":1,"40":1,"41":1,"48":1,"49":2,"51":3,"52":3,"54":2,"55":1,"62":2,"64":2,"67":2,"71":1,"78":1,"82":2,"106":1,"120":1,"136":1,"146":1,"152":2,"153":1,"172":1,"174":1,"178":1,"187":1,"190":1,"214":1}}],["saw",{"2":{"107":1}}],["say",{"2":{"83":1,"126":1}}],["satisfactory",{"2":{"250":1}}],["satisfying",{"2":{"49":2,"51":6,"135":1}}],["satisfies",{"2":{"46":2,"54":1}}],["satisfied",{"2":{"8":1,"244":1}}],["said",{"2":{"37":1}}],["saxe",{"2":{"28":1}}],["sorted",{"2":{"84":1}}],["soln",{"2":{"254":13,"255":2,"256":2,"259":2}}],["soln2orbit",{"2":{"254":3}}],["solution",{"2":{"235":5,"237":1}}],["solutions",{"2":{"28":1}}],["sol",{"2":{"203":2}}],["solver",{"2":{"214":6,"218":3,"220":7}}],["solver=tsit5",{"2":{"214":2,"218":1}}],["solvers",{"2":{"83":1}}],["solves",{"2":{"202":1}}],["solve",{"2":{"200":1,"202":4,"203":1,"214":2,"218":1,"230":1,"255":1,"256":1,"257":1,"258":1,"259":1}}],["society",{"2":{"63":1}}],["software",{"2":{"74":2}}],["softmax",{"2":{"63":5}}],["softfail",{"2":{"60":2}}],["soft",{"2":{"59":4}}],["sooner",{"2":{"37":1}}],["so",{"2":{"13":1,"77":1,"140":1,"144":3,"145":1,"146":1,"201":1,"214":2,"218":2,"232":1,"233":1,"244":1}}],["sometimes",{"2":{"171":2}}],["somewhere",{"2":{"44":1,"46":1}}],["some",{"0":{"200":1,"254":1},"2":{"8":1,"40":1,"44":2,"52":1,"54":1,"69":1,"83":1,"85":1,"97":1,"101":1,"103":1,"104":1,"105":1,"108":1,"110":1,"119":1,"123":1,"128":1,"136":1,"140":1,"144":7,"145":1,"155":1,"158":1,"160":2,"171":1,"217":1,"225":1,"235":1}}],["source",{"2":{"1":1,"2":3,"3":8,"4":2,"5":1,"7":3,"8":7,"9":2,"10":5,"11":1,"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"20":1,"22":2,"23":1,"28":8,"29":24,"31":2,"32":3,"33":2,"34":4,"35":1,"36":1,"38":5,"39":1,"40":2,"41":1,"44":3,"46":3,"48":6,"49":2,"50":3,"51":9,"52":6,"53":4,"54":7,"55":5,"56":2,"58":2,"59":2,"60":1,"62":6,"63":15,"64":8,"65":6,"66":3,"67":1,"68":1,"69":3,"70":1,"90":1,"170":2}}],["situations",{"2":{"124":1}}],["siamese",{"2":{"63":1}}],["siamesecontrastiveloss",{"2":{"63":3}}],["sig",{"2":{"249":2}}],["sigmoid",{"2":{"63":3,"184":1,"185":1,"249":2}}],["signify",{"2":{"85":1,"92":1}}],["significantly",{"2":{"133":1,"217":1}}],["significant",{"2":{"8":1,"196":1,"219":1}}],["signature",{"2":{"17":1,"39":1}}],["si+1",{"2":{"49":1,"51":3}}],["silently",{"2":{"41":1}}],["sided",{"2":{"254":2}}],["side",{"2":{"16":1,"69":1}}],["size=1000",{"2":{"183":1}}],["sizes",{"2":{"18":1,"28":2,"90":1}}],["sized",{"2":{"18":1,"19":1,"64":1}}],["size",{"0":{"11":1},"2":{"11":2,"13":2,"19":3,"23":2,"28":24,"29":72,"44":1,"46":1,"48":1,"49":16,"51":48,"52":11,"53":16,"54":8,"55":7,"56":8,"69":6,"100":1,"115":1,"150":2,"151":2,"153":2,"171":1,"179":2,"180":1,"183":7,"189":1,"192":3,"197":1,"204":1,"213":3,"214":2,"217":1,"221":1,"229":1,"238":1,"246":1,"249":4,"250":1,"251":1,"254":4,"260":1}}],["sin",{"2":{"254":1}}],["singular",{"0":{"135":1},"2":{"135":1,"136":2}}],["singleton",{"2":{"7":3,"49":2,"51":3,"90":2,"91":1}}],["single",{"2":{"7":2,"8":1,"15":1,"16":1,"49":6,"51":9,"52":13,"62":5,"63":1,"69":1,"77":2,"78":1,"90":1,"107":3,"136":1,"140":1,"146":1,"149":1,"153":4,"171":1,"179":1,"187":1,"195":1,"217":1,"228":1,"236":1,"245":1,"249":1}}],["since",{"2":{"8":1,"28":1,"39":1,"69":1,"71":1,"83":1,"90":1,"107":1,"113":1,"120":1,"137":1,"177":1,"184":1,"186":1,"202":2,"232":1}}],["simultaneously",{"2":{"137":1}}],["simulate",{"2":{"110":1,"254":1,"255":1,"256":2}}],["simulating",{"0":{"255":1},"2":{"28":1}}],["simd",{"2":{"20":1}}],["simplicity",{"2":{"149":1,"186":1,"235":1}}],["simplified",{"2":{"135":1}}],["simplest",{"2":{"48":1}}],["simplechainslayer",{"2":{"46":2,"193":1}}],["simplechains",{"0":{"190":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1},"2":{"46":14,"83":1,"190":3,"191":2,"193":1,"196":2}}],["simple",{"0":{"46":1,"181":1},"1":{"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1},"2":{"17":2,"20":1,"46":4,"52":1,"62":1,"69":1,"106":1,"131":2,"140":1,"142":1,"170":1,"176":1,"193":1,"196":1,"198":1}}],["simply",{"2":{"8":1,"13":1,"62":1,"66":1,"67":1,"69":1,"71":1,"107":1,"108":1,"118":1,"128":1,"137":1,"225":1,"249":1}}],["similarly",{"2":{"136":1}}],["similarity",{"2":{"5":1}}],["similar",{"2":{"2":1,"3":1,"7":1,"19":1,"52":1,"53":1,"63":1,"64":1,"65":1,"106":1,"107":1,"137":1,"139":1,"160":1,"175":1,"256":1}}],["ship",{"2":{"108":1}}],["shifted",{"2":{"28":1}}],["shift",{"2":{"19":1,"28":4,"55":4}}],["shuffle=true",{"2":{"183":2,"192":1,"213":1,"224":1,"236":2}}],["shuffle=false",{"2":{"5":1,"183":1,"192":1,"213":1,"224":1}}],["shuffle",{"2":{"56":1,"183":2,"192":2,"213":2}}],["shuffling",{"2":{"56":1}}],["shapedaxis",{"2":{"220":30}}],["shaped",{"2":{"51":9}}],["shape",{"2":{"49":1,"52":16,"54":1,"55":17,"109":4,"171":1,"179":4}}],["shate",{"2":{"41":1}}],["sharing",{"2":{"41":9}}],["share",{"2":{"41":4}}],["shared",{"2":{"19":2,"41":1,"93":1}}],["shooting",{"2":{"202":1}}],["shortcomings",{"0":{"124":1}}],["shortcut",{"2":{"8":1,"48":1}}],["short",{"2":{"52":1,"175":1}}],["shorthand",{"2":{"8":1}}],["shown",{"2":{"249":1}}],["showcasing",{"2":{"210":1}}],["showing",{"2":{"152":1}}],["shows",{"2":{"137":1,"148":1,"149":1,"152":1,"250":1}}],["showerror",{"2":{"116":1}}],["show",{"2":{"5":2,"77":1,"145":1,"172":1,"198":1,"247":1}}],["shouldn",{"2":{"62":1,"90":1,"137":1,"147":1}}],["should",{"0":{"113":1},"2":{"4":2,"15":1,"20":1,"31":2,"38":1,"41":1,"46":1,"49":5,"51":6,"52":1,"55":1,"56":1,"62":2,"63":1,"65":1,"69":1,"93":1,"94":1,"125":1,"130":1,"135":2,"136":2,"137":1,"139":1,"140":2,"155":1,"173":1,"174":1,"211":1,"242":1,"256":1}}],["stencils",{"2":{"254":2}}],["stepnorm=0",{"2":{"258":1}}],["steprangelen",{"2":{"201":2}}],["steps",{"2":{"62":1,"77":1,"107":1,"249":1}}],["step",{"2":{"62":10,"77":1,"78":1,"107":3,"179":1,"187":1,"195":1,"217":1,"228":1,"236":1,"245":2,"249":3}}],["stime",{"2":{"195":2,"217":2,"228":2}}],["stick",{"2":{"175":1}}],["still",{"0":{"143":1},"2":{"2":1,"8":1,"46":1,"109":1,"118":1,"130":1,"135":1,"136":1,"184":1,"220":1,"242":1,"250":1}}],["stop=6",{"2":{"250":4}}],["stochastic",{"2":{"82":1,"83":1,"179":1}}],["stores",{"2":{"50":1,"53":1,"68":1,"123":1,"140":1,"171":1,"214":1}}],["stored",{"2":{"49":1,"62":3}}],["store",{"2":{"34":2,"53":1,"82":2,"140":1,"202":1,"248":1,"256":1,"257":1}}],["storing",{"2":{"19":1,"117":1,"140":1}}],["std=1e",{"2":{"256":3}}],["std=0",{"2":{"28":1}}],["stdlib",{"2":{"174":1}}],["stdout",{"2":{"116":1}}],["std",{"2":{"19":1,"28":5,"249":1}}],["stylization",{"2":{"19":1,"55":1}}],["stage",{"2":{"144":1}}],["start=false",{"2":{"215":1}}],["start",{"2":{"106":1,"170":1,"220":13,"247":1}}],["starting",{"2":{"88":1,"130":1,"171":1,"218":1}}],["started",{"0":{"75":1,"84":1},"1":{"76":1,"77":1,"78":1,"79":1,"80":1},"2":{"139":1}}],["stacktraces",{"2":{"108":1}}],["stacktrace",{"2":{"97":2,"109":1}}],["stack",{"2":{"52":2,"235":6,"237":2}}],["stackedrnncells",{"2":{"52":1}}],["standard",{"2":{"28":3,"29":6,"49":2,"55":1,"77":1,"145":1,"256":1}}],["stabilities",{"2":{"219":1}}],["stability",{"0":{"220":1},"2":{"8":1,"19":4,"55":4,"63":1,"69":1,"85":1,"92":1,"166":1}}],["stablerng",{"2":{"145":3,"146":3,"147":2,"148":2,"152":3}}],["stablerngs",{"2":{"26":1,"144":1}}],["stable",{"2":{"62":1,"65":5,"69":2,"166":1,"186":1,"214":1,"220":1}}],["stats",{"2":{"62":2,"77":2,"107":1,"236":7}}],["stats=false",{"2":{"55":5}}],["stats=true",{"2":{"48":2,"55":7,"109":1,"110":2}}],["staticint",{"2":{"193":2}}],["staticarrayscore",{"2":{"153":4}}],["staticarrays",{"2":{"153":1}}],["staticsymbol",{"2":{"40":1,"64":2}}],["static",{"2":{"20":1,"46":1,"64":1,"68":1,"77":8,"106":3,"116":2,"153":7,"193":4,"220":58,"245":2}}],["staticbool",{"2":{"17":1,"40":1,"64":1}}],["statistics",{"2":{"19":2,"28":2,"55":7,"62":2,"100":1,"182":1,"191":1,"212":1,"223":1,"231":1,"240":1,"244":1,"249":1}}],["statements",{"2":{"69":1}}],["statefulneuralode",{"2":{"218":4,"219":1,"220":5}}],["statefulluxlayers",{"2":{"144":1}}],["statefulluxlayer",{"2":{"68":1,"98":4,"144":7,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"202":2,"218":2,"234":7,"236":2,"249":2,"256":2}}],["statefulrecurrentcell",{"2":{"52":2}}],["stateful",{"0":{"68":1,"218":1,"219":1},"2":{"52":2,"145":1,"146":1,"148":2,"220":6}}],["state=false",{"2":{"52":4}}],["state=zeros32",{"2":{"52":3}}],["stateless",{"2":{"8":1,"54":1}}],["statelength",{"2":{"7":1,"10":1,"135":4,"136":1}}],["state",{"0":{"138":1},"2":{"7":1,"8":3,"10":5,"17":2,"19":1,"38":1,"44":1,"48":6,"50":4,"52":64,"54":1,"55":4,"62":7,"68":4,"69":7,"77":8,"78":4,"82":3,"83":3,"107":4,"120":5,"135":5,"136":3,"138":4,"140":1,"184":3,"187":7,"195":8,"202":6,"228":11,"236":6,"244":1,"249":2,"256":1}}],["states",{"0":{"10":1},"2":{"3":1,"7":4,"8":1,"10":2,"38":4,"39":6,"40":1,"41":1,"44":1,"46":1,"48":16,"50":3,"52":5,"54":2,"55":7,"62":2,"63":2,"66":2,"67":4,"68":1,"69":5,"77":2,"78":2,"90":2,"107":1,"109":2,"110":1,"115":1,"120":2,"126":3,"129":1,"133":1,"135":5,"136":3,"138":1,"140":3,"179":2,"184":2,"187":2,"188":1,"193":2,"195":2,"214":1,"217":2,"228":4,"236":1,"242":1,"244":1,"245":4,"249":1,"256":1}}],["st",{"2":{"7":2,"8":4,"10":6,"38":2,"39":6,"41":1,"44":4,"46":2,"48":6,"52":1,"54":14,"62":1,"63":2,"64":2,"67":1,"68":4,"69":19,"77":4,"78":6,"98":2,"106":14,"107":6,"109":5,"110":7,"113":3,"115":7,"116":2,"117":10,"118":2,"120":2,"126":9,"127":7,"128":6,"129":2,"135":7,"136":11,"137":3,"138":2,"139":3,"140":6,"145":7,"146":7,"147":6,"148":6,"150":2,"151":2,"152":9,"153":8,"155":5,"179":6,"184":10,"186":4,"187":7,"188":3,"194":4,"195":2,"202":5,"214":4,"215":4,"216":4,"217":2,"218":7,"220":16,"227":4,"228":2,"234":7,"236":3,"244":1,"245":1,"249":2,"256":2}}],["strain",{"2":{"254":5}}],["strokewidth=2",{"2":{"241":1,"245":2,"248":2,"256":2,"259":4}}],["strokecolor=",{"2":{"241":1,"245":2,"248":2}}],["strongly",{"2":{"173":1}}],["struct",{"2":{"115":1,"133":1,"135":1,"140":3,"188":1,"201":1}}],["structs",{"2":{"66":3}}],["structures",{"2":{"65":2,"83":1,"155":1}}],["structured",{"2":{"52":3,"137":1,"225":1}}],["structure",{"2":{"3":5,"7":2,"8":3,"22":2,"34":1,"39":4,"41":1,"53":1,"65":3,"109":5,"110":13,"135":2,"136":2,"138":1,"140":1,"217":1}}],["strength",{"2":{"63":1}}],["stride=window",{"2":{"51":3}}],["stride=1",{"2":{"49":2}}],["stride",{"2":{"49":5,"51":6}}],["stridearray",{"2":{"46":1}}],["stridearrayscore",{"2":{"46":1}}],["strings",{"2":{"98":1}}],["string=",{"2":{"70":2}}],["string",{"2":{"1":2,"3":1,"58":1,"70":1}}],["surprise",{"2":{"155":1}}],["surpassing",{"2":{"28":2}}],["sure",{"2":{"137":1,"256":1}}],["super",{"2":{"254":1}}],["supertype",{"2":{"136":1}}],["suppose",{"2":{"250":1}}],["supposed",{"2":{"83":1,"97":1,"166":1}}],["supporting",{"2":{"103":1}}],["support",{"0":{"4":1,"72":1,"73":1,"80":1,"104":1,"163":1},"2":{"3":4,"4":4,"7":2,"16":2,"18":2,"38":1,"51":3,"63":2,"65":1,"68":1,"77":2,"78":2,"80":5,"83":3,"88":2,"92":1,"101":1,"102":1,"103":1,"105":1,"124":1,"130":2,"137":1,"142":1}}],["supports",{"2":{"3":1,"69":1,"100":1,"142":1,"163":1,"171":1,"172":1,"232":1}}],["supported",{"0":{"26":1},"2":{"2":1,"3":2,"20":1,"22":2,"23":1,"44":1,"51":3,"56":1,"62":6,"67":1,"77":1,"82":2,"92":1,"104":2,"105":1,"130":2,"160":1,"168":2}}],["supplied",{"2":{"32":1,"48":1}}],["supply",{"2":{"32":1}}],["subtracts",{"2":{"171":1}}],["subtract",{"2":{"171":1}}],["subtyping",{"2":{"136":1}}],["subtype",{"2":{"135":1,"136":2}}],["subarray",{"2":{"64":1}}],["suboptimal",{"2":{"62":1,"153":1}}],["sumit",{"2":{"63":1}}],["sum",{"2":{"58":3,"59":2,"69":5,"110":2,"139":2,"140":2,"145":1,"146":1,"147":2,"148":1,"150":1,"151":1,"186":1,"194":1,"216":1,"227":1,"234":3,"254":1}}],["summary",{"2":{"5":8,"140":1,"249":1}}],["suggests",{"2":{"54":1}}],["success",{"2":{"258":1}}],["successor",{"2":{"119":1}}],["successfully",{"2":{"1":1}}],["such",{"2":{"4":2,"6":1,"8":1,"49":2,"51":6,"69":1,"83":1,"106":1,"111":1,"169":1,"185":1,"225":1}}],["seaborn",{"2":{"250":3}}],["seamlessly",{"2":{"62":1}}],["seq",{"2":{"183":1}}],["sequentially",{"2":{"48":2,"52":5}}],["sequences",{"2":{"183":1}}],["sequence=true",{"2":{"52":2}}],["sequence",{"2":{"48":1,"52":9,"183":7,"184":3}}],["separately",{"2":{"172":1}}],["separation",{"2":{"83":1}}],["separating",{"2":{"83":1}}],["several",{"2":{"161":1,"175":1}}],["sensealg=reversediffadjoint",{"2":{"217":1}}],["sensealg=gaussadjoint",{"2":{"217":1}}],["sensealg=interpolatingadjoint",{"2":{"215":1,"217":1}}],["sensealg",{"2":{"215":1,"217":1,"220":13}}],["sensitivities",{"2":{"214":1,"217":1}}],["sensible",{"2":{"83":1}}],["sensibly",{"2":{"64":1}}],["sensical",{"2":{"69":1}}],["send",{"2":{"84":1}}],["sendbuf",{"2":{"34":6}}],["sendrecvbuf",{"2":{"34":6}}],["seyed",{"2":{"63":1}}],["segmentation",{"2":{"63":2}}],["sec",{"2":{"249":1}}],["section",{"2":{"46":1,"48":2,"76":1,"111":1,"146":1,"162":1,"172":1,"254":1}}],["seconds",{"2":{"249":2}}],["second",{"2":{"17":1,"48":1,"171":1,"177":1,"254":2}}],["server",{"2":{"180":1,"189":1,"196":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["serves",{"2":{"44":1}}],["serious",{"2":{"230":1}}],["seriously",{"2":{"77":1}}],["serializes",{"2":{"120":1}}],["serialization",{"2":{"120":1,"188":1}}],["series",{"2":{"72":2}}],["sergey",{"2":{"19":1}}],["semvar",{"2":{"98":1}}],["semver",{"2":{"37":1}}],["semantic",{"2":{"145":2}}],["semantically",{"2":{"48":1}}],["semantics",{"2":{"5":2,"155":1}}],["semi",{"2":{"28":2}}],["self",{"2":{"17":1}}],["selu",{"2":{"17":1}}],["selecting",{"2":{"154":1}}],["selection",{"0":{"164":1},"2":{"2":1,"3":1,"131":1,"132":1,"164":1}}],["selectdim",{"2":{"54":1}}],["select",{"2":{"2":4}}],["selected",{"0":{"210":1},"2":{"2":3,"3":1,"97":1}}],["selects",{"2":{"2":1}}],["seems",{"2":{"258":2}}],["seeding",{"2":{"77":1}}],["seed",{"2":{"48":1,"54":4,"77":1,"78":1,"83":1,"129":1,"135":1,"137":1,"170":1,"174":2,"179":1,"215":1,"236":3,"241":1,"248":1}}],["see",{"2":{"3":1,"7":3,"8":3,"11":1,"14":1,"15":2,"17":2,"19":4,"38":1,"40":2,"41":1,"44":1,"46":2,"48":4,"50":3,"52":5,"54":2,"55":4,"56":1,"59":1,"62":1,"63":1,"65":2,"67":1,"68":2,"69":2,"70":1,"76":1,"83":2,"87":1,"90":3,"92":1,"97":2,"99":2,"106":1,"107":1,"108":1,"109":3,"110":2,"119":1,"123":1,"136":1,"155":2,"159":1,"162":1,"165":1,"166":1,"168":1,"171":2,"172":1,"184":1,"196":1,"219":1,"250":2,"254":2}}],["session",{"2":{"1":1,"70":1}}],["setprogress",{"2":{"247":1}}],["sets",{"2":{"58":1,"166":2}}],["setindexing",{"2":{"16":1,"18":1}}],["setups",{"2":{"148":1}}],["setup",{"2":{"8":1,"39":1,"41":1,"44":1,"46":1,"48":1,"54":4,"69":6,"77":1,"78":1,"106":1,"107":1,"109":2,"110":2,"116":1,"118":1,"120":1,"126":1,"129":1,"135":2,"136":1,"137":1,"139":1,"140":1,"145":1,"146":1,"147":1,"148":1,"152":1,"153":1,"155":1,"179":1,"187":1,"195":1,"202":1,"215":1,"228":1,"236":1,"244":1,"249":1,"256":1}}],["setfield",{"2":{"7":1,"90":1,"223":1}}],["setting",{"0":{"257":1},"2":{"4":2,"49":2,"52":4,"53":3,"110":1,"144":1,"157":1,"166":2,"167":1}}],["set",{"2":{"1":2,"2":1,"4":6,"17":4,"19":4,"20":1,"28":1,"32":2,"38":2,"41":1,"44":2,"46":1,"49":2,"50":1,"52":18,"56":2,"58":1,"59":1,"63":2,"65":1,"68":1,"69":3,"70":5,"73":3,"85":1,"110":2,"123":1,"131":1,"144":1,"145":4,"157":2,"161":2,"162":1,"163":3,"164":3,"166":2,"179":3,"247":1}}],["os",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["oops",{"2":{"155":1}}],["ok",{"2":{"147":1}}],["own",{"2":{"124":1,"137":1,"171":1}}],["oh",{"2":{"110":1}}],["our",{"2":{"77":1,"83":1,"104":1,"110":3,"142":1,"145":1,"149":1,"155":2,"170":1,"171":1,"173":1,"174":1,"176":1,"179":2,"183":1,"184":1,"198":1,"202":3,"230":1,"244":1,"245":1,"248":1,"249":3,"250":4}}],["outside",{"2":{"65":1,"97":2}}],["outperforms",{"2":{"105":1}}],["outpad",{"2":{"49":3,"100":1}}],["outpad=0",{"2":{"49":1}}],["outputs",{"2":{"11":1,"40":1,"46":1,"48":2,"52":6,"54":2,"68":1,"82":1,"107":1,"146":1,"147":1,"174":1,"176":1,"256":1}}],["outputsize",{"2":{"11":3,"90":1}}],["output",{"2":{"11":1,"13":2,"16":1,"17":3,"18":1,"28":6,"40":1,"46":2,"48":7,"49":8,"51":33,"52":13,"53":7,"54":4,"55":1,"56":4,"63":6,"68":1,"69":2,"78":1,"100":1,"106":2,"109":4,"110":14,"135":1,"138":2,"153":13,"155":1,"174":1,"184":1,"214":1,"233":1,"249":2}}],["out",{"2":{"17":2,"28":2,"37":1,"48":1,"49":8,"50":2,"51":3,"52":15,"53":19,"65":1,"69":10,"78":6,"85":2,"108":1,"110":1,"133":1,"135":8,"138":1,"157":1,"184":3,"185":2,"217":1,"247":1}}],["odefunction",{"2":{"214":2,"218":1}}],["odeproblem",{"2":{"200":1,"202":1,"203":1,"214":2,"218":1,"255":1,"256":1,"259":1}}],["odesolution",{"2":{"214":1}}],["odes",{"0":{"211":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1},"2":{"83":1,"114":1,"211":2,"255":1,"256":1}}],["ode",{"0":{"214":1,"215":1,"219":1,"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"68":1,"114":1,"198":3,"200":3,"201":1,"202":2,"203":2,"214":2,"215":1,"255":3,"256":6,"257":1,"259":2}}],["odd",{"2":{"28":1}}],["old",{"2":{"62":1,"87":1,"91":2,"130":2}}],["older",{"0":{"43":1},"1":{"44":1},"2":{"97":1,"123":1}}],["o=σ",{"2":{"52":1}}],["oi=",{"2":{"49":1,"51":3}}],["o",{"2":{"49":4,"51":6}}],["observe",{"2":{"249":1}}],["observations",{"2":{"49":1}}],["obtain",{"2":{"120":1}}],["obtained",{"2":{"20":1}}],["obviously",{"2":{"108":1}}],["obj",{"2":{"62":2,"136":2,"137":1}}],["objects",{"2":{"63":1,"94":3,"106":1,"132":1,"155":1,"201":1}}],["objective",{"2":{"62":8,"257":1}}],["object",{"2":{"2":3,"3":4,"16":1,"22":2,"62":6,"63":2,"120":1,"131":3}}],["occurs",{"2":{"110":1}}],["occurred",{"2":{"110":3}}],["occurrences",{"2":{"10":3}}],["octavian",{"0":{"167":1},"2":{"18":2,"159":1,"167":1}}],["other",{"0":{"29":1,"42":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"5":1,"8":1,"16":1,"28":1,"29":1,"34":2,"46":1,"48":1,"49":1,"54":2,"56":1,"59":1,"63":1,"65":2,"77":1,"78":1,"88":1,"97":1,"105":1,"107":1,"133":1,"135":1,"136":1,"171":1,"175":1,"188":1,"198":1}}],["otherwisewhere",{"2":{"63":1}}],["otherwise",{"2":{"1":1,"2":2,"3":1,"53":1,"70":1}}],["opening",{"2":{"144":1}}],["open",{"2":{"103":1,"106":1,"111":1,"124":1,"153":1,"209":1,"210":2}}],["operator",{"2":{"65":1}}],["operators",{"2":{"54":1}}],["operating",{"2":{"52":1}}],["operation",{"2":{"16":2,"18":1,"20":4,"34":4,"51":3,"56":1,"175":1}}],["operations",{"0":{"14":1,"65":1},"2":{"2":1,"16":3,"18":1,"23":1,"46":1,"64":2,"65":1,"88":1,"144":2,"153":2,"167":1,"172":1}}],["operates",{"2":{"106":1}}],["operate",{"2":{"52":1,"140":1}}],["optprob",{"2":{"258":2}}],["optf",{"2":{"258":2}}],["opt",{"2":{"58":6,"120":5,"179":2,"202":10,"245":1}}],["optimal",{"2":{"202":1}}],["optim",{"2":{"69":2}}],["optimiser",{"0":{"186":1}}],["optimisers",{"2":{"35":1,"44":2,"62":2,"69":3,"77":10,"78":1,"106":1,"120":1,"140":2,"143":1,"179":2,"181":1,"182":1,"191":1,"198":1,"202":2,"212":1,"223":1,"231":1,"236":1,"240":1,"243":1}}],["optimizer",{"0":{"243":1},"2":{"35":4,"62":4,"120":3,"245":2}}],["optimizers",{"0":{"35":1},"2":{"258":1}}],["optimized",{"2":{"20":1,"159":1,"167":1,"202":1}}],["optimizationproblem",{"2":{"202":2,"258":1}}],["optimizationfunction",{"2":{"202":1,"258":1}}],["optimizationoptimjl",{"2":{"199":1,"253":1}}],["optimizationoptimisers",{"2":{"199":1}}],["optimizations",{"2":{"97":1,"106":1}}],["optimization",{"0":{"198":1},"1":{"199":1,"200":1,"201":1,"202":1,"203":1,"204":1},"2":{"28":1,"77":1,"78":3,"106":1,"198":4,"199":1,"202":8,"253":1,"258":4}}],["options",{"2":{"67":1,"164":1}}],["option",{"2":{"20":1,"56":2,"67":2,"103":2}}],["optional",{"0":{"159":1},"2":{"16":1,"18":1,"28":1,"38":3,"52":1,"63":1,"68":1,"69":2,"90":1,"167":1,"169":1}}],["optionally",{"2":{"7":1,"28":1,"55":1,"135":1,"169":1}}],["op",{"2":{"4":2,"5":1,"19":1,"34":6,"58":1,"64":3,"67":1,"95":1}}],["orbit₂",{"2":{"254":2}}],["orbit₁",{"2":{"254":2}}],["orbits",{"2":{"254":1}}],["orbit2",{"2":{"254":4}}],["orbit2tensor",{"2":{"254":2}}],["orbit1",{"2":{"254":4}}],["orbit",{"2":{"254":17}}],["orange",{"2":{"241":1,"245":1}}],["ordinarydiffeqloworderrk",{"2":{"253":1}}],["ordinarydiffeqcore",{"2":{"220":26}}],["ordinarydiffeq",{"2":{"214":1,"255":1}}],["ordinarydiffeqtsit5",{"2":{"199":1,"212":1,"220":13}}],["ordering",{"2":{"52":6}}],["order",{"0":{"24":1},"2":{"2":1,"38":1,"49":1,"102":1,"145":1,"147":1,"177":1,"232":3,"234":1,"254":2}}],["orcjit",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["originates",{"2":{"40":1}}],["originally",{"2":{"252":1}}],["original",{"2":{"28":1,"38":1,"41":1,"44":1,"106":1,"109":1,"173":2}}],["org",{"2":{"28":1,"54":1,"74":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"254":2,"260":1}}],["orthogonal",{"2":{"26":4,"28":6}}],["or",{"2":{"2":3,"3":2,"4":2,"8":1,"16":2,"17":3,"18":1,"19":2,"20":2,"28":2,"34":3,"38":1,"40":3,"44":1,"46":2,"48":7,"49":6,"51":9,"52":3,"53":7,"56":4,"62":1,"63":5,"64":2,"69":3,"70":1,"72":1,"76":1,"77":1,"83":2,"84":1,"103":1,"107":1,"110":2,"123":2,"124":1,"135":2,"136":1,"140":1,"142":1,"144":1,"147":1,"155":1,"158":2,"163":1,"169":1,"171":4,"172":1,"183":1,"209":1,"210":2,"254":1}}],["overview",{"0":{"102":1}}],["overloaded",{"2":{"97":1}}],["overloading",{"2":{"65":1}}],["overcome",{"2":{"44":1}}],["overridden",{"2":{"18":1}}],["overrides",{"2":{"59":1}}],["override",{"2":{"11":1,"44":1}}],["overfitting",{"2":{"17":2}}],["over",{"0":{"39":1,"83":1},"2":{"2":1,"5":2,"39":1,"40":1,"52":1,"54":1,"55":3,"68":1,"77":1,"125":1,"181":1,"201":1,"230":1,"233":1,"245":1,"257":1,"259":1}}],["onlinestats",{"2":{"231":1,"236":4}}],["online",{"2":{"210":1}}],["only",{"2":{"2":1,"8":1,"40":2,"49":1,"50":1,"51":3,"52":9,"53":1,"56":1,"59":3,"62":1,"63":3,"67":2,"69":1,"106":1,"120":1,"128":1,"136":1,"137":1,"144":2,"147":1,"148":1,"153":2,"176":2,"188":1,"213":1,"234":3}}],["once",{"2":{"11":1,"52":2,"67":1,"110":1,"135":1,"172":1,"173":1,"175":1}}],["onwards",{"2":{"7":1,"97":1}}],["on",{"0":{"112":1,"156":1,"222":1,"230":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"238":1},"2":{"2":2,"3":6,"5":1,"6":2,"7":1,"8":2,"13":2,"14":3,"15":1,"16":3,"18":3,"19":4,"24":2,"28":11,"34":2,"37":1,"38":1,"39":2,"46":1,"48":2,"49":2,"50":2,"51":6,"52":2,"53":1,"58":1,"63":5,"65":3,"68":2,"69":2,"70":1,"71":1,"74":1,"76":1,"77":1,"83":1,"84":2,"88":1,"90":2,"98":1,"99":2,"100":1,"103":2,"105":1,"106":2,"111":1,"113":2,"117":1,"120":1,"124":1,"133":2,"135":2,"138":1,"140":4,"144":4,"145":4,"146":1,"147":2,"153":3,"155":2,"156":1,"157":1,"159":1,"160":2,"161":2,"162":1,"165":1,"166":1,"168":1,"170":1,"171":1,"172":1,"174":3,"177":2,"180":1,"189":1,"196":2,"197":1,"198":1,"201":1,"202":2,"204":1,"209":1,"217":2,"221":1,"225":1,"229":1,"235":2,"238":1,"239":1,"246":1,"249":1,"251":1,"260":1}}],["one2two",{"2":{"254":3}}],["onecold",{"2":{"194":2,"216":2,"227":2}}],["onehotbatch",{"2":{"192":1,"195":1,"213":1,"224":2}}],["onehotarrays",{"2":{"191":1,"212":1,"223":1}}],["onearray",{"2":{"26":3}}],["oneapidevice",{"2":{"4":2}}],["oneapi",{"2":{"2":1,"3":2,"26":1,"72":2,"77":1,"78":1,"80":2,"130":2,"164":1}}],["onesc64",{"2":{"29":1}}],["onesc32",{"2":{"29":1}}],["onesc16",{"2":{"29":1}}],["ones64",{"2":{"29":1}}],["ones32",{"2":{"29":1}}],["ones16",{"2":{"29":1}}],["ones",{"2":{"8":1,"29":7,"69":1,"115":1,"171":2,"177":1,"220":1,"235":2,"248":1,"249":1}}],["one",{"2":{"1":1,"40":1,"48":2,"52":1,"55":2,"56":2,"63":1,"66":1,"120":1,"144":1,"158":2,"169":1,"171":1,"209":1,"210":1,"249":1,"254":4}}],["official",{"2":{"154":1,"180":1,"189":1,"197":1,"198":1,"204":1,"210":1,"221":1,"229":1,"238":1,"246":1,"247":2,"251":1,"260":1}}],["off",{"2":{"149":1}}],["offending",{"2":{"110":9}}],["ofcourse",{"2":{"109":1}}],["often",{"2":{"3":1,"53":1,"104":2,"114":1,"138":1,"149":1,"152":1,"155":1,"158":2,"171":2,"198":1,"249":1}}],["of",{"0":{"126":1,"128":1,"129":1,"148":1},"2":{"1":1,"3":7,"4":3,"6":1,"7":2,"8":17,"9":3,"10":6,"11":4,"14":2,"15":2,"16":1,"17":4,"18":1,"19":8,"22":6,"23":6,"28":31,"29":36,"32":2,"33":1,"37":2,"38":6,"39":11,"40":2,"41":9,"44":7,"46":3,"48":32,"49":38,"50":3,"51":42,"52":37,"53":32,"54":23,"55":30,"56":13,"57":1,"58":1,"59":7,"62":11,"63":13,"64":1,"65":10,"66":4,"67":10,"68":8,"69":16,"71":1,"74":3,"76":1,"78":2,"83":5,"85":3,"90":2,"91":1,"92":2,"93":2,"95":2,"97":6,"99":2,"100":1,"101":2,"105":1,"106":6,"107":1,"109":1,"110":6,"114":2,"118":1,"120":4,"123":2,"126":2,"127":1,"128":3,"133":3,"135":3,"136":4,"137":1,"138":3,"140":1,"145":1,"146":6,"147":1,"148":1,"149":6,"153":1,"155":1,"158":4,"159":1,"161":1,"162":1,"163":1,"166":4,"167":1,"168":1,"171":9,"172":1,"173":1,"175":4,"177":1,"179":3,"181":1,"183":1,"184":3,"185":2,"186":1,"188":1,"196":1,"198":1,"209":2,"210":2,"211":1,"214":5,"218":1,"219":1,"220":1,"225":2,"230":3,"232":1,"235":1,"245":2,"247":2,"248":2,"249":6,"250":3,"254":5,"255":2,"256":3}}],["bfgs",{"2":{"198":1,"202":1,"258":2}}],["bs",{"2":{"192":1,"213":1}}],["b=f",{"2":{"140":1}}],["b=layer",{"2":{"140":1}}],["b=",{"2":{"69":1}}],["b=zeros",{"2":{"69":1}}],["bc",{"2":{"234":7,"235":21,"236":120}}],["bce",{"2":{"63":5}}],["bcast",{"2":{"34":2}}],["bh2",{"2":{"254":1}}],["bh",{"2":{"254":1}}],["bhn",{"2":{"52":2}}],["bhz",{"2":{"52":2}}],["bhr",{"2":{"52":2}}],["bigger",{"2":{"171":1}}],["bigfloat",{"2":{"171":3}}],["bit",{"2":{"147":1,"202":1}}],["bibtex",{"2":{"74":2}}],["bilinear",{"2":{"53":3,"56":6,"99":1}}],["bidirectional",{"2":{"52":1}}],["bidirectionalrnn",{"2":{"52":1}}],["binarycrossentropy",{"2":{"186":2}}],["binarycrossentropyloss",{"2":{"63":7,"186":1}}],["binaryfocalloss",{"2":{"63":3}}],["binary",{"2":{"63":3,"183":1}}],["bin",{"2":{"52":1,"63":7}}],["biz",{"2":{"52":1}}],["bir",{"2":{"52":1}}],["bias=l",{"2":{"135":1}}],["bias=ps",{"2":{"69":1}}],["bias=false",{"2":{"52":6,"53":3}}],["bias=true",{"2":{"49":4,"52":2,"53":6}}],["bias=nothing",{"2":{"49":1,"52":3,"53":3}}],["bias=zeros32",{"2":{"49":1,"53":1,"55":4,"135":1,"256":3}}],["bias=zero",{"2":{"39":1}}],["biases",{"2":{"28":1,"249":1}}],["bias",{"0":{"15":1},"2":{"15":12,"16":3,"18":3,"19":15,"38":1,"39":4,"41":4,"49":12,"52":25,"53":22,"55":20,"69":1,"77":15,"100":4,"106":12,"110":3,"116":2,"126":4,"127":2,"128":2,"129":1,"135":6,"137":4,"145":3,"147":4,"148":4,"158":5,"166":1,"179":1,"220":30,"244":2,"249":1,"256":3,"258":3}}],["bright",{"2":{"250":3}}],["break",{"2":{"236":1}}],["breaking",{"0":{"87":1,"90":1,"94":1,"97":1,"98":1,"99":1},"2":{"92":1}}],["branched",{"2":{"48":1}}],["branchlayer",{"2":{"48":5}}],["broken=true",{"2":{"58":2}}],["broken=false",{"2":{"58":2}}],["broken",{"2":{"37":1,"58":5,"59":4,"60":1}}],["broadcastfunction",{"2":{"106":1}}],["broadcastable",{"2":{"55":1}}],["broadcast",{"2":{"34":1,"50":1,"54":1,"106":1,"249":1}}],["broadcasted",{"2":{"15":1,"16":2,"34":2,"63":1}}],["broadcasting",{"2":{"13":1,"20":1,"171":1}}],["bn=batchnorm",{"2":{"39":1}}],["black",{"2":{"241":1,"245":2,"248":2}}],["blas",{"2":{"8":1,"18":1}}],["blue",{"2":{"200":1,"203":2,"241":1,"248":1}}],["blocks",{"2":{"52":1,"62":1,"184":1}}],["block",{"2":{"48":1,"51":3,"69":3,"184":1}}],["blog",{"2":{"46":1}}],["blisblas",{"2":{"18":1}}],["b",{"2":{"16":4,"17":2,"18":3,"23":3,"51":6,"53":4,"69":6,"109":2,"131":1,"137":2,"140":9,"146":8,"158":2,"179":8,"236":8,"254":4}}],["bulk",{"2":{"249":1}}],["builds",{"2":{"109":1,"220":3,"236":4}}],["buildkite",{"2":{"109":1,"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"220":3,"221":1,"229":1,"236":4,"238":1,"246":1,"251":1,"260":1}}],["build",{"2":{"77":1,"163":3,"170":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["building",{"0":{"249":1},"2":{"62":1}}],["built",{"0":{"47":1},"1":{"48":1,"49":1,"50":1,"51":1,"52":1,"53":1,"54":1,"55":1,"56":1},"2":{"81":1,"133":2,"173":1,"196":1}}],["buffer",{"2":{"16":1,"18":1,"34":3}}],["bugs",{"2":{"84":1,"93":1,"137":1}}],["bug",{"2":{"11":1,"69":1,"153":1,"209":1}}],["but",{"2":{"3":2,"7":2,"8":1,"13":1,"14":1,"15":1,"19":1,"22":2,"28":1,"44":1,"46":3,"54":1,"56":2,"64":3,"65":5,"67":2,"69":4,"82":1,"83":2,"90":1,"101":1,"104":3,"105":2,"106":1,"109":3,"110":2,"119":1,"130":1,"135":2,"136":1,"137":1,"145":3,"146":1,"147":1,"152":1,"153":1,"155":2,"160":1,"167":1,"179":1,"184":2,"186":1,"201":1,"217":1,"219":1,"235":1,"242":1,"249":1,"256":1}}],["box",{"2":{"248":1}}],["boxing",{"2":{"68":1,"218":1,"220":1}}],["borrow",{"2":{"247":1}}],["borders",{"2":{"49":2,"51":3}}],["book",{"2":{"177":1}}],["boom",{"2":{"147":1}}],["bool=true",{"2":{"52":1}}],["bool=false",{"2":{"2":2,"44":2,"46":1,"52":4,"58":1,"215":1,"217":1}}],["boolean",{"2":{"8":1,"46":1,"60":1}}],["bool",{"2":{"2":1,"3":5,"8":3,"40":1,"46":1,"63":2,"64":1,"65":1,"220":26}}],["boilerplate",{"2":{"106":1,"107":1,"185":1}}],["bottom",{"2":{"69":1,"106":1}}],["both",{"2":{"2":1,"40":2,"63":2,"77":1,"81":1,"100":1,"110":3,"140":1,"163":1,"166":1,"210":1,"249":1}}],["body",{"2":{"69":2,"220":3,"254":9}}],["boundaries",{"2":{"250":1}}],["boundary",{"2":{"234":1}}],["bound",{"2":{"49":6,"52":18,"53":9}}],["bounds",{"2":{"49":2,"52":6,"53":3}}],["bypasses",{"2":{"69":2}}],["bypass",{"2":{"11":1,"136":1,"164":1}}],["by",{"0":{"127":1},"2":{"2":1,"7":1,"8":2,"11":1,"13":2,"16":1,"18":1,"19":1,"28":5,"46":1,"48":2,"49":5,"51":3,"52":5,"53":5,"54":1,"55":1,"56":1,"62":3,"63":2,"64":1,"66":1,"69":2,"76":1,"77":1,"83":2,"90":1,"97":1,"106":2,"110":1,"120":3,"127":3,"128":2,"131":1,"132":1,"133":1,"144":3,"145":2,"148":1,"149":1,"155":1,"156":1,"157":1,"163":1,"167":1,"171":2,"172":2,"173":1,"175":1,"181":1,"193":1,"201":1,"210":1,"220":1,"244":2,"247":1,"249":1,"250":1}}],["bayes",{"2":{"249":3}}],["bayesian",{"0":{"247":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"248":1,"250":1}}],["battery",{"2":{"83":1}}],["batches",{"2":{"192":1,"213":1}}],["batched",{"0":{"14":1,"23":1,"146":1},"2":{"11":1,"14":6,"23":1,"144":1,"146":8,"150":1,"151":1,"158":2}}],["batchsize=32",{"2":{"236":2}}],["batchsize=256",{"2":{"224":1}}],["batchsize=min",{"2":{"224":2}}],["batchsize=8",{"2":{"201":2}}],["batchsize=128",{"2":{"183":2}}],["batchsize=12",{"2":{"160":2}}],["batchsize=13",{"2":{"5":1}}],["batchsize",{"2":{"192":3,"213":3,"224":4}}],["batching",{"2":{"153":3}}],["batchlastindex",{"2":{"52":2,"64":1}}],["batchnorm",{"2":{"7":1,"19":1,"48":4,"55":8,"109":4,"110":4,"126":1,"129":1,"135":1,"138":1,"139":2,"142":1,"145":2,"146":1}}],["batch",{"2":{"5":1,"19":6,"23":3,"46":1,"49":3,"51":3,"52":13,"54":1,"55":6,"56":3,"87":1,"146":1,"153":6,"183":1,"201":1,"202":9,"236":8}}],["basically",{"2":{"136":1}}],["basic",{"2":{"62":2,"170":1}}],["basis",{"2":{"49":2,"53":1}}],["baseline",{"2":{"63":1}}],["base",{"2":{"50":1,"54":1,"58":3,"64":3,"97":1,"116":1,"137":1,"144":3,"147":1,"148":2,"198":1,"201":2,"215":1,"220":25,"256":2}}],["based",{"2":{"2":1,"3":2,"18":1,"19":1,"48":1,"65":1,"84":1,"99":2,"100":1,"145":1,"149":1,"170":1,"198":1}}],["bad",{"2":{"40":1,"155":1,"250":1}}],["ba",{"2":{"19":1}}],["backtracking",{"2":{"258":1}}],["background",{"2":{"135":1,"155":1,"255":1}}],["backward",{"2":{"40":2,"52":11,"64":1,"110":3,"111":1,"173":1}}],["back",{"2":{"2":1,"3":1,"13":1,"137":1}}],["backendtpu",{"2":{"73":1}}],["backendgpu",{"2":{"73":1}}],["backends=",{"2":{"59":2}}],["backends",{"0":{"31":1},"2":{"1":1,"2":4,"3":5,"5":1,"16":3,"18":4,"20":1,"22":6,"23":2,"59":7,"62":5,"82":2,"95":1,"104":3,"130":3,"153":1,"160":1,"162":1,"172":1,"177":1,"217":1}}],["backend",{"0":{"45":1,"131":1,"132":1,"164":1},"1":{"46":1},"2":{"1":17,"2":3,"3":1,"12":1,"22":6,"23":3,"31":5,"32":20,"33":4,"34":11,"35":1,"36":1,"59":1,"62":4,"73":4,"88":1,"120":14,"121":2,"123":1,"131":5,"144":1,"153":6,"164":4,"175":1,"198":1}}],["bernoulli",{"2":{"249":1}}],["became",{"2":{"155":1}}],["because",{"2":{"137":1,"258":1}}],["become",{"2":{"135":1,"181":1}}],["benchmarking",{"2":{"196":1}}],["benefits",{"2":{"106":1}}],["bengio",{"2":{"28":2,"54":1}}],["behind",{"2":{"90":1}}],["behaving",{"2":{"169":1}}],["behaviour",{"2":{"90":1}}],["behavior",{"2":{"3":2,"7":2,"8":1,"11":2,"28":1,"52":1,"67":3,"145":2,"155":1,"184":1}}],["behave",{"2":{"64":1}}],["behaves",{"2":{"48":1,"91":1,"160":1}}],["better",{"2":{"83":1,"84":1}}],["between",{"0":{"42":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"28":1,"41":1,"44":1,"53":1,"63":1,"83":1,"106":1,"123":1,"140":3,"250":3}}],["beginner",{"0":{"206":1}}],["begin",{"2":{"49":2,"51":3,"200":1,"203":1,"235":1,"237":1,"255":1,"256":1,"259":1}}],["beyond",{"2":{"48":1}}],["been",{"2":{"44":2,"83":1,"87":2,"90":6,"94":3,"97":10,"98":5,"100":1,"104":1,"122":1,"148":1,"170":1,"252":1}}],["below",{"2":{"39":1,"48":1,"67":2,"74":1,"214":1,"248":1,"249":2,"250":1}}],["beware",{"2":{"29":1,"137":1}}],["being",{"2":{"15":1,"17":2,"19":2,"55":4,"65":1,"88":1,"95":1,"103":2,"124":1,"144":1,"145":2,"155":1,"196":1,"202":1}}],["best",{"2":{"13":2,"16":1,"18":1,"44":1,"69":2,"103":2,"104":1}}],["before",{"2":{"8":2,"28":2,"32":1,"35":1,"36":1,"48":1,"78":1,"107":1,"120":1,"135":1,"174":1,"188":1}}],["be",{"2":{"1":2,"2":1,"3":10,"4":5,"6":1,"7":4,"8":2,"10":1,"13":2,"14":1,"15":3,"16":1,"17":4,"18":1,"19":20,"20":1,"23":5,"28":8,"32":7,"34":2,"36":2,"38":2,"40":3,"41":5,"44":5,"46":5,"48":13,"49":16,"50":4,"51":9,"52":17,"53":8,"54":8,"55":4,"56":3,"58":3,"59":3,"60":4,"62":9,"63":5,"65":3,"68":2,"69":16,"70":2,"71":1,"74":1,"77":1,"81":1,"82":3,"83":2,"84":1,"85":1,"93":1,"94":1,"97":1,"98":1,"106":3,"108":1,"111":1,"115":1,"120":2,"121":1,"125":1,"127":1,"130":2,"132":1,"133":1,"135":1,"137":2,"138":1,"139":1,"140":6,"142":1,"144":2,"145":5,"146":2,"148":1,"150":1,"155":1,"159":1,"164":1,"166":3,"167":2,"169":3,"171":1,"173":1,"174":1,"175":1,"176":1,"181":1,"184":1,"196":2,"202":1,"209":1,"210":1,"218":1,"226":1,"230":2,"235":1,"242":1,"248":1,"249":1,"254":4,"257":1}}],["ixy",{"2":{"254":2}}],["ixx",{"2":{"254":3}}],["iyy",{"2":{"254":3}}],["i∈",{"2":{"179":1}}],["i9rhw",{"2":{"145":2}}],["ibkvy",{"2":{"116":3}}],["iid",{"2":{"249":1}}],["iii",{"2":{"102":2,"104":1}}],["ii",{"2":{"102":2,"104":1}}],["ii+pi+p",{"2":{"49":1,"51":3}}],["i=σ",{"2":{"52":1}}],["ih",{"2":{"52":6,"100":1}}],["i+n",{"2":{"49":1,"51":3}}],["ignore",{"2":{"225":1,"256":1}}],["ignores",{"2":{"44":1}}],["ignored",{"2":{"2":1,"28":1,"53":3}}],["iclr",{"2":{"28":1}}],["ieee",{"2":{"28":2,"63":5}}],["ioffe",{"2":{"19":1}}],["imgs",{"2":{"192":5,"213":5,"224":4}}],["im",{"0":{"173":1}}],["immutability",{"2":{"174":1}}],["immutable",{"2":{"13":1,"16":1,"18":1,"65":1,"82":1,"83":1,"173":1}}],["immediately",{"2":{"106":1}}],["imbalanced",{"2":{"63":1}}],["images",{"2":{"55":3,"56":2,"192":1,"213":1}}],["image",{"2":{"49":2,"63":1}}],["imagenet",{"2":{"28":2,"119":1}}],["imply",{"2":{"138":1}}],["implements",{"2":{"57":1}}],["implemented",{"2":{"44":1,"83":1,"140":1,"144":1}}],["implementations",{"2":{"13":1,"14":1,"18":1,"20":1,"62":1,"65":1,"123":1,"167":1}}],["implementation",{"0":{"115":1,"141":1,"218":1},"1":{"142":1},"2":{"11":4,"13":3,"16":4,"18":6,"38":1,"48":1,"52":1,"62":1,"65":5,"68":2,"90":1,"123":1,"149":1,"214":1}}],["implement",{"0":{"225":1},"2":{"7":1,"38":1,"135":2,"140":1,"247":1}}],["implementing",{"0":{"140":1},"2":{"7":2,"83":1,"114":1,"211":1,"247":1}}],["improving",{"2":{"105":1}}],["imports",{"0":{"182":1,"191":1,"199":1,"212":1,"223":1,"231":1,"240":1,"253":1}}],["importing",{"2":{"135":1,"170":1,"247":1}}],["important",{"0":{"141":1},"1":{"142":1},"2":{"120":2,"130":1,"135":1,"140":1,"145":1,"146":1}}],["importantly",{"2":{"107":1,"171":1,"250":1}}],["imported",{"2":{"69":1}}],["import",{"2":{"44":1,"46":1,"79":1,"247":1}}],["i",{"2":{"5":10,"11":1,"28":1,"38":1,"44":1,"46":1,"48":9,"49":6,"51":21,"53":4,"54":8,"56":2,"62":2,"63":3,"67":1,"68":1,"69":1,"78":1,"90":2,"102":4,"104":1,"115":3,"120":1,"138":1,"147":1,"153":6,"174":4,"177":1,"179":4,"201":2,"236":3,"237":6,"248":12,"249":7,"250":10}}],["ith",{"2":{"48":1}}],["its",{"2":{"8":1,"51":3,"53":1,"54":1,"55":1,"71":1,"83":1,"250":1}}],["itself",{"2":{"3":1,"54":1,"109":1}}],["iter",{"2":{"78":4,"107":12,"202":2,"236":7,"257":1}}],["iteratively",{"2":{"48":1}}],["iterations",{"2":{"179":12,"249":2}}],["iteration",{"0":{"5":1},"2":{"5":2,"78":12,"107":4,"114":1,"160":1,"174":8,"201":1,"202":26,"236":101,"250":1,"257":1,"259":1}}],["iterate",{"2":{"5":1,"37":1}}],["iterates",{"2":{"5":1}}],["iterators",{"2":{"5":1,"184":1,"185":1,"235":1,"236":2,"237":1}}],["iterator",{"2":{"5":3,"107":1}}],["item",{"2":{"5":1}}],["it",{"2":{"2":1,"3":2,"4":5,"7":3,"8":6,"11":2,"13":3,"15":2,"16":2,"17":1,"28":3,"37":1,"38":1,"39":1,"41":1,"44":3,"48":2,"49":4,"51":3,"52":8,"54":1,"55":7,"56":3,"57":2,"60":2,"62":1,"63":5,"64":1,"65":3,"67":3,"68":1,"69":5,"74":1,"76":1,"78":1,"81":1,"83":4,"90":1,"98":1,"100":1,"101":1,"105":3,"106":5,"108":2,"109":1,"110":3,"113":1,"120":2,"121":1,"131":1,"133":2,"135":2,"136":5,"137":4,"144":3,"145":6,"146":2,"152":1,"153":2,"155":1,"156":2,"157":1,"160":1,"164":1,"167":1,"170":1,"171":2,"172":1,"173":2,"174":2,"177":1,"179":1,"184":4,"186":2,"198":1,"201":2,"202":2,"209":1,"210":1,"214":1,"225":2,"249":2,"254":1,"256":2}}],["inlined",{"2":{"236":4}}],["inplace",{"2":{"62":2}}],["input",{"0":{"112":1,"118":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1},"2":{"8":2,"13":2,"15":1,"16":1,"17":3,"18":2,"19":9,"22":2,"23":1,"28":4,"40":2,"46":3,"48":30,"49":7,"50":2,"51":6,"52":20,"53":12,"54":7,"55":14,"56":6,"63":4,"68":1,"69":1,"77":1,"106":1,"109":6,"110":14,"114":1,"120":1,"121":1,"123":1,"135":1,"138":2,"146":3,"148":1,"153":8,"155":2,"165":1,"166":1,"169":1,"184":1,"193":1,"214":1,"233":1,"256":1}}],["inputsize",{"2":{"52":1,"90":1}}],["inputs",{"0":{"146":1},"2":{"8":2,"11":1,"16":3,"18":1,"23":1,"39":1,"40":1,"48":9,"49":2,"50":3,"51":9,"52":11,"53":1,"54":6,"55":5,"56":2,"62":1,"63":4,"65":1,"68":1,"82":1,"98":1,"106":1,"123":1,"144":1,"146":2,"147":1,"148":1,"178":1,"214":2,"244":1,"249":1,"256":1}}],["in2",{"2":{"53":6}}],["in12",{"2":{"53":3}}],["in1",{"2":{"53":8}}],["injection",{"2":{"48":5}}],["independent",{"2":{"146":1}}],["index",{"2":{"48":1,"53":1,"54":2,"153":1,"250":1}}],["indexing",{"0":{"156":1},"2":{"20":1,"48":1,"54":1,"156":1}}],["indexed",{"2":{"2":2,"4":2,"54":1,"84":1}}],["individual",{"2":{"54":1,"202":1}}],["individually",{"2":{"48":1,"56":2,"146":1}}],["indices",{"2":{"53":2}}],["inner",{"2":{"38":2,"40":1,"127":1}}],["initilly",{"2":{"50":1}}],["initializing",{"0":{"168":1},"1":{"169":1},"2":{"69":1,"120":1}}],["initialize",{"0":{"215":1,"226":1},"2":{"32":5,"69":3,"77":1,"78":1,"120":3,"121":1,"123":1,"153":1,"179":1,"241":1,"249":1}}],["initialized",{"2":{"28":4,"32":2,"55":8}}],["initializers",{"2":{"28":1}}],["initializer",{"2":{"28":1,"52":10,"53":7,"69":3}}],["initializations",{"2":{"99":2}}],["initialization",{"0":{"32":1},"2":{"25":1,"28":2,"32":1,"46":1,"49":4,"99":1,"140":1,"168":1,"184":1,"225":1,"256":1}}],["initial",{"2":{"9":1,"10":1,"48":1,"52":11,"69":2,"83":1,"90":1,"140":1,"184":1,"255":1,"256":1,"258":2}}],["initialstates",{"2":{"7":3,"8":1,"10":1,"135":2,"140":2,"184":1}}],["initialparameters",{"2":{"7":3,"8":1,"9":1,"135":2,"140":2,"184":1,"225":3}}],["init",{"2":{"28":7,"49":10,"52":26,"53":21,"55":16,"64":6,"69":6,"99":2,"100":1,"121":1,"135":10,"140":4,"169":8,"184":2,"185":2,"256":6}}],["incase",{"2":{"256":1}}],["including",{"2":{"93":1,"106":1}}],["included",{"2":{"28":1,"83":1}}],["include",{"2":{"7":1,"8":1,"28":1}}],["inconvenient",{"2":{"113":1}}],["inconsistent",{"2":{"90":1}}],["incorrectly",{"2":{"155":1}}],["incorrect",{"0":{"109":1},"2":{"23":2,"69":1,"108":1,"111":1}}],["increase",{"2":{"49":1,"100":1}}],["ingredient",{"2":{"19":1,"55":1}}],["investigate",{"2":{"219":1}}],["investigated",{"2":{"124":1}}],["inversability",{"2":{"49":1}}],["inverse",{"2":{"17":1}}],["involving",{"2":{"68":1}}],["invokes",{"2":{"39":1}}],["invoked",{"2":{"2":1,"184":1}}],["invariant",{"2":{"63":1}}],["inv",{"2":{"49":2,"52":6,"53":3,"254":1}}],["invp",{"2":{"17":4}}],["informed",{"2":{"234":3}}],["informs",{"2":{"136":1}}],["information",{"2":{"8":1,"11":1,"17":1,"24":1,"40":1,"59":1,"67":1,"82":1,"83":1,"123":1,"157":1,"198":1,"210":1}}],["info",{"2":{"109":8,"110":20,"130":1,"132":1,"180":2,"189":2,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2,"247":2,"251":2,"260":2}}],["inf",{"2":{"63":1,"145":2,"146":2,"147":2,"148":2,"152":4}}],["inference",{"0":{"142":1},"2":{"40":1,"50":3,"55":6,"74":1,"142":1,"145":2,"153":2,"249":2}}],["inferred",{"2":{"19":1}}],["infinity",{"2":{"17":1}}],["int=50000",{"2":{"236":1}}],["int=32",{"2":{"233":1,"236":1}}],["int=0",{"2":{"34":5,"236":1}}],["intro",{"2":{"170":1}}],["introduction",{"2":{"91":1}}],["introductory",{"2":{"84":1}}],["introducing",{"2":{"44":1}}],["introduces",{"2":{"170":1}}],["introduced",{"2":{"28":1}}],["introduce",{"2":{"11":1,"68":1,"110":2}}],["int64",{"2":{"63":3,"77":16,"106":6,"116":4,"171":3,"173":2,"201":2,"220":99,"245":4}}],["int",{"2":{"19":1,"49":2,"52":2,"54":1,"55":1,"56":1,"135":4,"179":1,"214":2,"224":3,"248":1}}],["into",{"0":{"120":1},"2":{"8":1,"16":1,"28":2,"34":1,"40":1,"46":1,"48":2,"49":2,"51":3,"52":4,"54":1,"56":1,"63":1,"64":1,"66":4,"69":2,"77":1,"90":1,"106":1,"133":1,"135":1,"153":2,"171":2,"184":1,"192":1,"213":1}}],["intentionally",{"2":{"139":1}}],["interpolatingadjoint",{"2":{"220":13}}],["intermediate",{"0":{"207":1},"2":{"236":4}}],["interactiveutils",{"2":{"180":2,"189":2,"197":2,"204":2,"212":1,"221":2,"229":2,"238":2,"246":2,"251":2,"260":2}}],["interactive",{"2":{"161":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["interested",{"2":{"146":1,"250":1}}],["interoperability",{"0":{"42":1},"1":{"43":1,"44":1,"45":1,"46":1}}],["interval",{"2":{"28":2}}],["inter",{"2":{"23":1}}],["international",{"2":{"19":1,"28":6,"63":3}}],["internals",{"2":{"225":1,"249":1}}],["internally",{"2":{"44":1,"68":1,"90":1,"144":1,"153":1,"220":1}}],["internal",{"2":{"19":2,"20":3,"38":1,"46":1,"48":1,"54":2,"62":1,"68":2,"81":1,"83":1,"97":1,"120":1}}],["interfacem",{"2":{"46":1}}],["interface",{"0":{"133":1,"134":1,"137":1,"138":1},"1":{"134":1,"135":2,"136":2,"137":1,"138":1},"2":{"7":1,"36":2,"46":2,"133":2,"135":1,"198":1,"214":1,"249":1}}],["integral",{"2":{"63":1}}],["integrating",{"0":{"120":1}}],["integration",{"0":{"35":1,"36":1},"2":{"121":1}}],["integrated",{"2":{"8":1,"157":1}}],["integers",{"2":{"32":1,"49":11,"51":15,"53":5}}],["integer",{"2":{"2":3,"4":3,"28":11,"49":12,"51":9,"53":1,"54":1,"55":5,"67":3}}],["intelligence",{"2":{"28":2}}],["intel",{"2":{"3":1}}],["inside",{"0":{"153":1},"2":{"39":1,"67":1,"69":1,"127":1,"130":1,"140":3,"144":3,"153":3}}],["instructions",{"2":{"76":1}}],["institute",{"2":{"74":1}}],["installation",{"0":{"76":1},"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["install",{"0":{"71":1},"2":{"71":1,"72":1,"73":1,"76":1,"77":1,"79":1}}],["installed",{"2":{"32":4,"36":1,"120":1,"172":1}}],["instability",{"2":{"68":1,"157":1,"166":1}}],["instabilities",{"0":{"157":1},"2":{"44":1,"106":1,"157":2}}],["instance",{"2":{"19":3,"52":1,"55":3,"249":1}}],["instancenorm",{"2":{"19":2,"55":7,"99":1,"100":1}}],["instead",{"2":{"3":2,"4":2,"8":1,"31":2,"38":2,"44":1,"49":3,"58":1,"62":1,"67":1,"69":1,"93":1,"94":1,"95":1,"97":1,"98":4,"99":2,"113":1,"114":1,"120":1,"128":1,"133":1,"136":1,"137":1,"140":2,"147":1,"153":2,"155":1,"158":1,"160":1,"167":1,"171":1,"184":1,"211":1,"236":4,"247":1}}],["inspiration",{"2":{"5":1}}],["in",{"0":{"47":1,"99":1,"113":1},"1":{"48":1,"49":1,"50":1,"51":1,"52":1,"53":1,"54":1,"55":1,"56":1},"2":{"2":2,"3":1,"4":2,"5":2,"7":4,"8":5,"10":3,"13":3,"15":2,"17":3,"19":2,"22":2,"28":15,"32":1,"34":2,"35":1,"37":1,"38":2,"39":3,"40":3,"41":6,"44":5,"46":1,"48":20,"49":22,"50":2,"51":3,"52":23,"53":13,"54":6,"55":10,"57":1,"58":1,"62":7,"63":3,"64":4,"65":7,"67":1,"68":4,"69":18,"71":3,"74":2,"76":1,"78":7,"79":1,"81":2,"82":1,"83":3,"84":2,"85":1,"87":1,"90":1,"99":1,"100":1,"101":2,"103":1,"105":1,"107":4,"108":1,"109":1,"110":11,"111":3,"114":1,"115":1,"120":4,"123":2,"124":1,"125":2,"126":1,"127":1,"130":2,"131":1,"135":10,"138":1,"139":2,"140":1,"143":2,"144":3,"145":3,"146":8,"147":1,"148":1,"149":2,"152":1,"153":5,"155":2,"156":2,"157":1,"160":5,"161":2,"167":1,"168":1,"169":1,"170":2,"171":2,"173":1,"174":3,"175":1,"179":2,"181":1,"183":3,"184":6,"185":3,"187":3,"190":1,"194":1,"195":2,"198":2,"201":1,"202":4,"209":1,"216":1,"217":2,"219":1,"225":1,"227":1,"228":4,"230":1,"235":1,"236":1,"237":2,"239":1,"245":1,"247":1,"248":6,"249":2,"250":13,"255":1,"256":3}}],["idx",{"2":{"227":2,"228":12}}],["ideal",{"2":{"196":1}}],["ideally",{"2":{"20":1,"210":1}}],["identity",{"2":{"16":1,"19":5,"28":13,"48":1,"77":3,"106":1,"116":2,"220":9,"245":1}}],["id",{"2":{"2":7,"4":3}}],["if",{"2":{"1":2,"2":12,"3":9,"4":3,"6":1,"7":1,"8":5,"11":2,"13":4,"15":2,"16":2,"17":7,"18":2,"19":6,"23":2,"32":2,"38":3,"39":1,"40":5,"41":2,"44":2,"46":1,"48":17,"49":8,"50":10,"52":28,"53":12,"54":5,"55":14,"56":3,"58":1,"59":2,"60":3,"62":2,"63":8,"64":4,"65":3,"67":1,"68":3,"69":6,"70":1,"71":1,"74":2,"76":1,"77":1,"78":2,"83":1,"90":2,"97":1,"103":2,"104":1,"106":3,"107":1,"109":1,"110":1,"111":1,"123":1,"124":1,"126":1,"127":1,"130":1,"131":4,"132":1,"133":1,"135":3,"136":3,"137":3,"142":1,"143":1,"144":3,"145":2,"148":1,"153":1,"159":1,"163":3,"167":1,"168":2,"169":2,"172":2,"174":1,"175":1,"179":1,"180":3,"189":3,"197":3,"202":2,"204":3,"209":1,"210":2,"221":3,"229":3,"236":1,"238":3,"245":1,"246":3,"251":3,"254":3,"260":3}}],["isdefined",{"2":{"180":3,"189":3,"197":3,"204":3,"221":3,"229":3,"238":3,"246":3,"251":3,"260":3}}],["isnan",{"2":{"236":1}}],["isn",{"2":{"90":1,"143":1,"254":1}}],["issuing",{"2":{"67":1}}],["issue",{"2":{"67":1,"103":1,"106":1,"111":1,"124":1,"144":1,"145":2,"148":1,"159":1,"209":1,"210":2}}],["issues",{"2":{"52":1,"62":1,"69":1,"83":1,"84":1,"104":2,"105":1,"144":1,"153":1}}],["isbitstype",{"2":{"65":1}}],["istraining",{"2":{"64":4}}],["iszero",{"2":{"39":1}}],["isa",{"2":{"28":1,"39":1,"69":1}}],["isleaf",{"2":{"3":5,"10":1}}],["is",{"2":{"0":1,"1":4,"2":19,"3":18,"4":10,"5":3,"6":1,"7":6,"8":12,"10":2,"11":4,"13":2,"14":1,"15":3,"16":4,"17":6,"18":2,"19":7,"20":4,"22":4,"25":1,"28":9,"32":2,"36":2,"38":5,"39":4,"40":6,"44":3,"46":7,"48":23,"49":3,"50":7,"51":3,"52":38,"53":9,"54":10,"55":18,"56":3,"57":2,"60":2,"62":4,"63":36,"64":5,"65":9,"67":5,"68":3,"69":9,"70":2,"71":1,"76":1,"83":6,"85":1,"88":1,"90":1,"91":1,"93":1,"97":3,"99":1,"100":1,"101":1,"103":2,"105":2,"106":4,"108":2,"109":4,"110":5,"111":1,"113":1,"119":1,"120":4,"121":1,"122":1,"123":2,"124":2,"127":2,"130":1,"131":9,"132":1,"133":4,"135":3,"136":6,"137":2,"140":6,"142":3,"144":3,"145":12,"146":2,"149":1,"150":1,"152":1,"153":3,"155":2,"156":2,"157":1,"158":2,"160":3,"161":1,"163":2,"164":1,"169":3,"170":1,"171":4,"172":3,"173":4,"175":3,"179":1,"183":1,"184":1,"185":1,"186":2,"190":1,"196":2,"198":2,"202":4,"214":1,"217":2,"220":1,"225":1,"230":1,"236":4,"244":1,"245":1,"247":3,"248":1,"249":8,"250":2,"252":1,"254":1,"256":2}}],["fθ",{"2":{"250":1}}],["fw",{"2":{"179":1}}],["fwiw",{"2":{"136":1}}],["fd",{"2":{"145":8,"146":8,"147":8,"148":8}}],["f2",{"2":{"135":2}}],["ft",{"2":{"68":3}}],["f64",{"2":{"66":1,"256":1}}],["f32",{"2":{"66":1}}],["f16",{"2":{"66":1}}],["f1",{"2":{"63":1,"135":2}}],["fn=neuralode",{"2":{"215":1}}],["fn",{"2":{"62":2,"63":2,"69":8,"215":1}}],["f=σ",{"2":{"52":1}}],["f=identity",{"2":{"19":1}}],["fetch",{"2":{"137":1,"241":1}}],["feel",{"2":{"135":1}}],["feedforward",{"2":{"28":2,"249":1}}],["few",{"2":{"81":1,"171":1,"175":1}}],["fed",{"2":{"52":4}}],["features",{"0":{"37":1,"88":1,"91":1,"95":1,"100":1},"1":{"38":1,"39":1,"40":1,"41":1},"2":{"37":3,"57":1,"82":1,"85":1,"97":1,"124":2,"144":1,"192":1,"213":1}}],["feature",{"2":{"2":1,"28":2,"49":1,"51":6,"55":3,"68":1,"105":1,"106":1,"111":1,"144":2,"153":2,"171":1}}],["func",{"2":{"202":3}}],["functor",{"2":{"65":2,"66":3,"140":2}}],["functors",{"2":{"3":1,"5":1,"7":2,"10":3,"39":2,"65":2,"66":3,"90":1,"95":1,"98":1,"106":1,"126":1,"140":1,"247":1}}],["function3",{"2":{"148":4}}],["function2",{"2":{"147":4}}],["function1",{"2":{"145":5}}],["functions",{"0":{"20":1,"28":1,"29":1,"33":1,"63":1,"194":1,"216":1,"227":1,"234":1,"254":1},"2":{"7":1,"13":1,"26":1,"29":1,"44":1,"46":1,"48":1,"52":4,"62":1,"63":2,"64":1,"66":1,"81":1,"82":2,"83":1,"90":1,"97":2,"101":1,"106":1,"107":1,"123":1,"130":2,"131":1,"135":3,"140":1,"144":1,"148":1,"166":1,"167":1,"171":1,"173":1,"177":2,"225":1,"244":1,"254":1}}],["function",{"0":{"113":1,"145":1,"147":1,"148":1,"244":1},"1":{"146":1},"2":{"3":1,"4":4,"8":4,"10":2,"11":3,"13":4,"14":1,"15":2,"16":1,"17":2,"18":1,"19":7,"22":4,"23":3,"28":2,"31":2,"32":1,"38":1,"39":4,"46":1,"48":2,"49":4,"52":3,"53":4,"54":5,"56":1,"58":1,"59":1,"62":15,"63":2,"65":1,"67":3,"68":1,"69":11,"78":3,"83":1,"90":1,"97":2,"99":3,"106":9,"107":1,"110":2,"115":1,"117":1,"120":1,"126":1,"127":3,"128":2,"131":3,"135":3,"136":1,"137":1,"140":2,"143":1,"144":5,"145":1,"146":6,"148":2,"150":1,"151":1,"152":1,"153":2,"169":1,"172":2,"174":1,"176":1,"178":1,"179":1,"183":1,"185":1,"186":2,"192":1,"194":2,"195":1,"199":1,"202":4,"213":1,"214":4,"215":1,"216":2,"217":2,"218":2,"224":2,"225":1,"226":1,"227":2,"230":1,"231":1,"233":2,"234":14,"236":1,"241":2,"244":3,"245":2,"248":1,"249":5,"250":3,"254":16,"256":2,"257":7}}],["functionalities",{"2":{"30":1,"40":1}}],["functionality",{"0":{"40":1,"97":1,"98":1,"122":1},"2":{"6":1,"52":1,"62":1,"79":1,"90":2,"97":1,"133":2,"172":1}}],["functional",{"2":{"2":5,"3":5,"32":1,"119":1,"132":2,"166":1,"172":1,"180":2,"189":2,"197":2,"204":2,"210":1,"221":2,"229":2,"238":2,"246":2,"251":2,"260":2}}],["full",{"0":{"152":1},"2":{"146":1,"152":16}}],["fully",{"0":{"18":1},"2":{"28":1,"53":2,"63":1,"65":1,"104":1,"119":1}}],["future",{"2":{"144":1}}],["further",{"2":{"62":1}}],["fusion",{"2":{"16":1,"18":1}}],["fuse",{"2":{"16":1}}],["fuses",{"2":{"16":1}}],["fused",{"2":{"16":2,"18":2,"63":1,"158":3,"166":1}}],["fp32",{"2":{"13":1}}],["f",{"2":{"13":4,"16":2,"18":2,"22":5,"23":7,"39":3,"48":4,"54":6,"58":2,"59":6,"62":2,"65":4,"69":1,"140":1,"169":1,"176":3,"177":1,"178":1,"179":1,"202":1}}],["flexibility",{"2":{"137":1}}],["flexible",{"2":{"7":1}}],["flow",{"2":{"106":2}}],["float16",{"2":{"29":8,"66":1,"67":6}}],["floating",{"0":{"66":1},"2":{"13":1,"66":3}}],["float32",{"2":{"5":3,"28":6,"29":8,"44":1,"46":1,"48":2,"54":2,"63":6,"66":2,"67":5,"77":57,"78":1,"106":28,"107":1,"109":9,"110":27,"116":6,"117":3,"118":3,"126":10,"129":6,"131":3,"132":1,"135":2,"136":2,"137":3,"139":2,"145":10,"146":2,"147":11,"148":11,"152":2,"155":5,"168":5,"171":8,"172":1,"176":5,"177":4,"178":1,"179":8,"183":1,"192":1,"195":1,"201":4,"213":1,"220":77,"235":5,"236":1,"241":3,"244":4,"248":8,"256":7}}],["float64",{"2":{"5":4,"29":8,"63":1,"66":1,"67":4,"69":4,"153":7,"155":3,"171":5,"174":1,"201":4,"249":13,"255":1,"256":1,"257":1,"258":2}}],["flipkernel=true",{"2":{"100":1}}],["flat",{"2":{"217":1}}],["flattening",{"2":{"97":1}}],["flatten",{"2":{"54":1}}],["flattens",{"2":{"54":1}}],["flattenlayer",{"2":{"46":1,"54":3,"193":3,"215":1,"220":9,"226":1}}],["flattened",{"2":{"44":1,"54":3,"247":1}}],["flaky",{"2":{"122":1}}],["flag",{"2":{"46":1}}],["fluxlinear",{"2":{"140":5}}],["fluxlayer",{"2":{"44":2}}],["fluxmpifluxmodel",{"2":{"122":1}}],["fluxmpi",{"0":{"121":1},"1":{"122":1,"123":1},"2":{"119":1,"121":3,"122":1,"123":2}}],["flux",{"0":{"44":1,"83":1,"139":1,"143":1},"1":{"140":1,"141":1,"142":1,"143":1},"2":{"44":14,"52":3,"81":1,"83":4,"122":1,"135":2,"139":3,"140":5,"142":4,"143":1,"170":1,"247":1,"252":1}}],["fmaps",{"2":{"8":1}}],["fmap",{"2":{"7":3,"8":2,"10":1,"39":1,"65":7,"249":1}}],["framerate=10",{"2":{"237":1}}],["framework",{"2":{"81":1,"83":1,"133":1,"170":1,"173":1,"190":1}}],["frameworks",{"0":{"43":1},"1":{"44":1},"2":{"49":1,"52":1,"81":3,"133":1,"171":1,"256":1}}],["frequently",{"2":{"209":1}}],["frees",{"2":{"160":1}}],["freeze",{"2":{"38":5,"125":1,"126":6,"127":6,"128":5,"129":1}}],["freezing",{"0":{"38":1,"125":1,"126":1,"127":1,"128":1,"129":1},"1":{"126":1,"127":1,"128":1,"129":1},"2":{"125":1,"127":1,"128":2}}],["free",{"2":{"5":1,"28":1,"62":1,"254":1}}],["freeable",{"2":{"5":1}}],["friendly",{"2":{"81":2}}],["frozen",{"2":{"38":8,"126":9,"129":4}}],["frozenlayer",{"2":{"38":8}}],["fromfluxadaptor",{"2":{"44":5,"143":1}}],["from",{"0":{"43":1,"121":1,"139":1},"1":{"44":1,"122":1,"123":1,"140":1,"141":1,"142":1,"143":1},"2":{"4":1,"5":3,"7":2,"11":1,"13":1,"17":1,"20":1,"28":6,"29":12,"39":1,"44":1,"46":1,"48":1,"49":2,"50":1,"52":10,"53":1,"55":2,"62":5,"63":2,"64":1,"69":1,"77":1,"78":1,"83":1,"97":2,"98":1,"99":2,"130":1,"135":1,"136":1,"137":1,"145":2,"146":2,"147":1,"148":1,"155":1,"158":1,"160":3,"171":1,"179":2,"184":1,"190":1,"198":1,"202":1,"211":1,"220":3,"230":1,"239":1,"241":1,"243":1,"247":1,"249":3,"250":4,"252":2,"254":3,"256":1}}],["familiar",{"2":{"181":1,"225":1}}],["far",{"2":{"135":1}}],["farley",{"2":{"54":1}}],["fake",{"2":{"110":1}}],["facusapienza",{"2":{"145":1}}],["fact",{"2":{"123":1}}],["factor",{"2":{"17":2,"19":8,"28":2,"56":2,"63":2}}],["facilitates",{"2":{"68":1}}],["fausto",{"2":{"63":1}}],["favor",{"2":{"49":1}}],["failed",{"2":{"109":1}}],["failing",{"2":{"59":1}}],["failures",{"2":{"44":1}}],["fail",{"2":{"44":1,"59":4,"104":1}}],["fails",{"2":{"40":1,"58":1,"103":2}}],["fashionmnist",{"0":{"222":1},"1":{"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"224":1,"228":2}}],["fashion",{"2":{"32":1}}],["fastest",{"2":{"103":2,"150":1}}],["faster",{"0":{"158":1},"2":{"3":1,"13":2,"14":3,"103":1,"144":1,"158":2,"159":1}}],["fast",{"2":{"13":4,"15":2,"19":1,"20":1,"37":1,"46":1,"55":1,"97":1,"149":1,"158":2,"160":1,"177":1,"256":3}}],["fancy",{"2":{"106":1}}],["fan",{"2":{"28":6,"49":2}}],["fallback",{"2":{"7":1,"11":3,"16":2,"18":2,"20":1,"90":1,"143":1}}],["fall",{"2":{"3":1}}],["falls",{"2":{"2":1,"13":1}}],["false",{"2":{"2":2,"10":1,"17":1,"20":1,"40":1,"44":1,"46":2,"48":2,"49":2,"52":10,"53":3,"60":1,"63":5,"64":1,"65":1,"68":1,"99":1,"144":1,"156":1,"162":1,"163":1,"212":1,"214":3,"218":2,"220":13,"223":1,"231":1,"257":1}}],["fitting",{"0":{"239":1},"1":{"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1}}],["fit",{"2":{"202":1,"236":4,"239":1,"250":1}}],["figure",{"2":{"200":1,"203":1,"237":1,"241":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["figured",{"2":{"110":1}}],["fig",{"2":{"200":3,"203":3,"237":6,"241":3,"245":3,"248":3,"250":6,"255":3,"256":3,"259":6}}],["finetune",{"2":{"202":1}}],["fine",{"2":{"136":1}}],["finite",{"2":{"145":4,"146":2,"149":1}}],["finitediff",{"2":{"59":1,"144":1,"145":2,"146":2}}],["finish",{"2":{"108":1}}],["findmax",{"2":{"250":1}}],["find",{"2":{"69":1,"111":1,"153":1,"179":1,"209":1,"250":1}}],["final",{"2":{"69":2,"228":3}}],["finally",{"0":{"196":1},"2":{"32":1,"69":1,"120":1,"135":1,"136":1,"171":1,"179":1,"184":1,"220":1,"245":1,"259":1}}],["fix",{"2":{"104":1,"109":1,"110":2,"145":2,"149":1,"155":2,"209":1}}],["fixing",{"2":{"93":1,"148":1,"168":1}}],["fix1",{"2":{"50":1,"54":1,"144":3,"147":1,"148":2,"215":1,"220":9,"256":2}}],["fixed=",{"2":{"69":1}}],["fixed",{"2":{"17":1,"68":1,"69":4,"98":2,"109":4,"173":1}}],["fill",{"2":{"63":1,"171":1}}],["filter",{"2":{"28":1}}],["file",{"2":{"1":1,"131":3}}],["fields",{"2":{"48":10,"54":2,"62":1,"65":1,"137":1}}],["fieldnames",{"2":{"7":1,"184":1}}],["field",{"2":{"7":3,"8":1,"44":2,"48":1,"64":1,"184":1}}],["first",{"2":{"2":1,"8":1,"39":1,"41":1,"44":1,"46":1,"48":2,"51":3,"54":2,"69":6,"77":1,"105":1,"106":2,"110":4,"121":1,"131":1,"135":2,"137":2,"139":1,"140":1,"144":1,"145":1,"153":2,"155":2,"161":1,"173":1,"174":2,"176":1,"179":1,"184":3,"194":1,"196":1,"214":1,"216":1,"227":1,"232":1,"245":1,"247":1,"248":2,"250":3,"255":1,"256":2,"257":1,"259":1}}],["footnotes",{"0":{"105":1}}],["foldl",{"2":{"64":3}}],["follow",{"2":{"107":1}}],["follows",{"2":{"3":1,"5":1,"48":1}}],["following",{"2":{"2":1,"5":1,"20":1,"23":1,"39":1,"65":1,"66":1,"67":2,"71":2,"72":2,"73":2,"107":1,"126":1,"130":1,"133":1,"139":1,"144":1,"149":1,"154":1,"155":1,"159":1,"161":2,"163":1,"184":1,"234":1,"250":1}}],["focuses",{"2":{"63":1}}],["focalloss",{"2":{"63":2}}],["focal",{"2":{"63":4}}],["four",{"2":{"171":1}}],["fourth",{"2":{"63":1}}],["found",{"2":{"2":1,"74":1,"210":1}}],["forum",{"2":{"144":1}}],["forget",{"2":{"108":1}}],["formulas",{"2":{"254":1}}],["format",{"2":{"171":1}}],["formats",{"2":{"48":4,"54":1}}],["forms",{"2":{"63":1}}],["form",{"2":{"53":1,"69":1,"153":1,"167":1,"173":2}}],["forwarded",{"2":{"58":1}}],["forward",{"2":{"40":1,"52":1,"53":2,"64":1,"69":5,"78":1,"102":2,"110":1,"111":1,"145":1,"147":1,"148":1,"175":1,"176":2,"177":2,"249":1,"250":5}}],["forwarddiff",{"2":{"20":1,"59":1,"65":1,"102":1,"144":3,"145":2,"147":4,"148":2,"152":1,"175":2,"176":4,"177":1}}],["forbidden",{"2":{"28":1}}],["forces",{"2":{"83":1}}],["force",{"2":{"2":6,"44":4,"58":1}}],["for",{"0":{"45":1,"118":1,"146":1,"159":1,"161":1,"170":1,"257":1},"1":{"46":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"0":1,"1":1,"2":3,"3":14,"4":9,"5":3,"6":1,"7":11,"8":7,"11":1,"12":1,"13":1,"14":1,"16":5,"17":8,"18":5,"19":15,"20":3,"22":2,"23":3,"24":1,"25":1,"28":11,"31":2,"32":8,"33":2,"38":5,"39":3,"40":8,"41":2,"44":3,"46":5,"48":11,"49":10,"51":15,"52":24,"53":12,"54":6,"55":18,"56":8,"57":2,"58":1,"59":5,"62":5,"63":9,"64":5,"65":15,"66":1,"67":4,"68":2,"69":16,"70":3,"76":1,"77":2,"78":7,"79":1,"80":5,"81":1,"82":1,"83":9,"84":1,"87":2,"88":2,"90":2,"91":1,"92":1,"93":1,"94":1,"95":1,"97":1,"99":2,"100":1,"101":2,"103":11,"104":2,"105":1,"106":5,"107":4,"108":1,"109":1,"110":1,"113":1,"115":2,"119":1,"120":1,"123":1,"124":2,"125":1,"127":2,"130":1,"133":2,"135":6,"136":3,"137":3,"138":1,"139":1,"142":1,"144":6,"146":1,"147":2,"148":1,"149":1,"150":1,"152":2,"153":4,"154":2,"155":3,"157":1,"159":1,"160":3,"162":2,"164":1,"165":1,"166":5,"167":1,"168":2,"169":1,"170":1,"171":3,"172":1,"173":2,"174":2,"175":5,"176":2,"177":2,"179":3,"183":3,"184":4,"185":2,"186":1,"187":3,"190":1,"194":1,"195":2,"196":1,"198":2,"202":3,"210":1,"213":1,"214":2,"216":1,"217":5,"220":3,"227":1,"228":3,"230":1,"235":2,"236":1,"237":2,"242":1,"245":2,"248":5,"249":2,"250":5,"255":1}}],["tmp",{"2":{"254":4}}],["tmatch",{"2":{"8":3}}],["t∈",{"2":{"235":1}}],["ttest",{"2":{"217":1,"228":2}}],["ttraining",{"2":{"217":1,"228":2}}],["ttime",{"2":{"195":2,"217":3,"228":3}}],["td",{"0":{"118":1}}],["tdchain",{"0":{"116":1},"2":{"115":4,"117":1,"118":1}}],["tl",{"2":{"108":1}}],["tloss",{"2":{"107":1}}],["tpu",{"2":{"73":1,"106":1}}],["tsteps",{"2":{"255":5,"256":4,"259":6}}],["tstate",{"2":{"179":6,"217":7,"245":9}}],["tsit5",{"2":{"200":1,"202":1,"203":1,"220":13}}],["tspan=",{"2":{"214":2,"218":1}}],["tspan",{"2":{"200":5,"203":2,"214":5,"218":3,"220":7,"255":4,"256":1,"259":1}}],["tsung",{"2":{"63":2}}],["ts",{"2":{"62":16,"237":6,"248":1,"249":4}}],["t=rand",{"2":{"59":1}}],["t=float32",{"2":{"28":8}}],["tail",{"2":{"249":1}}],["tab>`",{"2":{"176":1}}],["table",{"2":{"53":1,"171":1}}],["tall",{"2":{"175":1}}],["tasklocalrng",{"2":{"77":1,"129":1,"170":1,"179":1}}],["tasks",{"2":{"63":3}}],["targets",{"2":{"63":1,"192":1,"213":1}}],["target",{"2":{"58":10,"63":2,"194":3,"216":3,"227":3,"234":6,"235":12,"236":10}}],["taking",{"2":{"54":1,"63":1,"64":1}}],["takeaway",{"2":{"137":1}}],["takes",{"2":{"8":1,"48":4,"63":2,"69":1,"78":1,"106":1,"202":1,"244":1,"250":1,"256":2}}],["taken",{"2":{"5":1,"28":1,"49":2,"53":1,"70":1,"99":1,"106":1,"148":1}}],["take",{"2":{"1":1,"29":1,"39":1,"49":1,"62":1,"63":2,"70":1,"77":1,"104":1,"114":1,"145":1,"169":1,"233":1,"235":1,"249":1}}],["tanh",{"2":{"41":2,"52":1,"77":11,"145":1,"146":1,"147":3,"148":3,"152":3,"202":2,"215":4,"220":36,"233":3,"249":3}}],["two",{"2":{"28":3,"48":4,"53":1,"55":1,"56":3,"65":2,"131":1,"135":1,"136":1,"137":1,"149":1,"169":1,"249":1,"254":4,"256":1}}],["turing",{"2":{"247":6}}],["tutorials",{"0":{"205":1,"206":1,"207":1,"208":1,"210":1},"1":{"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"69":1,"84":1,"210":1,"230":1,"236":4}}],["tutorial",{"2":{"68":1,"78":1,"170":3,"175":1,"181":2,"190":2,"198":6,"210":1,"230":1,"239":1,"247":2}}],["tu",{"2":{"22":1}}],["tuple=true",{"2":{"217":1}}],["tuples",{"2":{"7":2}}],["tuple",{"2":{"3":2,"7":2,"19":2,"20":1,"28":3,"38":4,"39":1,"46":1,"48":7,"49":11,"51":15,"52":22,"53":5,"55":1,"56":3,"62":1,"63":1,"65":1,"77":12,"82":1,"115":1,"147":1,"148":1,"153":1,"201":2,"215":2,"220":36,"249":1}}],["temporal",{"2":{"235":1}}],["te",{"2":{"195":4,"196":1,"217":2}}],["technology",{"2":{"74":1}}],["terrible",{"2":{"62":1}}],["terminate",{"2":{"202":1}}],["terminology",{"2":{"202":1}}],["terms",{"2":{"105":1}}],["term",{"2":{"52":1,"145":1,"152":1,"249":2}}],["tested",{"2":{"82":1,"83":1,"104":2,"175":1,"209":1}}],["tests",{"2":{"58":2,"59":2,"104":1,"124":2,"149":1}}],["testing",{"0":{"58":1},"2":{"57":3,"82":1}}],["test",{"0":{"60":1},"2":{"50":3,"55":2,"58":7,"59":10,"60":8,"83":1,"105":1,"106":1,"145":2,"153":1,"192":5,"195":3,"196":20,"213":5,"217":3,"224":5,"228":13,"255":1}}],["testmode`",{"2":{"145":2}}],["testmode",{"2":{"10":1,"50":3,"55":3,"106":3,"142":1,"187":1,"194":1,"216":1,"227":1,"236":1}}],["tends",{"2":{"17":1,"145":1,"146":1}}],["tensors",{"2":{"56":1}}],["tensorflow",{"2":{"52":1}}],["tensor",{"2":{"16":3,"28":1,"55":1,"171":1,"254":1}}],["tile",{"2":{"171":2}}],["tiles",{"2":{"171":1}}],["tightly",{"2":{"167":1}}],["tier",{"2":{"102":8,"104":3}}],["tied",{"0":{"41":1},"2":{"83":1}}],["title",{"2":{"74":2,"237":1,"250":1}}],["tips",{"2":{"154":2}}],["tip",{"2":{"8":1,"18":1,"44":1,"46":1,"52":1,"58":1,"69":1,"119":1,"135":1,"144":2,"210":1,"254":1}}],["timewrapper",{"2":{"201":7,"202":1}}],["timelastindex",{"2":{"52":2,"64":1}}],["timestep",{"2":{"255":1}}],["timespace",{"2":{"255":1}}],["times",{"2":{"48":3,"57":1,"171":1,"174":1,"198":1}}],["time",{"0":{"115":1},"2":{"3":1,"44":1,"52":2,"103":2,"106":1,"114":3,"117":5,"195":3,"196":20,"201":1,"217":3,"219":1,"228":2,"237":1,"255":2,"256":3,"257":1,"259":2}}],["typing",{"2":{"76":1}}],["typical",{"2":{"69":1}}],["typically",{"2":{"7":1,"52":1,"83":1,"147":1,"160":1,"186":1,"235":1,"256":1}}],["typed",{"2":{"176":1}}],["typeof",{"2":{"68":1,"77":8,"106":3,"110":1,"116":2,"135":4,"220":80,"245":2,"256":2}}],["types",{"0":{"7":1,"26":1,"112":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1},"2":{"3":1,"8":2,"20":2,"38":1,"65":6,"92":2,"95":1,"137":1,"138":1,"166":1,"168":2}}],["type",{"0":{"67":1,"155":1,"157":1,"220":1},"2":{"3":13,"4":8,"7":6,"8":5,"16":1,"20":1,"26":2,"28":7,"29":1,"32":5,"44":1,"46":1,"48":2,"55":1,"63":1,"65":6,"67":12,"68":3,"69":4,"106":1,"109":5,"110":13,"113":1,"120":3,"126":1,"127":1,"136":1,"137":3,"138":1,"155":2,"157":2,"166":2,"169":1,"184":2,"187":2,"219":1,"220":1,"224":1}}],["tr",{"2":{"149":2,"152":11,"195":4,"217":2}}],["trying",{"2":{"135":1,"202":1}}],["try",{"2":{"103":1,"106":1,"109":2,"110":1,"116":1,"137":2,"152":1,"171":1,"179":1,"188":1,"210":1}}],["treat",{"2":{"59":1,"136":1}}],["treated",{"2":{"15":1,"69":1}}],["truth",{"2":{"147":1,"179":1}}],["truncation",{"2":{"97":2}}],["truncated",{"2":{"26":4,"28":2}}],["truly",{"2":{"83":1}}],["true",{"0":{"255":1},"2":{"2":2,"3":3,"10":1,"17":6,"19":4,"28":2,"39":1,"40":2,"41":1,"44":2,"46":1,"48":3,"49":2,"50":4,"52":11,"56":1,"59":2,"63":40,"64":3,"65":1,"68":1,"69":4,"77":8,"82":1,"99":1,"106":3,"109":1,"110":2,"116":2,"126":2,"129":2,"131":1,"132":1,"145":3,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"163":2,"167":1,"179":2,"186":4,"194":1,"200":2,"202":2,"216":1,"218":1,"220":58,"227":1,"234":3,"236":2,"241":1,"245":3,"247":2,"249":1,"255":1,"256":2}}],["trivial",{"2":{"83":1,"133":1,"173":1,"202":1,"220":26}}],["trilinear",{"2":{"56":6}}],["tries",{"2":{"16":1}}],["trigger",{"2":{"3":4,"4":2,"120":1,"130":1,"131":2}}],["trace",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":7,"150":3,"151":2,"152":10,"254":4}}],["tracing",{"2":{"65":1,"106":1}}],["tracking",{"0":{"110":1},"2":{"100":1,"111":2}}],["track",{"2":{"40":1,"48":2,"55":12,"67":1,"109":1,"110":5,"155":1}}],["tracker",{"2":{"16":1,"18":1,"20":1,"59":1,"62":1,"65":1,"102":1,"116":1,"236":12,"247":1}}],["tracked",{"2":{"109":1}}],["trackedarray",{"2":{"8":1,"116":2}}],["trackedreals",{"2":{"8":1}}],["traditional",{"2":{"53":1,"249":1}}],["transitioned",{"2":{"130":1}}],["transposed",{"2":{"49":2}}],["transpose",{"2":{"49":3,"171":1}}],["transformation",{"2":{"55":1}}],["transformations",{"2":{"44":1}}],["transforms",{"2":{"51":3}}],["transform",{"2":{"44":1}}],["transformed",{"2":{"15":1}}],["transferred",{"2":{"3":1,"140":1}}],["transferring",{"2":{"0":1,"5":1,"160":1}}],["transfer",{"0":{"2":1,"160":1},"2":{"2":1,"140":1,"160":2,"188":1}}],["train=1024",{"2":{"224":1}}],["trainstate",{"0":{"107":1},"2":{"62":11,"77":3,"78":1,"97":1,"106":2,"107":3,"179":1,"187":1,"195":1,"198":1,"217":1,"228":1,"233":1,"236":1,"245":4}}],["trained2",{"2":{"187":2}}],["trained",{"2":{"46":1,"187":2,"188":8,"202":2,"203":2,"236":5,"237":1,"259":3}}],["train",{"0":{"219":1},"2":{"46":1,"52":24,"62":6,"69":1,"77":8,"78":7,"106":1,"107":10,"179":4,"183":4,"187":12,"190":1,"192":6,"195":15,"196":3,"198":2,"202":3,"213":6,"217":8,"224":10,"228":24,"233":1,"235":1,"236":9,"245":1}}],["trainmode",{"2":{"10":1}}],["trainable",{"2":{"7":1,"38":1,"49":2,"52":4,"53":3,"55":1,"62":2,"69":9,"140":9}}],["training=val",{"2":{"50":3}}],["training",{"0":{"62":1,"119":1,"142":1,"181":1,"187":1,"195":1,"196":1,"198":1,"200":1,"202":1,"217":1,"222":1,"228":1,"230":1,"236":1,"245":1,"252":1,"257":1,"258":1},"1":{"120":1,"121":1,"122":1,"123":1,"124":1,"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1,"199":1,"200":1,"201":1,"202":1,"203":1,"204":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"238":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"4":3,"10":2,"17":9,"19":8,"28":2,"31":2,"32":4,"40":1,"50":6,"55":9,"62":14,"63":2,"64":3,"74":1,"77":8,"78":3,"83":2,"97":1,"98":2,"106":1,"107":5,"109":1,"110":3,"119":2,"126":2,"128":1,"129":2,"142":2,"179":5,"181":1,"187":2,"190":1,"195":4,"196":20,"198":1,"217":5,"219":1,"228":2,"233":1,"236":2,"244":1,"245":5,"257":2,"258":1}}],["t",{"2":{"2":1,"3":5,"4":6,"8":1,"13":2,"16":1,"18":1,"28":20,"29":1,"39":1,"41":1,"44":1,"52":1,"54":1,"57":1,"59":1,"62":1,"64":1,"69":6,"77":1,"78":1,"83":1,"90":2,"93":1,"97":1,"104":2,"105":1,"108":1,"111":1,"113":1,"114":1,"115":4,"117":2,"123":1,"124":1,"135":4,"137":3,"138":1,"140":1,"142":1,"143":1,"144":2,"145":1,"146":1,"147":10,"149":1,"153":1,"171":2,"173":2,"177":1,"179":1,"183":1,"184":4,"185":2,"188":1,"192":1,"195":3,"198":1,"200":6,"201":11,"202":8,"203":9,"209":1,"213":1,"214":5,"218":3,"225":1,"226":1,"228":2,"235":6,"236":12,"245":1,"254":16,"255":1,"256":1,"257":2}}],["thousands",{"2":{"171":1}}],["though",{"2":{"135":1,"159":1,"175":1}}],["those",{"2":{"7":1,"8":1,"56":1,"59":1,"79":1,"110":1,"124":1,"136":1,"171":1,"214":1,"250":1}}],["thunk",{"2":{"106":1}}],["thumb",{"2":{"56":1}}],["thus",{"2":{"19":1,"48":1,"56":1}}],["threads",{"2":{"180":3,"189":3,"197":3,"204":3,"221":3,"229":3,"238":3,"246":3,"251":3,"260":3}}],["three",{"2":{"54":1,"171":1}}],["through",{"2":{"5":1,"8":1,"48":4,"55":4,"76":1,"84":2,"114":1,"137":1,"140":1,"154":1,"173":1,"184":3,"201":1,"250":1}}],["throws",{"2":{"60":1}}],["throw",{"2":{"2":1,"3":1,"40":1,"62":1,"67":1,"90":1,"236":1}}],["thrown",{"2":{"2":2,"38":1,"40":2}}],["than",{"2":{"28":2,"175":1,"217":1}}],["that",{"2":{"2":3,"3":8,"5":1,"7":4,"8":5,"10":1,"11":1,"13":2,"16":1,"17":1,"18":2,"20":1,"28":3,"34":2,"37":1,"38":1,"40":1,"41":2,"46":2,"48":5,"49":2,"50":2,"51":6,"53":1,"54":4,"55":6,"57":1,"62":1,"63":1,"64":1,"65":3,"68":1,"69":5,"77":1,"79":1,"83":2,"85":2,"90":2,"93":2,"105":3,"106":1,"107":2,"108":2,"109":3,"110":1,"111":2,"120":5,"121":1,"123":1,"135":3,"136":4,"137":1,"138":2,"139":1,"140":2,"143":1,"144":3,"145":2,"146":3,"147":2,"149":2,"152":3,"153":2,"155":3,"156":1,"161":1,"163":1,"167":1,"171":1,"173":3,"174":1,"176":1,"177":1,"184":3,"188":2,"198":2,"202":4,"209":1,"217":1,"220":1,"232":1,"233":1,"235":1,"242":1,"244":1,"249":1,"250":3,"256":3}}],["third",{"2":{"171":2}}],["thirteenth",{"2":{"28":2}}],["think",{"2":{"69":1,"135":1}}],["things",{"2":{"69":1,"135":1,"137":1,"171":1}}],["thing",{"2":{"54":2,"153":1,"155":1}}],["this",{"2":{"2":3,"3":8,"4":10,"5":1,"7":5,"8":6,"10":1,"11":6,"13":3,"14":1,"16":3,"18":3,"19":2,"20":4,"22":2,"23":2,"25":1,"28":5,"31":2,"32":3,"35":1,"36":1,"37":2,"39":2,"40":5,"44":6,"46":3,"48":3,"49":5,"51":3,"52":6,"53":5,"54":1,"55":5,"56":3,"57":2,"58":1,"59":2,"62":6,"63":4,"64":1,"65":12,"67":7,"68":6,"69":11,"74":2,"79":1,"82":1,"83":2,"85":2,"90":6,"91":1,"92":1,"93":3,"98":1,"99":1,"100":1,"101":1,"103":2,"105":1,"106":5,"107":3,"108":2,"109":3,"111":2,"120":3,"122":2,"124":2,"125":2,"127":1,"130":1,"131":2,"133":1,"135":8,"137":6,"138":1,"140":2,"142":3,"144":6,"145":9,"146":5,"147":1,"148":1,"149":3,"150":1,"152":2,"153":5,"155":7,"156":1,"157":1,"159":1,"160":4,"162":1,"163":2,"164":3,"166":1,"167":1,"170":1,"171":5,"173":4,"175":1,"179":1,"180":1,"181":2,"183":1,"185":2,"189":1,"190":1,"193":1,"196":1,"197":1,"198":3,"202":5,"204":1,"217":1,"218":1,"220":2,"221":1,"225":1,"229":1,"230":2,"238":1,"239":1,"242":1,"244":1,"246":1,"247":1,"249":4,"251":1,"252":1,"254":3,"256":2,"258":1,"260":1}}],["theoretical",{"2":{"254":1}}],["theoretically",{"2":{"175":1}}],["theorem",{"2":{"54":1}}],["thesis",{"2":{"74":1}}],["these",{"2":{"7":5,"8":1,"29":1,"30":1,"32":1,"37":1,"44":3,"46":3,"55":1,"62":1,"77":1,"81":2,"83":1,"90":2,"97":3,"104":7,"114":1,"124":1,"135":2,"136":2,"144":1,"148":1,"159":1,"163":1,"166":1,"167":3,"170":1,"173":1,"177":1,"209":3,"210":1,"211":1,"249":1}}],["they",{"2":{"62":1,"69":2,"110":1,"126":1,"140":1,"171":1,"175":2}}],["there",{"2":{"38":1,"46":1,"62":1,"77":1,"92":1,"103":1,"105":1,"110":1,"111":1,"120":1,"144":1,"147":2,"155":1,"159":1,"169":1,"171":1,"196":1}}],["them",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"37":1,"39":1,"48":1,"53":1,"56":1,"63":1,"69":1,"77":1,"81":1,"82":1,"90":1,"97":1,"104":1,"153":1,"155":1,"166":1,"171":2,"174":1,"177":1,"210":2,"233":1,"249":1,"256":2}}],["their",{"2":{"7":2,"49":1,"84":1,"158":1,"171":1,"173":2}}],["then",{"2":{"1":2,"2":6,"3":1,"8":1,"17":2,"19":1,"23":2,"38":3,"40":2,"48":5,"49":4,"50":4,"52":9,"53":7,"54":1,"58":1,"59":2,"60":1,"63":6,"64":1,"65":2,"68":2,"70":1,"74":1,"76":1,"83":1,"106":1,"131":3,"135":1,"136":3,"149":2,"161":1}}],["the",{"0":{"107":1,"116":1,"117":1,"118":1,"128":1,"148":2,"150":1,"151":1,"152":1,"170":1,"185":1,"187":1,"188":1,"193":1,"195":1,"196":1,"201":1,"202":1,"203":1,"214":1,"215":1,"219":1,"226":1,"233":1,"234":1,"235":1,"237":1,"249":1,"255":1,"257":1,"258":1,"259":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"1":6,"2":12,"3":31,"4":14,"5":13,"6":3,"7":19,"8":22,"9":4,"10":7,"11":6,"13":10,"14":2,"15":3,"16":11,"17":16,"18":6,"19":21,"20":5,"22":20,"23":13,"24":1,"28":60,"29":25,"30":1,"32":10,"33":4,"34":11,"35":5,"36":3,"37":1,"38":14,"39":22,"40":15,"41":14,"44":12,"46":9,"48":46,"49":38,"50":6,"51":45,"52":69,"53":19,"54":33,"55":58,"56":14,"58":6,"59":9,"60":4,"62":35,"63":55,"64":4,"65":13,"66":6,"67":19,"68":18,"69":49,"70":6,"71":7,"72":2,"73":2,"76":3,"77":6,"78":8,"79":1,"81":1,"82":5,"83":6,"84":4,"85":2,"87":2,"90":2,"91":2,"92":2,"93":4,"94":1,"97":11,"99":9,"100":4,"101":3,"103":7,"104":2,"105":1,"106":23,"107":7,"108":3,"109":6,"110":19,"113":2,"114":2,"115":1,"117":3,"118":2,"119":1,"120":22,"121":3,"122":1,"123":6,"126":1,"127":7,"128":3,"130":3,"131":5,"133":6,"135":20,"136":10,"137":4,"138":5,"139":4,"140":6,"142":4,"143":1,"144":9,"145":5,"146":16,"147":5,"148":13,"149":11,"150":2,"152":7,"153":10,"154":1,"155":9,"157":1,"158":3,"159":2,"160":8,"161":4,"162":1,"163":1,"164":1,"165":1,"166":10,"167":4,"169":11,"170":2,"171":12,"172":7,"173":2,"174":5,"175":6,"176":1,"177":4,"178":2,"179":6,"181":2,"183":6,"184":24,"185":3,"186":2,"187":6,"188":5,"190":2,"192":2,"193":3,"195":2,"196":5,"198":5,"201":8,"202":11,"209":1,"210":6,"213":2,"214":13,"215":1,"217":3,"218":2,"219":3,"220":5,"225":6,"228":1,"230":2,"232":3,"233":2,"234":4,"235":3,"241":4,"244":3,"245":2,"247":5,"248":3,"249":16,"250":22,"252":1,"254":7,"255":2,"256":10,"257":4,"258":2,"259":2}}],["top",{"2":{"106":1,"133":1}}],["topic",{"2":{"52":1}}],["toolchain",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["tools",{"2":{"40":2,"69":1,"108":1,"170":1,"175":1}}],["too",{"2":{"81":1,"175":1,"250":1}}],["towards",{"2":{"65":1,"173":1,"198":1}}],["toarray",{"2":{"46":1}}],["tosimplechainsadaptor",{"2":{"46":4,"83":1,"193":2}}],["together",{"2":{"39":1}}],["total",{"2":{"9":1,"10":1,"33":2,"41":1,"48":4,"55":3,"69":4,"77":1,"78":1,"109":1,"110":1,"115":1,"193":2,"194":6,"216":6,"227":6,"236":3,"242":1,"256":1}}],["to",{"0":{"44":1,"46":1,"60":1,"71":1,"84":1,"85":1,"120":1,"139":1,"154":1,"252":1},"1":{"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"140":1,"141":1,"142":1,"143":1,"155":1,"156":1,"157":1,"158":1,"159":1,"160":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"1":3,"2":7,"3":10,"4":5,"5":5,"6":2,"7":4,"8":10,"10":2,"11":2,"13":7,"14":2,"15":4,"16":5,"17":13,"18":3,"19":16,"20":2,"22":8,"23":5,"28":14,"32":9,"34":6,"36":1,"37":1,"38":7,"39":2,"40":10,"41":5,"44":14,"46":17,"48":18,"49":19,"50":13,"51":9,"52":59,"53":7,"54":9,"55":22,"56":8,"57":2,"58":3,"59":10,"62":8,"63":14,"64":3,"65":9,"66":4,"67":4,"68":4,"69":21,"70":1,"71":2,"72":1,"73":1,"74":1,"76":3,"77":2,"78":2,"81":2,"82":1,"83":9,"84":2,"85":5,"87":1,"90":5,"91":1,"92":1,"93":5,"94":2,"97":6,"98":3,"99":6,"100":3,"104":3,"105":1,"106":14,"107":8,"108":4,"109":1,"110":7,"113":2,"114":2,"115":1,"119":1,"120":5,"121":3,"123":4,"125":1,"126":1,"127":1,"128":1,"130":3,"131":2,"133":5,"135":7,"136":5,"137":4,"138":2,"139":3,"140":8,"142":1,"143":3,"144":7,"145":10,"146":6,"147":1,"148":4,"149":6,"152":1,"153":9,"155":7,"156":1,"157":2,"158":1,"160":3,"161":4,"162":2,"163":3,"164":4,"166":4,"167":3,"168":1,"169":3,"170":4,"171":7,"172":1,"173":3,"175":1,"176":1,"177":3,"179":2,"181":2,"183":3,"184":10,"186":1,"188":1,"190":2,"192":1,"193":3,"198":4,"201":6,"202":7,"209":1,"210":6,"211":2,"213":1,"214":6,"215":1,"217":1,"218":1,"220":11,"225":1,"226":1,"230":2,"234":2,"235":2,"244":1,"247":1,"248":3,"249":12,"250":5,"254":5,"256":6,"257":2,"258":2}}],["toml",{"2":{"1":1,"57":1}}],["l3",{"2":{"259":2}}],["lb",{"2":{"256":1,"259":1}}],["lbfgs",{"2":{"198":1,"202":3}}],["lr",{"2":{"236":2}}],["ld",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["llvm",{"2":{"180":2,"189":2,"197":1,"204":2,"221":2,"229":2,"238":2,"246":2,"251":1,"260":1}}],["ll",{"2":{"170":1,"248":1,"249":1,"256":2}}],["l=",{"2":{"63":1}}],["ls",{"2":{"63":4}}],["lstmcell",{"2":{"52":6,"99":1,"100":1,"184":3,"185":1}}],["lstm",{"0":{"181":1},"1":{"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1},"2":{"7":1,"52":1,"184":17,"185":4}}],["lpnorm",{"2":{"100":1}}],["lppool",{"2":{"51":1,"100":1}}],["lp",{"2":{"51":4,"249":1,"250":1}}],["l2",{"2":{"41":5,"136":4,"256":2,"259":2}}],["l2=dense",{"2":{"41":1}}],["l1",{"2":{"41":5,"136":4,"256":2,"259":2}}],["l1=dense",{"2":{"41":1}}],["lecture",{"2":{"211":1}}],["lecun",{"2":{"63":1}}],["leveraging",{"2":{"153":1}}],["levels=10",{"2":{"237":1}}],["level",{"2":{"28":2,"101":1,"107":1,"153":1,"166":2}}],["lets",{"2":{"137":1,"195":1,"217":1,"228":1,"247":1}}],["let",{"2":{"69":1,"106":2,"107":1,"108":1,"109":2,"110":7,"121":1,"126":1,"135":1,"136":2,"137":2,"140":2,"144":3,"145":2,"147":1,"148":1,"152":3,"153":1,"155":1,"170":2,"171":1,"174":1,"176":1,"177":1,"178":1,"179":4,"184":1,"186":1,"188":1,"202":2,"217":1,"219":1,"241":1,"242":1,"245":1,"249":1,"255":1,"256":1,"257":1,"259":2}}],["len",{"2":{"59":2,"183":1,"235":11}}],["length=25",{"2":{"250":4}}],["length=bc",{"2":{"235":3}}],["length=grid",{"2":{"235":1}}],["length=datasize",{"2":{"200":1,"255":1}}],["length=50",{"2":{"183":1}}],["length",{"2":{"28":7,"29":24,"41":2,"48":2,"49":2,"51":6,"135":8,"136":2,"183":5,"186":1,"194":1,"201":2,"216":1,"227":1,"237":8,"249":3,"254":2,"259":1}}],["less",{"2":{"54":1,"81":1,"104":1,"175":1}}],["leibler",{"2":{"63":1}}],["lei",{"2":{"19":1}}],["lempitsky",{"2":{"19":1,"55":1}}],["leakyrelu",{"2":{"99":1}}],["learn",{"2":{"171":1}}],["learned",{"2":{"83":1}}],["learnable",{"2":{"53":1,"55":4}}],["learning",{"2":{"2":1,"17":1,"19":1,"25":1,"28":3,"63":1,"149":1,"155":1,"158":1,"168":1,"169":1,"171":1,"179":1,"198":1,"254":1}}],["leads",{"2":{"137":1}}],["leading",{"2":{"56":1}}],["lead",{"2":{"13":1,"40":1,"44":1,"65":1,"69":2,"130":1,"153":1,"202":1}}],["leaf",{"2":{"3":1,"8":1,"65":2,"77":12}}],["leaves",{"2":{"3":1,"65":4}}],["l",{"2":{"8":5,"9":2,"10":2,"38":13,"39":5,"44":2,"46":1,"48":2,"50":1,"115":5,"126":2,"127":2,"135":16,"140":4,"179":1,"184":2,"202":4,"214":4,"241":2,"245":2,"255":2,"257":3}}],["lag",{"2":{"236":1}}],["latter",{"2":{"158":2}}],["later",{"2":{"248":1}}],["latest",{"2":{"71":1,"210":1}}],["latentsize",{"2":{"52":3}}],["lazy",{"2":{"140":1}}],["language",{"2":{"65":2}}],["label=l",{"2":{"200":2,"203":4}}],["labels",{"2":{"63":2,"183":3,"192":2,"213":2,"224":4}}],["label",{"2":{"63":18}}],["larger",{"0":{"209":1},"2":{"83":1,"171":1,"209":1}}],["large",{"2":{"44":1,"83":3,"175":1}}],["layerfreezing",{"2":{"128":1}}],["layer2",{"2":{"48":1}}],["layer1",{"2":{"48":1}}],["layernorm",{"2":{"19":1,"55":4,"87":1}}],["layer",{"0":{"11":1,"39":1,"68":1,"69":1,"127":1,"134":1,"135":1,"136":1,"214":1,"215":1,"218":1,"225":1},"1":{"135":1,"136":1},"2":{"7":13,"8":9,"9":4,"10":4,"11":4,"19":3,"38":7,"39":6,"40":13,"44":10,"46":11,"48":67,"49":5,"50":3,"51":12,"52":2,"53":5,"54":8,"55":24,"56":3,"67":5,"68":7,"69":11,"77":47,"83":2,"97":2,"98":1,"99":1,"106":24,"108":2,"109":30,"110":69,"113":2,"115":3,"117":5,"126":9,"127":9,"128":5,"129":9,"135":8,"136":12,"140":5,"144":1,"145":4,"146":1,"147":4,"148":6,"153":2,"155":1,"166":1,"179":1,"184":1,"193":18,"214":2,"220":211,"225":1,"226":1,"242":2,"244":4,"245":4,"249":1,"256":12,"258":4}}],["layers",{"0":{"16":1,"18":1,"47":1,"49":1,"50":1,"51":1,"52":1,"53":1,"54":1,"55":1,"78":1,"126":1,"140":1,"143":1},"1":{"48":1,"49":1,"50":1,"51":1,"52":1,"53":1,"54":1,"55":1,"56":1},"2":{"6":1,"7":9,"8":2,"28":8,"39":3,"40":2,"44":2,"48":39,"49":1,"54":12,"67":1,"68":2,"78":1,"82":4,"83":4,"88":1,"97":2,"100":1,"109":9,"110":19,"115":4,"126":1,"133":2,"135":1,"136":3,"139":1,"140":3,"144":1,"155":1,"159":1,"165":1,"166":1,"184":1,"185":1,"214":1,"219":1,"220":6,"225":1,"249":2}}],["last",{"2":{"2":1,"15":1,"28":1,"52":1,"54":2,"56":2,"68":1,"69":1,"115":1,"120":1,"146":1,"214":1,"248":2}}],["luxlinear",{"2":{"140":7}}],["luxlib",{"0":{"12":1,"86":1},"1":{"13":1,"14":1,"15":1,"16":1,"17":1,"18":1,"19":1,"20":1,"87":1,"88":1},"2":{"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"20":1,"58":2,"70":2,"87":1,"145":4,"158":8,"161":1,"166":2,"167":2,"256":2}}],["luxflux",{"2":{"139":1,"140":2}}],["luxdeviceutils",{"2":{"93":3}}],["luxdl",{"2":{"52":1,"71":1,"79":1}}],["luxops",{"0":{"64":1},"2":{"64":8,"184":1,"185":1}}],["luxtestutils",{"0":{"57":1},"1":{"58":1,"59":1,"60":1},"2":{"58":3,"59":2,"60":1}}],["luxcorearrayinterfacereversediffext",{"2":{"116":2}}],["luxcorearrayinterfacetrackerext",{"2":{"116":2}}],["luxcore",{"0":{"6":1,"89":1},"1":{"7":1,"8":1,"9":1,"10":1,"11":1,"90":1,"91":1},"2":{"6":2,"7":3,"8":7,"9":2,"10":5,"11":1,"70":2,"90":2,"116":5,"133":3,"135":14,"136":8,"137":1,"161":1,"166":3}}],["luxcudadevice",{"2":{"94":1}}],["luxcuda",{"2":{"3":1,"72":2,"77":1,"78":1,"80":1,"130":1,"153":1,"168":1,"172":3,"182":1,"199":1,"212":1,"223":1,"231":1,"240":1}}],["lux",{"0":{"42":1,"44":1,"45":1,"46":1,"71":1,"81":1,"83":1,"85":1,"96":1,"106":1,"108":1,"133":1,"139":1,"161":1,"170":1,"198":1},"1":{"43":1,"44":1,"45":1,"46":2,"82":1,"83":1,"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":2,"98":2,"99":2,"100":2,"107":1,"109":1,"110":1,"111":1,"134":1,"135":1,"136":1,"137":1,"138":1,"140":1,"141":1,"142":1,"143":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1,"199":1,"200":1,"201":1,"202":1,"203":1,"204":1},"2":{"4":4,"6":4,"7":3,"8":4,"11":1,"12":1,"22":2,"23":1,"30":1,"31":2,"32":3,"33":2,"34":4,"35":1,"36":1,"38":13,"39":4,"40":8,"41":3,"44":11,"46":15,"48":11,"49":3,"50":7,"51":9,"52":9,"53":4,"54":12,"55":8,"56":2,"57":1,"58":2,"62":9,"63":16,"64":9,"65":9,"66":4,"67":2,"68":5,"69":19,"70":1,"71":5,"72":4,"73":3,"74":1,"76":4,"77":8,"78":4,"79":1,"80":1,"81":1,"83":9,"84":1,"85":2,"90":3,"93":1,"94":2,"97":6,"98":8,"101":2,"104":1,"105":2,"106":7,"107":1,"108":3,"109":6,"110":4,"111":1,"113":1,"115":3,"116":1,"117":4,"118":1,"119":1,"123":1,"126":6,"127":2,"128":2,"129":3,"130":3,"131":1,"133":7,"135":3,"136":2,"137":1,"139":3,"140":13,"142":2,"143":2,"144":4,"145":1,"146":1,"147":1,"148":1,"152":1,"153":8,"155":6,"157":2,"161":3,"166":3,"169":1,"170":3,"172":1,"173":1,"174":2,"175":1,"177":1,"179":2,"181":2,"182":1,"184":7,"185":1,"187":2,"190":1,"191":1,"193":4,"194":1,"195":1,"196":2,"198":2,"199":1,"202":1,"209":1,"210":3,"212":1,"214":5,"215":1,"216":1,"218":4,"220":96,"223":1,"225":4,"226":2,"227":1,"228":1,"230":2,"231":1,"232":1,"233":1,"236":6,"240":1,"244":2,"245":4,"247":4,"249":7,"253":1,"256":2}}],["lt",{"2":{"4":2,"5":1,"28":3,"63":1,"69":2,"79":1,"90":2,"131":1,"136":2,"144":12,"200":1,"203":1}}],["literate",{"2":{"180":2,"189":2,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2,"251":2,"260":2}}],["literature",{"2":{"149":1}}],["little",{"2":{"106":1,"110":1}}],["lib64",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["libllvm",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["libopenlibm",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["libm",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["libraries",{"2":{"144":1,"169":1,"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1,"247":2,"249":1}}],["library",{"2":{"74":1,"139":1,"173":1,"180":1,"188":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["lib",{"2":{"109":1,"180":1,"189":1,"197":1,"204":1,"220":3,"221":1,"229":1,"236":4,"238":1,"246":1}}],["linux",{"2":{"180":2,"189":2,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2,"251":2,"260":2}}],["linked",{"2":{"210":1}}],["link",{"2":{"170":1}}],["linking",{"2":{"48":1}}],["lin",{"2":{"63":2}}],["liner",{"2":{"249":1}}],["linewidth=3",{"2":{"241":1,"245":1,"250":3}}],["linewidth=2",{"2":{"237":1,"255":1,"256":2,"259":3}}],["linewidth=4",{"2":{"200":2,"203":4,"259":1}}],["linesearch=linesearches",{"2":{"258":1}}],["linesearches",{"2":{"253":1}}],["linestyle=",{"2":{"200":2,"203":2}}],["lines",{"2":{"200":2,"203":4,"241":1,"245":1,"255":1,"256":2,"259":4}}],["line",{"2":{"59":1}}],["linearalgebra",{"2":{"144":1,"247":1}}],["linear",{"0":{"53":1,"179":1},"2":{"28":1,"54":1,"56":1,"69":4,"135":12,"136":23,"153":1,"179":1}}],["lighter",{"2":{"133":1}}],["light",{"2":{"25":1}}],["lightweight",{"2":{"0":1}}],["like",{"2":{"15":1,"16":2,"18":1,"20":1,"23":1,"28":1,"49":2,"52":3,"57":1,"62":1,"63":3,"65":1,"69":3,"83":2,"91":1,"133":1,"135":2,"137":1,"139":1,"140":2,"160":1,"169":1,"171":3,"177":1,"198":1,"214":1,"217":1,"248":1,"255":1,"256":1}}],["limits=extrema",{"2":{"237":1}}],["limiter",{"2":{"220":26}}],["limitations",{"2":{"44":1,"62":1,"83":1}}],["limit",{"2":{"13":1,"17":1,"64":1,"180":2,"189":2,"197":1,"204":2,"221":2,"229":2,"238":2,"246":2,"251":1,"260":1}}],["lists",{"2":{"41":1,"101":1}}],["listed",{"2":{"37":1,"67":1}}],["list",{"2":{"3":1,"32":1,"41":5,"48":4,"54":1,"58":1,"59":3,"78":1,"85":2,"161":1,"168":1,"171":1,"210":1,"228":4}}],["lost",{"2":{"170":1}}],["lossfn",{"2":{"179":3,"186":2,"187":2}}],["lossfunctions",{"2":{"63":1}}],["losses",{"2":{"63":1,"257":2,"259":3}}],["loss",{"0":{"63":1,"145":1,"147":1,"148":1,"186":1,"234":1,"244":1},"1":{"146":1},"2":{"62":5,"63":35,"69":5,"77":2,"78":14,"106":5,"107":4,"144":1,"145":10,"146":9,"147":4,"148":6,"179":19,"186":4,"187":456,"194":1,"195":2,"202":30,"227":1,"228":1,"230":1,"232":1,"234":23,"236":436,"244":4,"245":10,"257":6,"258":1,"259":2}}],["lower",{"2":{"217":1}}],["low",{"2":{"107":1}}],["love",{"2":{"104":1}}],["lotka",{"2":{"200":2}}],["lot",{"2":{"83":1,"174":1}}],["logs",{"2":{"120":1}}],["logging",{"2":{"120":1,"247":1}}],["logarithm",{"2":{"63":1}}],["logarithmic",{"2":{"63":1}}],["log⁡",{"2":{"63":1}}],["log",{"2":{"63":2,"64":2,"249":1,"250":1}}],["logitcrossentropy",{"2":{"216":1,"217":1}}],["logitbinarycrossentropy",{"2":{"186":1}}],["logitbce",{"2":{"63":5}}],["logit",{"2":{"63":1}}],["logits=val",{"2":{"63":2,"194":1,"216":1,"227":1}}],["logits",{"2":{"63":4}}],["longer",{"2":{"97":2,"122":3}}],["long",{"2":{"44":1,"52":1,"137":1}}],["location",{"2":{"40":3,"109":4,"110":10}}],["local",{"2":{"4":1,"33":2,"69":1,"120":1,"121":1,"180":2,"189":2,"196":1,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2}}],["localpreferences",{"2":{"1":1,"131":2}}],["loosely",{"2":{"170":1}}],["looks",{"2":{"160":1}}],["lookup",{"2":{"53":1}}],["look",{"2":{"38":1,"56":1,"177":1,"184":1,"211":1}}],["loops",{"2":{"20":1,"65":1}}],["loopedarrayop",{"2":{"20":1}}],["loopvectorization",{"2":{"13":1,"14":2,"18":1,"20":1,"159":1,"167":1}}],["loop",{"0":{"167":1,"195":1},"2":{"13":1,"20":1,"159":1,"167":1,"245":1}}],["looping",{"2":{"5":1}}],["lo",{"2":{"28":2}}],["loadmnist",{"2":{"192":2,"195":1,"213":2,"217":1}}],["loader",{"2":{"107":1,"187":4}}],["loaded",{"2":{"2":2,"3":7,"4":2,"10":1,"11":1,"18":1,"36":1,"90":2,"120":1,"131":2,"143":1,"167":1,"172":1}}],["load",{"2":{"13":1,"14":1,"18":1,"36":1,"57":1,"97":1,"107":1,"130":1,"159":1,"188":1,"192":1,"213":1,"224":4,"228":1}}],["loading",{"0":{"160":1,"192":1,"213":1,"224":1},"2":{"7":1,"14":1,"18":1,"44":1,"46":1,"80":1,"160":1,"188":1}}],["=ϕwhere",{"2":{"255":1,"256":1}}],["=χu",{"2":{"255":1,"256":1}}],["=∫θp",{"2":{"250":1}}],["=wx+b",{"2":{"179":1}}],["=x",{"2":{"176":1}}],["=12xtx",{"2":{"176":1}}],["=1v∑i=1vvitjvinote",{"2":{"149":1}}],["=1v∑i=1vvitaviwe",{"2":{"149":1}}],["=e",{"2":{"149":1}}],["=i",{"2":{"149":1}}],["=vcat",{"2":{"52":1}}],["=val",{"2":{"46":1,"63":1}}],["=true",{"2":{"40":1}}],["=static",{"2":{"40":1}}],["=>",{"2":{"38":2,"39":3,"41":8,"44":2,"46":5,"48":6,"49":2,"52":4,"53":4,"55":12,"69":8,"77":6,"78":8,"106":6,"107":3,"109":17,"110":14,"115":4,"139":4,"145":2,"146":2,"147":4,"148":4,"152":4,"155":1,"179":1,"184":2,"185":2,"193":15,"215":5,"226":1,"233":4,"242":4,"245":2,"249":3,"256":6}}],["=0",{"2":{"28":2}}],["=nothing",{"2":{"2":1,"52":1,"55":1,"63":2}}],["==3",{"2":{"254":1}}],["===",{"2":{"41":4}}],["==",{"2":{"1":1,"3":1,"17":1,"23":1,"28":4,"49":10,"51":15,"53":3,"56":2,"64":2,"68":1,"69":1,"78":2,"107":2,"115":1,"120":1,"127":1,"128":2,"137":2,"138":1,"179":2,"186":1,"194":1,"202":1,"216":1,"227":1,"228":2,"236":2,"245":2,"249":1,"254":5}}],["=",{"2":{"1":2,"2":2,"3":1,"5":8,"15":1,"17":3,"19":6,"23":1,"28":14,"31":3,"38":2,"39":5,"41":8,"44":4,"46":5,"48":40,"49":7,"50":6,"51":9,"52":6,"53":14,"54":25,"55":15,"56":3,"58":3,"59":2,"63":59,"65":1,"68":2,"69":46,"72":4,"73":3,"74":13,"77":52,"78":26,"90":2,"106":61,"107":10,"109":25,"110":47,"115":10,"116":3,"117":3,"118":2,"120":7,"121":1,"126":29,"127":1,"129":23,"131":4,"132":7,"135":8,"136":9,"137":5,"139":7,"140":12,"145":21,"146":12,"147":20,"148":21,"150":2,"151":2,"152":12,"153":19,"155":5,"160":6,"168":9,"169":6,"170":1,"171":8,"172":1,"173":3,"174":3,"176":3,"177":4,"178":1,"179":17,"180":8,"183":6,"184":6,"185":5,"186":5,"187":13,"189":8,"192":7,"193":21,"194":5,"195":12,"196":1,"197":8,"199":2,"200":12,"201":3,"202":19,"203":6,"204":8,"213":7,"214":6,"215":5,"216":5,"217":10,"218":3,"220":136,"221":8,"224":4,"225":2,"226":3,"227":5,"228":21,"229":8,"231":2,"234":13,"235":19,"236":20,"237":11,"238":8,"241":8,"242":3,"243":1,"244":12,"245":13,"246":8,"248":21,"249":19,"250":19,"251":7,"254":55,"255":20,"256":38,"257":4,"258":14,"259":13,"260":7}}],["jointly",{"2":{"184":1}}],["journal",{"2":{"17":1}}],["jvi",{"2":{"149":1}}],["jvp",{"0":{"22":1},"2":{"22":1,"151":6,"152":10,"175":4,"177":4}}],["j∈rd×d",{"2":{"149":1}}],["jet`",{"2":{"58":1}}],["jet",{"0":{"58":1},"2":{"58":14}}],["j",{"2":{"23":2,"28":1,"145":2,"146":3,"148":2,"149":1,"152":8}}],["jax",{"2":{"170":1}}],["jacobian`",{"2":{"145":1}}],["jacobian",{"0":{"145":1,"146":1,"148":1,"150":1,"151":1,"152":1,"177":1,"178":1},"1":{"146":1},"2":{"22":11,"23":6,"144":3,"145":3,"146":9,"148":5,"149":4,"150":1,"151":1,"152":17,"175":6,"177":2,"178":1}}],["jamie",{"2":{"19":1}}],["jimmy",{"2":{"19":1}}],["just",{"2":{"3":1,"48":1,"50":3,"55":3,"69":1,"77":1,"123":1,"131":1,"133":2,"169":1,"171":1,"177":1,"217":1}}],["julia∂t",{"2":{"147":1}}],["julia∂x",{"2":{"145":1,"146":1,"148":1}}],["juliazygote",{"2":{"110":1}}],["juliazeros",{"2":{"171":2}}],["juliazerosc64",{"2":{"29":1}}],["juliazerosc32",{"2":{"29":1}}],["juliazerosc16",{"2":{"29":1}}],["juliazeros64",{"2":{"29":1}}],["juliazeros32",{"2":{"29":1}}],["juliazeros16",{"2":{"29":1}}],["juliaxt",{"2":{"117":1}}],["juliax",{"2":{"106":1,"131":1,"155":2,"171":11,"173":2,"179":1}}],["juliaxlogy",{"2":{"64":1}}],["juliaxlogx",{"2":{"64":1}}],["juliaxla",{"2":{"2":1}}],["juliaeachslice",{"2":{"64":1}}],["juliaembedding",{"2":{"53":1}}],["juliakldivergenceloss",{"2":{"63":1}}],["juliakaiming",{"2":{"28":2}}],["juliahuberloss",{"2":{"63":1}}],["juliahingeloss",{"2":{"63":1}}],["juliapkg>",{"2":{"161":1}}],["juliaprob",{"2":{"255":1,"256":1,"259":1}}],["juliaprintln",{"2":{"152":1,"176":1}}],["juliapred",{"2":{"106":2}}],["juliaps",{"2":{"110":1,"120":1,"179":1,"187":1}}],["juliapoissonloss",{"2":{"63":1}}],["juliapixelshuffle",{"2":{"56":1}}],["juliaparallel",{"2":{"48":1}}],["juliaparameterlength",{"2":{"9":1}}],["juliapairwisefusion",{"2":{"48":1}}],["juliaw",{"2":{"171":1,"179":1}}],["juliaweights",{"2":{"168":1,"169":3}}],["juliaweightnorm",{"2":{"55":1}}],["juliawrappedfunction",{"2":{"54":1}}],["juliann",{"2":{"153":1}}],["julian",{"2":{"78":1,"179":1}}],["julianooplayer",{"2":{"54":1}}],["juliancclbackend",{"2":{"31":1}}],["juliamodel",{"2":{"106":2,"107":1,"109":2,"110":4,"118":1,"136":1,"146":1,"152":1,"179":1,"220":2,"242":1}}],["juliamultigate",{"2":{"64":1}}],["juliamsleloss",{"2":{"63":1}}],["juliamseloss",{"2":{"63":1}}],["juliamatch",{"2":{"67":1}}],["juliamaeloss",{"2":{"63":1}}],["juliamaxout",{"2":{"54":1}}],["juliamaxpool",{"2":{"51":1}}],["juliameanpool",{"2":{"51":1}}],["juliampibackend",{"2":{"31":1}}],["juliavjp",{"2":{"178":1,"245":1}}],["juliavariationalhiddendropout",{"2":{"50":1}}],["juliavector",{"2":{"22":1}}],["juliay",{"2":{"48":2}}],["juliausing",{"2":{"58":1,"72":8,"73":4,"77":1,"78":1,"106":1,"109":1,"110":1,"115":1,"126":1,"129":1,"135":1,"137":1,"139":2,"140":2,"144":1,"153":1,"155":1,"156":1,"161":1,"168":2,"170":1,"172":1,"175":1,"179":1,"180":1,"182":1,"189":1,"191":1,"197":1,"199":1,"204":1,"212":1,"221":1,"223":1,"229":1,"231":1,"238":1,"240":1,"246":1,"251":1,"253":1,"260":1}}],["juliaupsample",{"2":{"56":1}}],["juliaupdate",{"2":{"10":1}}],["juliaunfreeze",{"2":{"38":2}}],["juliarng",{"2":{"78":1,"109":1,"116":1,"135":1,"140":2,"170":1,"174":1,"241":1}}],["juliarnncell",{"2":{"52":1}}],["juliarandom",{"2":{"174":1}}],["juliarandc64",{"2":{"29":1}}],["juliarandc32",{"2":{"29":1}}],["juliarandc16",{"2":{"29":1}}],["juliarand64",{"2":{"29":1}}],["juliarandnc64",{"2":{"29":1}}],["juliarandnc32",{"2":{"29":1}}],["juliarandnc16",{"2":{"29":1}}],["juliarandn64",{"2":{"29":1}}],["juliarandn32",{"2":{"29":1}}],["juliarandn16",{"2":{"29":1}}],["juliarand32",{"2":{"29":1}}],["juliarand16",{"2":{"29":1}}],["juliarecursive",{"2":{"65":6}}],["juliarecurrence",{"2":{"52":1}}],["juliareversesequence",{"2":{"54":1}}],["juliareshapelayer",{"2":{"54":1}}],["juliares",{"2":{"48":2}}],["juliareset",{"2":{"3":1}}],["juliarepeatedlayer",{"2":{"48":1}}],["juliareplicate",{"2":{"8":1}}],["juliareduce",{"2":{"34":1}}],["juliaopt",{"2":{"120":1,"243":1}}],["juliaonesc64",{"2":{"29":1}}],["juliaonesc32",{"2":{"29":1}}],["juliaonesc16",{"2":{"29":1}}],["juliaones64",{"2":{"29":1}}],["juliaones32",{"2":{"29":1}}],["juliaones16",{"2":{"29":1}}],["juliaorthogonal",{"2":{"28":1}}],["juliaoutputsize",{"2":{"11":1}}],["juliajvp",{"2":{"177":1}}],["juliajet",{"2":{"58":1}}],["juliajacobian",{"2":{"22":1}}],["juliajulia>",{"2":{"5":1,"28":2,"38":1,"39":1,"41":1,"44":1,"46":1,"48":3,"54":4,"55":3,"58":1,"59":1,"63":15,"69":6,"71":2}}],["julialux",{"2":{"193":1}}],["julialuxops",{"2":{"64":1}}],["julialength",{"2":{"171":1}}],["julialang",{"2":{"109":1,"180":1,"189":1,"197":1,"204":1,"220":3,"221":1,"229":1,"236":4,"238":1,"246":1,"251":1,"260":1}}],["julialayer",{"2":{"39":1}}],["julialayernorm",{"2":{"19":1,"55":1}}],["julialstmcell",{"2":{"52":1}}],["julialppool",{"2":{"51":1}}],["julialossfn",{"2":{"179":1}}],["julialoss",{"2":{"106":1,"257":1}}],["julialocal",{"2":{"33":1}}],["julialoaded",{"2":{"3":1}}],["juliabegin",{"2":{"241":1,"245":1,"259":1}}],["juliabackend",{"2":{"120":1,"153":2}}],["juliabatchnorm",{"2":{"19":1,"55":1}}],["juliabatched",{"2":{"14":1,"23":1}}],["juliabinaryfocalloss",{"2":{"63":1}}],["juliabinarycrossentropyloss",{"2":{"63":1}}],["juliabilinear",{"2":{"53":1}}],["juliabidirectionalrnn",{"2":{"52":1}}],["juliabias",{"2":{"15":2}}],["juliabranchlayer",{"2":{"48":1}}],["juliabcast",{"2":{"34":1}}],["juliafig",{"2":{"250":2}}],["juliaf",{"2":{"176":1,"177":1}}],["juliafmap",{"2":{"106":1}}],["juliaf64",{"2":{"66":1}}],["juliaf32",{"2":{"66":1}}],["juliaf16",{"2":{"66":1}}],["juliafor",{"2":{"174":1}}],["juliafoldl",{"2":{"64":1}}],["juliafocalloss",{"2":{"63":1}}],["juliaflattenlayer",{"2":{"54":1}}],["juliafluxlayer",{"2":{"44":1}}],["juliafromfluxadaptor",{"2":{"44":1}}],["juliafrozenlayer",{"2":{"38":1}}],["juliafreeze",{"2":{"38":2}}],["juliafunction",{"2":{"106":2,"110":1,"117":1,"135":2,"145":1,"147":1,"148":1,"150":1,"151":1,"152":1,"179":1,"183":1,"184":2,"185":1,"187":1,"192":1,"195":1,"200":1,"202":1,"213":1,"214":2,"215":1,"217":1,"224":1,"225":2,"226":1,"228":1,"234":1,"236":1,"241":1,"245":1,"249":1,"254":4,"255":1,"256":1}}],["juliafunctional",{"2":{"3":1}}],["juliafused",{"2":{"16":1,"18":1}}],["juliafast",{"2":{"13":2}}],["juliatstate",{"2":{"245":1}}],["juliats",{"2":{"237":1}}],["juliatest",{"2":{"59":1}}],["juliatestmode",{"2":{"10":1}}],["juliatosimplechainsadaptor",{"2":{"46":1}}],["juliatotal",{"2":{"33":1}}],["juliatr",{"2":{"152":1,"196":1}}],["juliatry",{"2":{"109":1,"110":1}}],["juliatrain",{"2":{"196":1,"217":4,"219":1}}],["juliatrainstate",{"2":{"62":1}}],["juliatrainmode",{"2":{"10":1}}],["juliatruncated",{"2":{"28":1}}],["juliaimport",{"2":{"76":1}}],["juliaistraining",{"2":{"64":1}}],["juliaisleaf",{"2":{"3":1}}],["juliaidentity",{"2":{"28":1}}],["juliainput",{"2":{"153":1}}],["juliainternal",{"2":{"20":1}}],["juliainstancenorm",{"2":{"19":1,"55":1}}],["juliainitialized",{"2":{"32":1}}],["juliainitialize",{"2":{"32":1}}],["juliainitialstates",{"2":{"10":1}}],["juliainitialparameters",{"2":{"9":1}}],["juliadudt",{"2":{"203":1}}],["juliadataloader",{"2":{"160":2}}],["juliadata",{"2":{"120":1}}],["juliadicecoeffloss",{"2":{"63":1}}],["juliadistributedutils",{"2":{"120":1}}],["juliadistributeddatacontainer",{"2":{"36":1}}],["juliadistributedoptimizer",{"2":{"35":1}}],["juliadisplay",{"2":{"8":1}}],["juliadropout",{"2":{"17":1,"50":1}}],["juliadense",{"2":{"53":1}}],["juliadebuglayer",{"2":{"40":1}}],["juliadeviceiterator",{"2":{"5":1}}],["juliadefault",{"2":{"3":1}}],["juliacdev",{"2":{"131":1,"132":1}}],["juliacrossentropyloss",{"2":{"63":1}}],["juliacompute",{"2":{"62":1}}],["juliaconst",{"2":{"106":1,"186":1,"194":1,"216":1,"227":1,"244":1,"249":1,"256":2,"257":2}}],["juliaconvtranspose",{"2":{"49":1}}],["juliaconv",{"2":{"49":1}}],["juliacontains",{"2":{"8":1}}],["juliachain",{"2":{"48":1}}],["juliacheck",{"2":{"8":1}}],["juliacpu",{"2":{"2":1}}],["juliaadtype",{"2":{"258":1}}],["juliaadaptor",{"2":{"193":1}}],["juliaadaptivemeanpool",{"2":{"51":1}}],["juliaadaptivemaxpool",{"2":{"51":1}}],["juliaadaptivelppool",{"2":{"51":1}}],["juliaadapt",{"2":{"44":1,"46":1}}],["juliaanalytical",{"2":{"235":1}}],["juliaallreduce",{"2":{"34":1}}],["juliaalphadropout",{"2":{"50":1}}],["juliaalpha",{"2":{"17":1}}],["juliaapply",{"2":{"8":1,"62":2}}],["juliaabstract",{"2":{"7":3}}],["julia>",{"2":{"5":4,"28":3,"39":6,"41":3,"44":6,"46":7,"48":3,"54":8,"58":1,"59":2,"63":45,"69":34,"71":2}}],["juliastruct",{"2":{"117":1,"136":1,"137":1,"184":1,"201":1,"214":1,"218":1,"233":1}}],["juliastatefulluxlayer",{"2":{"68":1}}],["juliastatefulrecurrentcell",{"2":{"52":1}}],["juliastatelength",{"2":{"10":1}}],["juliastateless",{"2":{"8":1}}],["juliasquaredhingeloss",{"2":{"63":1}}],["juliasize",{"2":{"171":1}}],["juliasiamesecontrastiveloss",{"2":{"63":1}}],["juliasingle",{"2":{"62":2}}],["juliasimplechainslayer",{"2":{"46":1}}],["juliaselectdim",{"2":{"54":1}}],["juliasetup",{"2":{"8":1}}],["juliaset",{"2":{"4":2,"70":1}}],["juliascale",{"2":{"53":1}}],["juliaskipconnection",{"2":{"48":1}}],["juliashare",{"2":{"41":1}}],["juliasynchronize",{"2":{"34":1}}],["juliasparse",{"2":{"28":1}}],["juliasupported",{"2":{"3":1}}],["juliagdev",{"2":{"131":1,"153":1}}],["juliagenericlossfunction",{"2":{"63":1}}],["juliagetproperty",{"2":{"64":1}}],["juliaget",{"2":{"3":2,"32":1}}],["juliagrucell",{"2":{"52":1}}],["juliagroupnorm",{"2":{"19":1,"55":1}}],["juliaglobalmeanpool",{"2":{"51":1}}],["juliaglobalmaxpool",{"2":{"51":1}}],["juliagloballppool",{"2":{"51":1}}],["juliaglorot",{"2":{"28":2}}],["juliagpu",{"2":{"1":1,"2":1}}],["julia",{"0":{"170":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"1":1,"16":1,"20":1,"40":1,"44":1,"58":1,"59":1,"60":1,"69":3,"70":1,"71":2,"74":1,"76":2,"77":3,"79":1,"81":1,"83":1,"84":1,"97":2,"101":1,"106":1,"116":3,"127":2,"128":2,"130":1,"131":1,"135":1,"144":1,"145":3,"147":1,"148":1,"152":1,"153":2,"154":1,"155":1,"167":2,"168":4,"170":2,"171":7,"174":1,"175":1,"180":12,"188":2,"189":12,"197":9,"204":12,"220":2,"221":12,"229":12,"234":1,"238":12,"246":12,"247":1,"248":1,"249":4,"250":2,"251":9,"254":1,"260":9}}],["jld2",{"2":{"182":1,"188":3}}],["jll",{"2":{"180":2,"189":2,"204":2,"221":2,"229":2,"238":2,"246":2}}],["jl`",{"2":{"18":1,"145":2}}],["jl",{"0":{"35":1,"36":1,"58":1,"71":1,"86":1,"89":1,"92":1,"93":1,"96":1,"106":1,"121":1,"161":1,"198":1},"1":{"87":1,"88":1,"90":1,"91":1,"94":1,"95":1,"97":1,"98":1,"99":1,"100":1,"107":1,"122":1,"123":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"199":1,"200":1,"201":1,"202":1,"203":1,"204":1},"2":{"0":1,"2":3,"3":8,"6":5,"7":2,"8":1,"10":2,"11":1,"12":1,"13":3,"14":1,"18":2,"20":2,"22":1,"23":1,"26":2,"32":8,"35":1,"36":2,"44":1,"46":4,"52":1,"57":1,"58":1,"59":6,"62":7,"63":2,"64":1,"70":1,"71":4,"76":2,"78":3,"79":1,"80":5,"83":2,"90":3,"93":5,"95":1,"97":1,"102":8,"103":7,"105":1,"106":3,"107":1,"109":2,"116":3,"119":1,"120":1,"121":1,"122":1,"123":1,"130":1,"133":5,"135":1,"145":2,"153":3,"157":2,"158":2,"159":2,"160":3,"161":3,"163":1,"167":2,"168":1,"170":1,"172":1,"173":1,"175":4,"177":1,"179":1,"180":1,"181":2,"189":1,"190":4,"193":1,"196":1,"197":1,"198":3,"202":3,"204":1,"210":3,"211":1,"214":1,"220":6,"221":1,"229":1,"230":3,"236":4,"238":1,"243":1,"246":1,"247":2,"251":1,"252":1,"255":1,"256":1,"260":1}}],["p=res",{"2":{"259":1}}],["p=θ",{"2":{"257":1}}],["p=params",{"2":{"256":1}}],["p=2",{"2":{"51":3}}],["p^",{"2":{"255":1,"256":1}}],["p^2",{"2":{"255":1}}],["pdes",{"2":{"230":1}}],["pde",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"238":1},"2":{"235":8,"236":2,"237":3}}],["pcie",{"2":{"180":1,"189":1,"204":1,"221":1,"246":1}}],["physics",{"2":{"234":6,"236":108,"256":1}}],["philosophies",{"2":{"140":1}}],["phase",{"2":{"55":4}}],["pptional",{"2":{"77":1,"78":1}}],["pkg>",{"2":{"161":2}}],["pkg",{"2":{"71":4,"72":9,"73":2,"76":2,"77":1,"79":2,"161":1,"180":2,"189":2,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2,"251":2,"260":2}}],["p2",{"2":{"63":6}}],["p1",{"2":{"63":4}}],["pipe",{"2":{"201":1}}],["pipeline",{"2":{"118":1}}],["pipelines",{"2":{"62":1}}],["pitfalls",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1}}],["pinn2dpde",{"2":{"236":8}}],["pinn",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"238":1},"2":{"230":1,"233":4,"236":5,"237":1}}],["pinns",{"2":{"147":1,"230":1}}],["pin",{"2":{"108":2}}],["pi",{"2":{"63":1}}],["pixel",{"2":{"56":2}}],["pixelshuffle",{"2":{"56":1}}],["pixels",{"2":{"51":3,"56":2}}],["pytorch",{"2":{"49":3,"53":1,"83":1,"99":3,"100":1,"170":1}}],["pmlr",{"2":{"19":1}}],["p",{"2":{"17":7,"44":1,"50":12,"69":2,"200":2,"202":4,"203":2,"214":6,"218":2,"250":2,"254":3,"255":6,"256":2,"258":1}}],["peel",{"2":{"184":1,"185":1}}],["peel`",{"2":{"184":1}}],["people",{"2":{"135":1}}],["pesky",{"2":{"110":1}}],["pended",{"2":{"97":1}}],["penalties",{"2":{"69":1}}],["penultimate",{"2":{"15":1,"16":1,"52":2}}],["perspective",{"2":{"254":1}}],["perceptron",{"2":{"239":1}}],["per",{"2":{"55":4,"146":1,"249":2}}],["periodic",{"2":{"49":1}}],["period",{"2":{"37":1}}],["perfect",{"2":{"28":1}}],["performs",{"2":{"131":1,"202":1}}],["performing",{"2":{"51":3}}],["performed",{"2":{"46":2,"49":1,"249":1}}],["performance",{"0":{"154":1,"159":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"13":1,"28":2,"40":1,"62":1,"69":3,"103":2,"104":2,"105":1,"130":2,"153":1,"154":2,"155":2,"159":1}}],["perform",{"2":{"8":1,"34":2,"49":3,"62":2,"140":1,"249":1,"254":1}}],["plot",{"2":{"245":1,"248":3,"250":9,"255":1,"256":1,"259":1}}],["plotting",{"0":{"203":1}}],["platform",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["plan",{"2":{"144":1}}],["places",{"2":{"69":2}}],["place",{"2":{"13":1,"15":2,"65":5,"123":1,"130":1}}],["plugin",{"2":{"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["plus",{"2":{"38":1,"41":1,"48":6,"55":7,"69":3,"77":1,"78":1,"109":2,"110":2,"115":1,"193":2,"242":1,"256":1}}],["please",{"2":{"4":2,"74":2,"76":1,"78":1,"84":1,"105":1,"111":1,"124":1,"144":1,"184":1,"198":1,"209":1,"210":3,"230":1}}],["push",{"2":{"257":1}}],["pullback",{"2":{"110":2}}],["pull",{"2":{"84":1,"90":1}}],["publisher",{"2":{"74":1}}],["public",{"2":{"3":1,"63":1}}],["purpose",{"2":{"175":1}}],["purposes",{"2":{"67":1,"137":1,"202":1,"213":1}}],["pure",{"2":{"8":1,"81":1,"82":1,"83":1,"173":1}}],["pseudorandom",{"2":{"174":1}}],["pseudo",{"2":{"50":3,"170":1}}],["ps",{"2":{"7":2,"8":5,"34":2,"38":2,"39":18,"41":17,"44":4,"46":2,"48":4,"54":9,"63":2,"67":1,"68":5,"69":22,"77":3,"78":6,"106":16,"107":6,"109":5,"110":12,"113":3,"115":2,"116":2,"117":6,"118":2,"120":2,"126":9,"127":7,"128":6,"129":2,"135":7,"136":7,"137":10,"139":5,"140":7,"144":3,"145":9,"146":9,"147":8,"148":11,"150":2,"151":2,"152":9,"153":7,"155":5,"179":7,"184":4,"186":2,"187":3,"188":3,"194":2,"195":2,"202":4,"203":1,"214":2,"215":5,"216":2,"217":2,"218":3,"220":15,"225":2,"227":2,"228":2,"234":4,"236":3,"244":1,"245":1,"249":10,"250":1,"256":2}}],["power",{"2":{"250":1}}],["powerful",{"2":{"171":1}}],["poisson",{"2":{"63":1}}],["poissonloss",{"2":{"63":2}}],["points",{"2":{"140":1,"179":1,"241":1,"248":4,"249":1,"250":1}}],["point",{"0":{"66":1},"2":{"4":2,"13":1,"66":3,"108":2,"120":1,"125":1,"130":1,"171":1,"255":1,"256":1}}],["pooling",{"0":{"51":1},"2":{"51":21,"100":1}}],["populated",{"2":{"184":2}}],["populate",{"2":{"40":1}}],["polynomial",{"0":{"239":1},"1":{"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"239":1,"241":1}}],["polyester",{"2":{"20":1}}],["polyalgorithm",{"2":{"18":1}}],["position=",{"2":{"200":1,"203":1,"256":1,"259":1}}],["position",{"2":{"54":2,"254":1}}],["possibly",{"2":{"49":2,"51":3,"103":1}}],["possible",{"2":{"1":1,"2":1,"7":1,"11":1,"13":3,"15":1,"16":2,"18":1,"32":2,"44":2,"104":1,"145":1}}],["posterior",{"2":{"250":2}}],["posted",{"2":{"144":1}}],["post",{"2":{"7":1,"46":1}}],["potentially",{"2":{"7":1,"11":1,"46":1}}],["painful",{"2":{"108":1}}],["pairs",{"2":{"63":1,"220":13}}],["pair",{"2":{"49":2,"52":1,"63":1}}],["pairwisefusion",{"2":{"48":3}}],["past",{"2":{"83":1}}],["passes",{"2":{"48":2,"111":1,"173":1}}],["passed",{"2":{"10":1,"19":1,"41":2,"48":6,"52":5,"54":6,"58":1,"59":1,"69":1,"82":1,"99":1,"136":1}}],["passing",{"2":{"8":1,"48":1,"55":1,"69":2,"120":2,"168":2}}],["pass",{"2":{"8":1,"22":2,"48":1,"49":1,"53":2,"55":1,"59":1,"60":1,"64":2,"66":1,"69":3,"78":1,"83":1,"97":1,"110":3,"120":1,"135":1,"168":1,"184":3}}],["pal2023efficient",{"2":{"74":1}}],["pal2023lux",{"2":{"74":1}}],["pal",{"2":{"74":2}}],["pattern",{"2":{"63":1,"106":1,"160":1,"248":1}}],["path",{"2":{"10":1,"48":1,"98":2,"126":2,"180":2,"189":2,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2,"251":1,"254":4,"260":1}}],["pad",{"2":{"49":6,"51":3}}],["pad=0",{"2":{"49":2,"51":3}}],["padding",{"2":{"28":1,"49":9,"51":12}}],["pads",{"2":{"28":1}}],["page",{"2":{"24":1,"37":1,"38":1,"68":1,"85":1,"106":1,"108":1,"153":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["parse",{"2":{"146":1}}],["parallelism",{"2":{"160":1}}],["parallel=true",{"2":{"160":2}}],["parallel",{"0":{"119":1},"1":{"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"39":1,"48":12,"54":1}}],["params=nothing",{"2":{"254":2}}],["params",{"2":{"38":14,"39":6,"55":3,"62":2,"126":3,"129":1,"254":4,"255":3,"256":6,"257":2,"258":1,"259":1}}],["parameterized",{"2":{"250":1}}],["parameterization",{"2":{"74":1,"247":1}}],["parameter",{"0":{"38":1,"137":1},"2":{"28":3,"38":2,"41":2,"49":4,"54":1,"55":2,"63":1,"69":6,"77":1,"82":1,"83":4,"110":1,"120":1,"128":2,"135":5,"136":2,"137":5,"140":1,"202":2,"217":1,"247":1,"249":2}}],["parameterlength",{"2":{"7":1,"9":1,"135":4,"136":1,"226":1,"249":3}}],["parameters",{"0":{"9":1,"41":1,"125":1,"128":1,"148":1},"1":{"126":1,"127":1,"128":1,"129":1},"2":{"3":1,"7":5,"8":1,"9":2,"28":1,"35":1,"38":13,"39":7,"40":1,"41":23,"44":4,"46":1,"48":24,"49":2,"52":9,"53":4,"54":2,"55":31,"62":9,"63":1,"66":2,"67":4,"68":5,"69":17,"77":5,"78":7,"81":1,"83":2,"90":2,"100":1,"107":1,"109":6,"110":7,"115":5,"120":1,"125":2,"128":3,"133":1,"135":5,"136":3,"137":7,"140":6,"144":1,"148":1,"175":1,"179":6,"184":2,"187":2,"188":1,"193":12,"195":2,"202":5,"214":1,"217":2,"225":1,"228":4,"236":1,"242":3,"244":1,"245":3,"249":79,"250":2,"256":6}}],["party",{"0":{"210":1}}],["parts",{"2":{"202":1}}],["particle",{"2":{"255":2,"256":1}}],["particles",{"2":{"254":1}}],["participate",{"0":{"113":1}}],["particular",{"0":{"126":1},"2":{"108":1,"126":1}}],["partition",{"2":{"36":1}}],["part",{"0":{"128":1,"129":1},"2":{"3":1,"64":1,"85":1,"145":1,"198":1,"209":1,"214":1}}],["packages",{"0":{"42":1,"79":1,"199":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"3":3,"22":4,"23":2,"46":1,"62":2,"70":1,"77":1,"78":1,"79":3,"80":1,"90":1,"93":1,"101":2,"104":3,"105":2,"116":3,"130":1,"145":2,"154":1,"159":1,"166":1,"167":3,"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1,"247":1}}],["package",{"0":{"182":1,"191":1,"212":1,"223":1,"231":1,"240":1,"253":1},"2":{"0":1,"3":1,"4":3,"25":1,"26":1,"35":1,"57":3,"72":1,"73":1,"76":1,"79":1,"93":3,"97":1,"101":1,"102":1,"105":1,"131":2,"164":1,"169":1,"172":1,"210":1,"230":1}}],["practitioners",{"2":{"202":1}}],["prngs",{"2":{"174":1}}],["prng",{"2":{"170":1,"174":2}}],["prs",{"2":{"144":1}}],["primarily",{"2":{"249":1}}],["primitives",{"0":{"34":1,"158":1},"2":{"158":1,"249":1}}],["principles",{"0":{"82":1}}],["printf",{"2":{"78":2,"106":1,"107":1,"179":2,"182":1,"187":2,"191":1,"195":1,"199":1,"202":1,"212":1,"217":1,"223":1,"228":2,"231":1,"236":1,"240":1,"245":1,"253":1,"257":1}}],["printouts",{"2":{"83":1}}],["printout",{"2":{"69":1}}],["print",{"2":{"67":1,"69":1}}],["printing",{"2":{"46":1,"54":1,"78":1,"83":1}}],["println",{"2":{"39":1,"109":2,"110":2,"135":2,"136":3,"137":2,"145":2,"146":2,"147":2,"148":2,"152":6,"173":2,"174":2,"176":2,"177":1,"178":1,"179":3,"180":2,"189":2,"197":2,"204":2,"221":2,"228":1,"229":2,"238":2,"246":2,"251":2,"260":2}}],["prints",{"2":{"4":2,"110":1}}],["printed",{"2":{"2":1,"8":1,"44":1,"69":1}}],["priority",{"2":{"104":2}}],["prior",{"2":{"49":1,"249":1}}],["pr",{"2":{"11":1,"209":1,"210":1}}],["precompile",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["precision",{"0":{"66":1},"2":{"16":2,"66":1,"155":3,"171":1}}],["press",{"2":{"161":1}}],["pressing",{"2":{"76":1}}],["presence",{"2":{"69":1}}],["presented",{"2":{"177":1}}],["presently",{"2":{"106":1}}],["present",{"2":{"44":1,"46":1,"49":2,"52":10,"53":2,"58":1,"64":1,"97":1,"109":1,"110":1,"131":2}}],["preserving",{"2":{"46":1,"56":1}}],["preservation",{"2":{"44":1}}],["preserved",{"2":{"46":1}}],["preserve",{"2":{"44":7,"46":1}}],["preds",{"2":{"249":2}}],["pred=ŷ",{"2":{"186":1}}],["predictive",{"2":{"250":1}}],["predictions",{"2":{"245":1,"249":2,"250":2}}],["prediction",{"0":{"250":1},"2":{"63":2,"249":1,"250":1}}],["predictor",{"2":{"237":1}}],["predict",{"2":{"183":1,"250":6}}],["predicted",{"2":{"63":3,"194":2,"216":2,"227":2,"250":2}}],["pred",{"2":{"63":8,"106":4,"186":5,"202":2,"203":3,"237":4,"245":2,"257":4}}],["pretty",{"2":{"54":1,"78":1,"83":1,"121":1,"145":1}}],["preprint",{"2":{"19":2,"55":1}}],["preprocessing",{"2":{"8":1}}],["prevents",{"2":{"97":1}}],["prevent",{"2":{"17":2,"32":1,"62":1,"63":1,"97":1,"135":1}}],["previously",{"2":{"7":1,"11":1,"83":1,"90":1,"99":1,"122":1}}],["previous",{"2":{"5":1,"7":1,"83":1,"146":1}}],["pre",{"2":{"7":1,"11":1,"49":1,"76":1,"77":1,"97":1}}],["prefer",{"2":{"158":1}}],["preferably",{"2":{"20":1}}],["preferred",{"2":{"5":1,"20":1}}],["preferencetools",{"2":{"161":2}}],["preference",{"2":{"1":2,"2":1,"67":1,"70":2,"144":1,"155":1,"157":1,"161":6,"164":2,"165":1,"166":3,"167":1}}],["preferences",{"0":{"1":1,"161":1},"1":{"162":1,"163":1,"164":1,"165":1,"166":1,"167":1},"2":{"67":1,"70":4,"97":1,"157":2,"161":4,"163":3,"166":2}}],["progress",{"2":{"247":3}}],["programming",{"2":{"170":1}}],["proto",{"2":{"195":4}}],["probabilistic",{"2":{"249":2}}],["probability",{"2":{"17":3,"50":3,"63":1}}],["prob",{"2":{"200":2,"202":9,"203":2,"214":4,"218":2,"255":1,"256":1,"257":1,"259":1}}],["problem",{"0":{"232":1},"2":{"109":1,"110":1,"145":1,"148":1,"173":1,"179":2,"202":1,"218":1,"220":1,"232":1,"242":1}}],["problematic",{"2":{"108":1,"109":1}}],["problems",{"0":{"109":1},"2":{"81":1,"106":1}}],["promotion",{"0":{"155":1},"2":{"155":1}}],["promotions",{"2":{"67":1,"155":1}}],["promote",{"2":{"16":1,"155":1}}],["project",{"2":{"57":1}}],["produce",{"2":{"64":1,"90":1}}],["products",{"2":{"22":2}}],["product",{"0":{"150":1,"151":1,"177":1,"178":1},"2":{"22":8,"144":2,"149":4,"150":1,"151":1,"175":5,"177":2,"178":1,"235":1,"237":1}}],["prod",{"2":{"28":1}}],["providing",{"2":{"25":1,"193":1}}],["provides",{"2":{"46":1,"59":1,"68":1,"77":1,"95":1,"133":1,"161":1,"168":1,"177":1,"198":1,"218":1}}],["provide",{"2":{"11":1,"28":1,"44":1,"62":2,"69":1,"79":1,"101":1,"149":1,"172":1}}],["provided",{"2":{"4":2,"5":1,"17":1,"37":1,"39":2,"52":3,"64":1,"120":1,"133":1,"172":1,"244":1,"250":1}}],["propagating",{"2":{"68":1}}],["propagated",{"2":{"48":1}}],["propagate",{"2":{"8":1}}],["proper",{"2":{"118":1,"170":1}}],["properly",{"2":{"40":1}}],["properties",{"2":{"23":1,"48":1}}],["proportion",{"2":{"28":3}}],["proceeding",{"2":{"135":1}}],["proceedings",{"2":{"19":1,"28":5,"63":2}}],["proceeds",{"2":{"52":8}}],["processor",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["processes",{"2":{"35":1,"36":1,"120":3,"160":1}}],["processing",{"2":{"17":1,"52":1}}],["process",{"2":{"4":1,"120":1,"160":1,"192":1,"213":1,"235":1}}],["ddot",{"2":{"254":2}}],["ddp",{"2":{"119":1}}],["dt^2",{"2":{"254":1}}],["dt2",{"2":{"254":3}}],["dt",{"2":{"254":21,"255":4,"256":2,"257":2,"259":2}}],["dset",{"2":{"224":4}}],["dnns",{"2":{"108":1}}],["dl",{"2":{"64":1,"256":1}}],["dynamic",{"2":{"57":1}}],["dynamics",{"2":{"28":1}}],["d2",{"2":{"41":5,"115":1,"116":3,"254":3}}],["d2=dense",{"2":{"41":1,"115":1,"118":1}}],["d3",{"2":{"41":9}}],["d3=chain",{"2":{"41":1}}],["d1",{"2":{"41":5,"115":1,"116":3}}],["d1=dense",{"2":{"41":1,"115":1,"118":1}}],["d1×",{"2":{"19":2,"55":2}}],["driver",{"2":{"180":3,"189":3,"204":3,"221":3,"229":3,"238":3,"246":3}}],["dr",{"2":{"108":1}}],["drawn",{"2":{"28":5,"250":1}}],["drop",{"2":{"63":1,"144":1}}],["dropped",{"2":{"17":2,"22":2}}],["dropout",{"0":{"17":1,"50":1},"2":{"17":14,"50":17,"83":1,"126":1,"129":1,"138":1,"142":1}}],["duration",{"2":{"249":2}}],["during",{"2":{"3":1,"55":4}}],["dudt",{"2":{"202":2,"203":1,"214":4,"218":2}}],["du",{"2":{"200":3}}],["due",{"2":{"146":1,"153":1,"220":1}}],["dummy",{"2":{"77":1}}],["dual",{"2":{"20":1}}],["dmitry",{"2":{"19":1,"55":1}}],["d",{"2":{"8":1,"49":4,"51":6,"54":2,"56":4,"69":11,"104":1,"126":2,"127":5,"128":6,"137":4,"183":4,"217":2,"254":2}}],["domain",{"2":{"235":1}}],["domainerror",{"2":{"40":1,"110":2}}],["doubt",{"2":{"135":1}}],["dot",{"2":{"109":1,"200":2,"203":2,"220":3,"236":4}}],["doi",{"2":{"74":2,"254":2}}],["doing",{"2":{"65":4,"107":1,"225":1,"249":1}}],["document",{"2":{"101":1}}],["documented",{"2":{"97":1}}],["documentations",{"2":{"154":1}}],["documentation",{"2":{"7":1,"8":1,"14":1,"56":1,"59":1,"70":1,"79":1,"84":1,"99":2,"135":1,"165":1,"166":1,"196":1,"210":1}}],["doctor",{"0":{"166":1},"2":{"70":4,"157":1,"166":3}}],["docstrings",{"2":{"135":1}}],["docstring",{"2":{"97":1}}],["docs",{"2":{"69":2,"87":1,"198":1,"236":4,"247":2}}],["downstream",{"2":{"83":1,"105":1,"210":1}}],["downright",{"2":{"83":1,"133":1}}],["down",{"0":{"110":1},"2":{"63":2,"67":1,"109":1,"110":1,"111":1,"136":1,"155":1,"171":2}}],["does",{"2":{"13":1,"19":2,"54":1,"67":1}}],["doesn",{"2":{"13":2,"39":1,"41":1,"52":1,"64":1,"69":3,"113":1,"135":2,"137":1,"138":1,"142":1,"145":1,"226":1}}],["do",{"2":{"6":1,"48":1,"54":2,"69":14,"76":1,"78":2,"92":1,"106":2,"107":1,"135":1,"136":1,"137":1,"138":1,"142":1,"144":1,"149":2,"153":1,"155":1,"160":3,"171":2,"173":1,"184":1,"185":1,"193":1,"202":2,"214":1,"225":1,"237":1,"242":1,"249":1,"250":1,"254":1,"256":1}}],["done",{"2":{"13":2,"39":1,"67":1,"77":1,"120":1,"131":1,"146":1,"167":1}}],["don",{"2":{"4":2,"16":1,"18":1,"29":1,"57":1,"69":2,"77":1,"83":1,"104":2,"105":1,"108":1,"111":1,"123":1,"124":1,"135":2,"137":1,"140":1,"144":1,"146":1,"171":2,"173":2,"177":1,"183":1,"184":1,"188":1,"192":1,"201":1,"213":1,"225":1}}],["digits=2",{"2":{"228":4}}],["digits",{"2":{"171":2}}],["direactly",{"2":{"97":1}}],["direct",{"2":{"97":1}}],["direction",{"2":{"55":2}}],["directly",{"2":{"3":1,"5":1,"6":1,"8":1,"17":1,"31":2,"38":1,"48":2,"54":1,"56":1,"62":1,"65":2,"94":1,"98":1,"120":1,"132":1,"144":1,"168":1,"211":1}}],["diverges",{"2":{"63":1}}],["divergence",{"2":{"63":2}}],["divisible",{"2":{"49":2}}],["divides",{"2":{"56":1}}],["divide",{"2":{"49":2}}],["dice",{"2":{"63":2}}],["dicecoeffloss",{"2":{"63":4}}],["diagonal",{"2":{"53":1,"249":1}}],["dilation",{"2":{"49":2,"51":3}}],["dilation=1",{"2":{"49":2,"51":3}}],["diffeqsol",{"2":{"214":3,"215":1,"220":9}}],["diffeqflux",{"2":{"211":1}}],["differ",{"2":{"175":1}}],["differern",{"2":{"146":1}}],["differences",{"0":{"123":1},"2":{"119":1,"140":1,"145":2}}],["difference",{"2":{"50":1,"90":1,"106":3,"136":1,"145":2,"146":2,"149":1,"219":1,"254":2}}],["differentiate",{"2":{"106":1,"152":1,"173":1}}],["differentiationinterface",{"2":{"144":1,"177":2}}],["differentiation",{"0":{"21":1,"101":1,"144":1,"162":1,"175":1},"1":{"22":1,"23":1,"24":1,"102":1,"103":1,"104":1,"105":1,"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1,"176":1,"177":1,"178":1},"2":{"24":1,"106":1,"137":1,"144":1,"147":1,"153":1,"162":2,"170":1,"177":1}}],["differential",{"2":{"74":1,"257":1}}],["differently",{"2":{"48":1}}],["different",{"0":{"45":1},"1":{"46":1},"2":{"18":1,"20":1,"39":1,"48":1,"63":1,"70":1,"99":1,"140":1,"146":1,"165":1,"174":1,"202":1}}],["diff",{"2":{"145":1,"147":1,"148":1,"175":1}}],["diffractor",{"2":{"102":1}}],["difficulty",{"2":{"28":2}}],["dimwhere",{"2":{"63":1}}],["dim",{"2":{"54":6,"107":1,"179":7}}],["dims=3",{"2":{"183":1}}],["dims=ndims",{"2":{"115":1}}],["dims=colon",{"2":{"55":1}}],["dims=2",{"2":{"52":1}}],["dims=",{"2":{"50":2}}],["dims=pad",{"2":{"49":1}}],["dims=1",{"2":{"19":1,"63":1,"235":5}}],["dims",{"2":{"17":2,"19":3,"28":11,"46":2,"49":1,"50":4,"52":31,"53":55,"54":7,"55":3,"63":2,"64":1,"87":1,"135":12,"169":4,"184":9,"185":7,"233":11,"236":2}}],["dimensionmismatch",{"2":{"109":2}}],["dimensionality",{"2":{"63":1}}],["dimensional",{"2":{"19":2,"53":3,"56":2,"177":1}}],["dimensions",{"2":{"19":2,"23":2,"28":11,"46":1,"49":3,"51":9,"52":2,"53":10,"54":5,"55":4,"56":8,"100":1,"179":1,"193":1}}],["dimension",{"0":{"109":1},"2":{"15":2,"16":1,"19":1,"46":1,"49":4,"50":2,"51":6,"52":8,"53":1,"54":9,"55":14,"64":1,"87":1,"108":1,"115":1,"118":1,"169":1}}],["disabling",{"0":{"167":1},"2":{"159":1}}],["disabled",{"2":{"8":1,"49":2,"53":3,"156":1}}],["disable",{"2":{"8":1,"40":1,"70":3,"97":2,"110":1,"144":1,"156":1,"162":1,"166":1,"167":2}}],["disallow",{"2":{"156":1}}],["disaster",{"2":{"155":1}}],["discrete",{"2":{"214":1,"217":1}}],["discouraged",{"2":{"145":2}}],["discourse",{"2":{"144":1,"145":1,"147":1}}],["discuss",{"2":{"198":1}}],["discussed",{"2":{"111":1}}],["discussions",{"2":{"84":1}}],["discussion",{"2":{"52":1,"147":1,"177":1}}],["disruptive",{"2":{"81":1}}],["dissimilar",{"2":{"63":1}}],["distinguish",{"2":{"140":2}}],["distinction",{"2":{"83":1}}],["distinct",{"2":{"52":1}}],["distance",{"2":{"63":1}}],["distributions",{"2":{"63":1,"249":2}}],["distribution",{"2":{"28":8,"29":12,"49":2,"52":6,"53":3,"63":5,"250":1}}],["distributeddatacontainer",{"2":{"36":1,"120":2,"121":1}}],["distributedoptimizer",{"2":{"35":2,"120":2,"121":1}}],["distributedutils",{"0":{"120":1},"2":{"4":2,"30":1,"31":2,"32":3,"33":2,"34":6,"35":1,"36":1,"119":1,"120":11,"121":5,"122":1,"123":3}}],["distributed",{"0":{"30":1,"119":1},"1":{"31":1,"32":1,"33":1,"34":1,"35":1,"36":1,"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"4":3,"28":2,"31":4,"32":7,"83":1,"120":2}}],["disjoint",{"2":{"41":1}}],["displays",{"2":{"248":1,"250":1}}],["displayed",{"2":{"131":1}}],["display",{"2":{"8":1,"83":1,"136":1}}],["dispatch=",{"2":{"225":1}}],["dispatch=nothing",{"2":{"69":1}}],["dispatch₋₋₋",{"2":{"220":3}}],["dispatching",{"0":{"112":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1}}],["dispatchdoctor",{"2":{"8":1,"70":1,"157":1}}],["dispatches",{"2":{"3":1,"8":1,"20":2,"64":1,"69":1,"133":1}}],["dispatch",{"0":{"113":1,"117":1,"166":1},"2":{"3":1,"57":1,"69":2,"70":4,"113":2,"117":1,"118":1,"138":1,"153":1,"157":1,"166":3}}],["date",{"2":{"210":2}}],["datapoints",{"2":{"241":1}}],["datasize",{"2":{"200":1,"255":1}}],["datasets",{"0":{"224":1},"2":{"183":1,"224":2,"228":1}}],["dataset",{"0":{"183":1,"241":1},"2":{"36":1,"160":2,"183":7,"192":3,"202":1,"213":3,"224":2,"241":2,"248":2}}],["datatype",{"2":{"8":1}}],["dataloaders",{"2":{"183":3,"187":2,"228":3}}],["dataloader",{"0":{"201":1},"2":{"5":8,"95":1,"107":4,"160":5,"183":5,"192":3,"194":2,"195":5,"201":5,"202":4,"212":1,"213":3,"216":2,"217":5,"224":2,"227":2,"228":9,"236":6}}],["data",{"0":{"2":1,"119":1,"160":1,"200":1,"235":1,"248":1},"1":{"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"0":1,"2":1,"3":3,"5":1,"36":2,"49":5,"51":9,"52":1,"54":1,"55":6,"56":2,"62":6,"63":1,"69":10,"78":10,"83":1,"95":1,"106":1,"107":2,"120":3,"135":1,"153":1,"155":1,"160":5,"179":1,"183":8,"192":6,"200":3,"201":3,"202":3,"203":2,"213":6,"227":2,"228":16,"234":5,"235":10,"236":113,"239":1,"241":4,"244":1,"245":5,"248":6,"249":1,"250":7,"255":3,"256":2,"257":1,"259":2}}],["danger",{"2":{"2":1,"4":2,"23":1,"32":1}}],["derivative",{"2":{"147":1}}],["derivatives",{"2":{"145":1,"146":1,"177":1,"232":1,"256":1}}],["de",{"2":{"145":2}}],["demonstration",{"2":{"202":1,"213":1,"230":1}}],["demonstrative",{"2":{"137":1}}],["demonstrate",{"2":{"140":1,"145":1,"179":1,"190":1}}],["deal",{"2":{"147":1,"214":1}}],["deadlocks",{"2":{"124":1}}],["deactivate",{"2":{"52":3}}],["decision",{"2":{"161":1}}],["decides",{"2":{"142":1}}],["declared",{"2":{"69":1}}],["decouples",{"2":{"55":1}}],["decrease",{"2":{"13":1}}],["debuglayer",{"2":{"40":3,"109":4,"110":4}}],["debug",{"2":{"40":5,"98":1,"108":4,"109":6,"110":10,"155":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["debugging",{"0":{"40":1,"108":1},"1":{"109":1,"110":1,"111":1},"2":{"40":3,"67":1,"108":2}}],["delta=0",{"2":{"63":1}}],["delta",{"2":{"63":2}}],["delving",{"2":{"28":2}}],["deleted",{"2":{"1":1}}],["descent",{"2":{"179":2}}],["describe",{"2":{"153":1}}],["describes",{"2":{"46":1,"108":1,"255":1,"256":1}}],["described",{"2":{"28":3,"48":2}}],["despite",{"2":{"62":1}}],["destructure",{"2":{"44":3,"83":2,"143":1}}],["desirable",{"2":{"167":1,"249":1}}],["desired",{"2":{"1":1,"49":1,"100":1}}],["design",{"0":{"82":1},"2":{"5":1,"81":1,"140":1}}],["density",{"2":{"249":1}}],["denselayerparameters",{"2":{"137":3}}],["dense",{"2":{"18":2,"38":2,"39":18,"41":4,"44":2,"46":3,"48":12,"53":2,"54":1,"55":12,"63":2,"69":10,"77":17,"78":6,"88":1,"99":1,"106":9,"107":3,"109":17,"110":14,"115":6,"116":2,"118":2,"126":9,"127":3,"129":3,"135":1,"137":2,"139":5,"145":2,"146":2,"147":4,"148":4,"152":4,"153":2,"155":1,"158":1,"166":1,"179":2,"184":2,"185":1,"193":9,"202":3,"215":5,"220":45,"226":4,"233":4,"242":4,"245":4,"249":4,"256":6}}],["denom",{"2":{"255":3,"256":3}}],["denominator",{"2":{"19":4,"55":4}}],["denotes",{"2":{"3":4}}],["detour",{"2":{"175":1}}],["detection",{"2":{"63":2}}],["detected",{"2":{"40":1,"110":2,"236":1}}],["deterministic",{"2":{"28":1,"83":1}}],["determined",{"2":{"77":1}}],["determines",{"2":{"46":1}}],["determine",{"2":{"3":1,"17":2,"19":2,"65":1,"123":1,"249":1}}],["details",{"0":{"141":1},"1":{"142":1},"2":{"14":1,"17":2,"19":4,"40":1,"41":1,"44":1,"46":2,"55":1,"56":1,"59":1,"68":1,"69":2,"70":1,"87":1,"90":1,"92":1,"97":1,"99":2,"135":1,"162":1,"165":1,"166":1,"172":1}}],["detailed",{"2":{"7":1,"38":1,"109":1}}],["depot",{"2":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["depots",{"2":{"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["depwarn",{"2":{"97":1}}],["deprecated",{"2":{"87":1,"236":4}}],["deprecation",{"2":{"37":1,"93":2,"145":2}}],["deprecations",{"2":{"2":1,"97":1}}],["depthwise",{"2":{"49":2}}],["depths",{"2":{"39":1}}],["dependent",{"0":{"115":1},"2":{"114":1}}],["dependencies",{"0":{"159":1},"2":{"57":1,"90":1,"167":1,"249":1}}],["dependency",{"2":{"6":1,"25":1,"57":1,"133":1,"146":1}}],["depend",{"2":{"6":1,"19":1,"133":1,"135":1}}],["depending",{"2":{"6":1,"8":1,"28":1}}],["defiing",{"0":{"256":1}}],["definition",{"0":{"232":1}}],["definitions",{"2":{"136":1}}],["definitely",{"2":{"152":1}}],["defining",{"0":{"78":1,"186":1},"2":{"0":1,"3":2,"11":1,"69":1,"78":1,"106":1,"113":1,"133":1,"135":1,"136":1,"137":1,"184":1,"185":2,"193":1,"225":1}}],["define",{"0":{"193":1,"195":1,"201":1,"214":1,"216":1,"227":1,"233":1,"234":1,"254":1},"2":{"7":1,"20":1,"78":1,"110":2,"113":2,"114":1,"117":1,"133":2,"135":3,"136":1,"137":2,"140":2,"153":1,"179":1,"184":3,"185":1,"186":1,"201":1,"202":1,"214":1,"233":1,"234":1,"249":3,"254":2,"256":2,"257":2}}],["defines",{"2":{"6":1,"7":2,"65":1,"254":1,"255":1}}],["defined",{"2":{"3":1,"8":1,"11":1}}],["defer",{"2":{"177":1}}],["defaults",{"0":{"99":1},"2":{"52":3,"68":1,"99":4,"256":1}}],["default",{"2":{"3":2,"8":1,"19":10,"26":5,"28":8,"29":24,"39":1,"44":1,"46":3,"52":1,"54":4,"55":1,"66":1,"67":1,"69":1,"73":4,"77":1,"78":1,"83":1,"87":1,"90":2,"97":1,"99":2,"106":1,"107":1,"129":1,"135":1,"137":5,"139":2,"140":2,"142":2,"155":1,"156":1,"163":1,"166":1,"167":1,"168":4,"169":1,"170":1,"171":1,"173":1,"179":1,"180":1,"184":1,"189":1,"197":1,"201":1,"202":1,"204":1,"215":1,"220":1,"221":1,"225":1,"229":1,"236":1,"238":1,"246":1,"248":1,"251":1,"256":1,"260":1}}],["dev=gpu",{"2":{"215":1}}],["developer",{"2":{"106":1}}],["developed",{"2":{"83":1,"210":1}}],["deviate",{"2":{"256":1}}],["deviation",{"2":{"28":3,"55":1}}],["devicememory",{"2":{"5":3,"77":18,"131":1,"153":2,"168":2,"172":1,"220":18}}],["deviceiterator",{"2":{"5":3,"95":2,"107":2,"201":1}}],["device",{"0":{"160":1},"2":{"2":25,"3":32,"4":11,"5":3,"20":1,"32":1,"72":5,"73":4,"77":2,"78":1,"93":1,"94":3,"98":2,"106":5,"107":5,"123":4,"131":8,"132":2,"140":3,"153":1,"160":5,"180":1,"187":2,"189":1,"199":2,"201":1,"204":1,"215":1,"217":2,"220":1,"221":1,"228":1,"231":2,"244":2,"245":1,"246":1}}],["devices",{"2":{"0":1,"32":4,"140":1,"229":1,"238":1}}],["dev",{"2":{"2":1,"3":1,"4":2,"5":4,"20":2,"72":4,"73":3,"77":5,"78":4,"187":3,"215":2,"217":3,"228":4,"244":3,"245":2}}],["deepcopy",{"2":{"62":1}}],["deep",{"2":{"2":1,"19":1,"25":1,"28":6,"74":1,"83":1,"155":1,"158":1,"168":1,"169":1}}],["axes",{"2":{"225":3}}],["axislegend",{"2":{"200":1,"203":1,"241":1,"245":1,"255":1,"256":1,"259":1}}],["axis",{"2":{"200":1,"203":1,"220":44,"237":1,"241":1,"245":1,"248":1,"250":1,"255":1,"256":1,"259":2}}],["ax",{"2":{"200":4,"203":6,"237":4,"241":4,"245":5,"248":3,"255":4,"256":6,"259":11}}],["a100",{"2":{"180":1,"189":1,"204":1,"221":1,"246":1}}],["ai",{"2":{"177":1}}],["aims",{"2":{"28":1}}],["a∈rd×d",{"2":{"149":1}}],["a=layer",{"2":{"140":1}}],["a×b×x",{"2":{"140":1}}],["aware",{"0":{"163":1},"2":{"123":2,"124":1,"163":4}}],["ah",{"2":{"110":1}}],["ahmadi",{"2":{"63":1}}],["ahmad",{"2":{"63":1}}],["author",{"2":{"74":2}}],["autojacvec=reversediffvjp",{"2":{"217":1}}],["autojacvec=zygotevjp",{"2":{"215":1,"217":1}}],["auto",{"2":{"142":1,"155":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["autoreactant",{"2":{"62":1}}],["autoreversediff",{"2":{"59":1,"62":2}}],["autoenzyme",{"2":{"59":1,"62":1,"107":2}}],["autofinitediff",{"2":{"59":1}}],["autoforwarddiff",{"2":{"22":1,"23":1,"59":1,"146":1,"151":1,"177":2}}],["autotracker",{"2":{"59":1,"62":1}}],["autozygote",{"2":{"22":1,"23":1,"59":1,"62":1,"77":2,"78":1,"150":1,"178":1,"179":1,"187":1,"195":2,"202":1,"217":1,"228":1,"236":1,"245":2,"258":1}}],["autodiff",{"2":{"17":2,"19":2,"81":1,"137":1,"144":1,"145":2,"175":2}}],["autoselection",{"2":{"2":1}}],["automatically",{"2":{"5":1,"7":2,"8":1,"17":3,"19":2,"52":2,"123":1,"142":1,"143":1,"144":1,"159":1,"183":1,"184":2,"185":1,"192":1,"213":1,"249":1}}],["automatic",{"0":{"21":1,"101":1,"131":1,"144":1,"162":1,"165":1,"175":1},"1":{"22":1,"23":1,"24":1,"102":1,"103":1,"104":1,"105":1,"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1,"176":1,"177":1,"178":1},"2":{"2":1,"3":1,"8":1,"24":1,"106":1,"131":2,"132":1,"137":1,"144":2,"153":1,"162":4,"164":1,"170":1,"177":1}}],["affected",{"2":{"149":1}}],["affects",{"2":{"49":1}}],["affine",{"2":{"55":1}}],["affine=false",{"2":{"55":5,"99":1}}],["affine=true",{"2":{"48":2,"55":20,"99":1,"109":1,"110":2}}],["afterwards",{"2":{"55":1}}],["after",{"2":{"17":2,"40":1,"48":2,"50":1,"51":3,"55":4,"69":1,"83":1,"108":1,"120":1,"123":1,"160":1,"179":14,"184":1}}],["abyss",{"2":{"170":1}}],["ability",{"2":{"133":1}}],["ab7d",{"2":{"116":3,"145":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["able",{"2":{"90":1,"181":1}}],["above",{"2":{"48":2,"64":1,"76":1,"148":1,"169":1,"172":1,"250":1}}],["about",{"2":{"40":1,"67":1,"83":1,"135":1,"171":1,"174":1,"184":1,"235":1,"250":1}}],["abstol",{"2":{"220":13}}],["abstol=1",{"2":{"215":1}}],["abstraction",{"2":{"93":1}}],["abstractions",{"2":{"20":1}}],["abstractexplicitcontainerlayer",{"2":{"90":2,"91":1}}],["abstractexplicitlayer",{"2":{"90":1}}],["abstractrecurrentcell",{"2":{"52":2}}],["abstractrng=utils",{"2":{"28":8,"29":24}}],["abstractrng",{"2":{"7":2,"8":2,"9":1,"10":1,"17":4,"28":3,"135":2,"140":2,"225":1,"241":1}}],["abstracttimeseriesdatabatchordering=batchlastindex",{"2":{"52":2}}],["abstractadtype",{"2":{"22":2,"23":1,"62":1}}],["abstractarrays",{"2":{"59":1}}],["abstractarray",{"2":{"13":2,"16":2,"19":1,"20":1,"23":1,"28":16,"29":60,"50":3,"53":6,"54":6,"55":1,"64":1,"65":1,"117":1,"140":2,"144":3,"148":1,"184":1,"185":1,"202":1,"203":1,"234":3}}],["abstractmatrix",{"2":{"18":2,"54":1,"135":1,"136":1,"214":1}}],["abstractvector",{"2":{"16":1,"18":1,"54":1,"249":1,"254":2}}],["abstractluxdistributedbacked",{"2":{"35":1}}],["abstractluxdistributedbackend",{"2":{"32":3,"33":2,"34":7,"36":1}}],["abstractluxwrapperlayer",{"2":{"7":4,"39":1,"90":1,"91":2,"115":1,"136":1,"214":1,"218":1}}],["abstractluxcontainerlayer",{"2":{"7":5,"90":1,"136":3,"184":2,"233":1}}],["abstractluxlayers",{"2":{"48":1}}],["abstractluxlayer",{"2":{"7":5,"8":2,"38":4,"39":3,"40":1,"46":3,"48":4,"55":1,"68":1,"90":1,"113":2,"116":3,"117":2,"135":2,"140":1,"214":3,"218":2,"225":2}}],["abstract",{"0":{"7":1},"2":{"3":2,"6":1,"7":3,"53":4,"202":1}}],["abstractdevice",{"2":{"2":1,"3":9,"4":4,"5":1}}],["abstractgpudevice",{"2":{"1":1,"132":1}}],["absolute",{"2":{"63":1,"155":1}}],["absolutely",{"2":{"62":1}}],["abs2",{"2":{"59":1,"63":1,"69":3,"145":2,"146":2,"147":2,"148":1,"234":3,"249":1}}],["abs",{"2":{"28":1,"54":1,"237":2,"254":1}}],["amazing",{"2":{"210":1}}],["am",{"2":{"147":1,"256":1}}],["ambiguous",{"2":{"65":1,"90":1}}],["amp",{"0":{"22":1,"154":1,"170":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"22":2,"23":1,"49":4,"52":1,"53":1,"54":1,"55":1,"62":1,"81":2}}],["amd",{"2":{"3":1,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["amdgpudevice",{"2":{"4":2,"123":1,"132":2,"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["amdgpu",{"2":{"2":2,"3":1,"26":2,"32":1,"72":2,"77":1,"78":1,"80":2,"104":1,"124":1,"130":2,"180":2,"189":2,"197":2,"204":2,"221":2,"229":2,"238":2,"246":2,"251":2,"260":2}}],["average",{"2":{"250":3}}],["averages",{"2":{"34":2,"35":1}}],["avik",{"2":{"74":2}}],["avg",{"2":{"34":2}}],["avoids",{"2":{"218":1}}],["avoiding",{"2":{"146":1}}],["avoid",{"2":{"17":1,"52":1,"83":1,"123":1,"218":1,"220":1,"249":2}}],["available",{"2":{"13":2,"16":1,"18":1,"30":1,"36":1,"76":1,"79":1,"101":1,"106":1,"131":1,"132":1,"143":1,"159":1,"180":1,"189":1,"204":1,"221":1,"229":2,"238":2,"246":1}}],["aka",{"2":{"16":1,"18":1}}],["academic",{"2":{"74":1}}],["achieved",{"2":{"49":1,"146":1}}],["achieve",{"2":{"28":1,"52":1}}],["acc",{"2":{"187":2,"195":8,"196":2,"217":4,"228":13}}],["account",{"2":{"69":1}}],["according",{"2":{"28":1}}],["accordingly",{"2":{"19":2,"55":2}}],["accidental",{"2":{"67":1}}],["accumulated",{"2":{"202":1}}],["accumulates",{"2":{"55":2}}],["accuracy",{"0":{"186":1},"2":{"13":1,"186":2,"187":102,"194":2,"195":4,"196":40,"216":2,"217":94,"219":18,"227":2,"228":212}}],["accelerate",{"2":{"167":1}}],["accelerating",{"2":{"19":1}}],["acceptance",{"2":{"249":1}}],["accept",{"2":{"137":1,"249":1}}],["accepts",{"2":{"56":1}}],["access",{"2":{"48":2,"69":1,"72":1,"73":1,"135":1,"136":1,"137":1}}],["accessors",{"2":{"41":1}}],["accessing",{"2":{"39":1,"44":1,"46":1}}],["acts",{"2":{"249":1}}],["action",{"2":{"70":1}}],["activity",{"2":{"59":2}}],["activates",{"2":{"166":1}}],["activate",{"2":{"59":1}}],["activated",{"2":{"52":4}}],["activation=tanh",{"2":{"52":1}}],["activation=identity",{"2":{"49":2,"53":4,"55":4}}],["activations",{"2":{"13":1,"16":1,"249":1}}],["activation",{"0":{"13":1,"15":1},"2":{"13":7,"15":12,"16":4,"18":3,"19":5,"49":6,"52":3,"53":10,"55":12,"69":1,"97":3,"99":3,"158":7,"166":1,"249":1,"256":3}}],["actively",{"2":{"88":1,"106":1}}],["active",{"2":{"44":2}}],["act=relu",{"2":{"69":2,"78":1}}],["actually",{"2":{"90":1,"133":1}}],["actual",{"2":{"40":1,"176":2,"245":1}}],["act",{"2":{"13":2,"19":2,"69":8,"78":5,"233":4}}],["across",{"2":{"0":1,"19":2,"34":2,"35":1,"36":1,"82":2,"120":3,"171":2,"250":1}}],["adjust",{"2":{"236":1}}],["adjoint",{"2":{"202":2}}],["adam",{"2":{"69":1,"77":15,"78":1,"107":1,"187":1,"195":1,"202":4,"217":1,"228":1,"236":1,"243":3,"245":1}}],["adapted",{"2":{"252":2}}],["adaptive=false",{"2":{"255":1,"256":1,"257":1,"259":1}}],["adaptivemeanpool",{"2":{"51":1}}],["adaptivemaxpool",{"2":{"51":1}}],["adaptive",{"2":{"51":3}}],["adaptivelppool",{"2":{"51":1,"100":1}}],["adaptor",{"2":{"46":4,"94":1,"193":1}}],["adapt",{"2":{"3":4,"44":6,"46":6}}],["ads",{"2":{"65":1}}],["adtype=autotracker",{"2":{"249":1}}],["adtypes",{"2":{"62":1,"144":1,"182":1,"191":1,"223":1,"231":1,"240":1,"245":1}}],["adtype",{"2":{"59":1,"258":1}}],["advances",{"2":{"17":1}}],["advancedvi",{"2":{"247":1}}],["advanced",{"0":{"208":1},"2":{"7":2,"106":1}}],["ad",{"0":{"23":1,"24":1},"2":{"16":1,"18":1,"22":6,"23":1,"24":1,"40":2,"62":3,"63":1,"65":2,"68":2,"77":2,"82":1,"83":3,"101":4,"102":2,"105":2,"144":4,"147":2,"154":1,"162":1,"170":1,"173":1,"175":2,"176":5,"177":2,"217":1,"230":1,"232":2,"234":1,"237":1,"245":1}}],["adding",{"2":{"124":1}}],["additional",{"0":{"79":1},"2":{"58":1,"59":1,"64":1,"79":1,"80":1,"139":1,"172":1,"177":1,"179":1}}],["additionally",{"2":{"7":1,"62":1,"63":1,"64":2,"101":1,"133":1,"140":1,"149":1,"153":1,"160":1,"188":1,"201":1,"234":1}}],["addition",{"2":{"15":1,"65":1}}],["address",{"2":{"81":1}}],["add",{"2":{"57":1,"65":3,"71":2,"72":5,"73":1,"76":2,"77":1,"79":1,"100":1,"104":1,"145":1,"161":3,"171":1,"172":1,"210":1}}],["added",{"2":{"15":1,"19":4,"49":2,"51":3,"54":1,"55":4,"63":2,"85":1,"100":1}}],["adds",{"2":{"14":1,"40":1,"171":1}}],["at=train",{"2":{"192":1,"213":1}}],["at=0",{"2":{"183":1}}],["atleast",{"2":{"19":1}}],["attempts",{"2":{"14":1,"18":1,"44":1}}],["at",{"2":{"4":2,"17":1,"34":2,"38":1,"49":2,"51":3,"52":1,"56":1,"63":1,"69":1,"77":1,"106":1,"109":4,"110":10,"124":1,"125":1,"130":1,"137":1,"166":2,"177":1,"184":1,"210":2,"236":4,"250":2,"254":2,"255":1}}],["april",{"2":{"74":1}}],["append",{"2":{"248":2}}],["appendix",{"0":{"180":1,"189":1,"197":1,"204":1,"221":1,"229":1,"238":1,"246":1,"251":1,"260":1}}],["appreciate",{"2":{"69":1}}],["approach",{"0":{"131":1},"2":{"106":1,"202":1}}],["approx",{"2":{"59":1}}],["approximation",{"2":{"54":1}}],["approximately",{"2":{"28":1}}],["appropriate",{"2":{"8":1,"28":1}}],["applications",{"2":{"83":1,"173":1}}],["applicable",{"2":{"2":1,"4":1}}],["applies",{"2":{"15":1,"19":1,"48":1,"55":6,"56":2,"120":1}}],["applied",{"2":{"8":1,"28":3,"48":1,"50":4,"55":4,"63":2,"65":1,"140":2}}],["applychain",{"2":{"220":3}}],["applying",{"2":{"17":2,"28":1,"48":1}}],["apply",{"0":{"13":1},"2":{"8":7,"48":1,"49":2,"50":2,"51":3,"62":5,"65":1,"77":3,"113":1,"115":1,"116":4,"117":3,"135":1,"136":1,"153":1,"166":1,"245":1,"249":1}}],["appleaccelerate",{"2":{"18":1}}],["apple",{"2":{"3":1,"80":1}}],["api",{"0":{"27":1,"62":1,"107":1,"185":1},"1":{"28":1,"29":1},"2":{"3":1,"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"34":3,"44":1,"48":9,"54":2,"62":2,"63":2,"77":3,"85":1,"87":2,"92":1,"106":2,"107":1,"125":1,"139":1,"179":1,"180":1,"181":1,"185":1,"189":1,"190":1,"198":1,"204":1,"214":1,"218":1,"220":1,"221":1,"229":1,"238":1,"244":1,"246":1,"256":2}}],["astroinformatics",{"2":{"252":1}}],["ask",{"2":{"171":3}}],["asymmetric",{"2":{"49":2,"51":3}}],["assert",{"2":{"174":1,"249":1,"254":5}}],["associated",{"2":{"81":1}}],["assume",{"2":{"106":1,"254":1}}],["assumed",{"2":{"59":1}}],["assumes",{"2":{"11":1,"48":1,"99":1}}],["assigned",{"2":{"69":1}}],["assign",{"2":{"32":1}}],["as",{"2":{"3":2,"4":2,"5":3,"7":4,"13":2,"15":2,"17":2,"19":3,"22":2,"28":5,"40":1,"41":1,"44":1,"46":1,"48":10,"49":1,"51":3,"52":3,"54":3,"55":1,"56":1,"58":2,"59":2,"60":4,"62":2,"63":8,"64":2,"65":1,"67":2,"69":10,"71":1,"74":1,"78":2,"83":2,"85":1,"97":2,"98":2,"103":1,"106":6,"107":1,"110":1,"114":1,"120":1,"121":1,"123":2,"136":3,"137":2,"145":1,"146":1,"147":2,"148":1,"152":1,"153":2,"155":2,"165":1,"169":2,"170":1,"171":1,"173":1,"174":2,"176":1,"185":1,"190":2,"196":1,"198":1,"202":1,"214":1,"220":1,"225":1,"247":1,"249":6,"250":1,"256":2}}],["agent",{"2":{"109":1,"220":3,"236":4}}],["aggressive",{"2":{"93":1}}],["aggregation",{"2":{"63":1}}],["aggregated",{"2":{"63":1}}],["agg=sum",{"2":{"63":2}}],["agg=mean",{"2":{"63":1}}],["agg",{"2":{"63":26}}],["agnostic",{"2":{"3":3,"34":3,"160":1}}],["against",{"2":{"104":1,"149":1}}],["again",{"2":{"3":1,"120":1,"146":1}}],["algebras",{"2":{"177":1}}],["algebraic",{"2":{"146":1}}],["algorithm",{"2":{"2":1,"149":1,"247":1}}],["already",{"2":{"70":1,"77":1,"81":1,"174":1,"244":1}}],["alias",{"2":{"64":1}}],["aliased",{"2":{"62":3}}],["aliases",{"2":{"15":1}}],["aligned",{"2":{"56":1}}],["align",{"2":{"56":2,"99":2,"100":1}}],["alternate",{"0":{"218":1}}],["alternatives",{"2":{"158":1}}],["alternatively",{"2":{"52":2,"56":1,"59":1,"76":1,"120":1,"155":1}}],["alternative",{"2":{"3":1}}],["alts",{"2":{"54":2}}],["almost",{"2":{"44":1}}],["always",{"2":{"22":2,"37":1,"38":1,"44":1,"63":1,"65":1,"99":1,"138":1}}],["al",{"2":{"17":2,"28":2,"63":2,"252":1}}],["alpha=0",{"2":{"241":1,"245":2,"255":2,"256":4,"259":7}}],["alphadropout",{"2":{"50":4}}],["alpha",{"2":{"17":4,"249":2}}],["along",{"2":{"15":1,"19":1,"28":1,"48":1,"49":1,"50":2,"52":2,"55":1,"63":1,"64":1,"83":1,"115":1}}],["also",{"2":{"3":1,"7":1,"15":2,"28":2,"32":2,"38":1,"48":2,"50":5,"54":1,"55":7,"56":2,"64":1,"65":2,"67":1,"69":1,"76":1,"83":2,"85":1,"93":1,"135":2,"168":1,"171":1,"185":1,"187":1,"214":1,"217":2,"249":1,"250":1}}],["allocate",{"2":{"153":1}}],["allow",{"2":{"41":1,"59":1,"65":1,"93":1,"97":1,"155":1}}],["allowscalar",{"2":{"156":1,"212":1,"223":1,"231":1}}],["allows",{"2":{"6":1,"7":1,"8":1,"22":2,"34":2,"48":1,"52":1,"54":1,"65":1,"69":1,"84":1,"100":1,"136":1,"137":1,"217":1}}],["allreduce",{"2":{"34":3,"35":1,"122":1}}],["all",{"2":{"3":1,"6":1,"7":1,"8":1,"10":3,"28":1,"34":5,"37":1,"38":3,"39":2,"46":2,"48":3,"49":2,"51":6,"53":1,"54":2,"56":2,"58":2,"59":1,"62":2,"65":1,"66":1,"68":1,"77":1,"79":2,"82":3,"83":3,"93":1,"95":1,"106":1,"120":3,"123":1,"128":2,"135":1,"160":1,"166":1,"167":1,"169":1,"171":3,"172":1,"173":1,"201":1,"233":1,"248":1,"249":2}}],["artifact",{"2":{"180":1,"189":1,"204":1,"221":1,"229":1,"238":1,"246":1}}],["artificially",{"2":{"110":1}}],["artificial",{"2":{"28":2,"248":2}}],["arranged",{"2":{"171":1,"248":1}}],["arrayandtime",{"2":{"117":8,"118":1}}],["array",{"2":{"8":1,"13":5,"17":4,"19":5,"20":1,"23":2,"26":3,"28":9,"46":7,"49":5,"51":6,"52":2,"53":4,"54":6,"55":16,"63":1,"65":2,"69":1,"106":1,"107":2,"117":3,"146":1,"147":1,"169":1,"171":8,"172":1,"173":6,"194":1,"200":1,"202":1,"214":3,"215":1,"220":9,"248":4,"249":1,"255":1,"256":1,"257":1,"259":1}}],["arrays",{"0":{"156":1,"171":1,"172":1},"1":{"172":1},"2":{"3":1,"11":1,"16":2,"18":3,"20":7,"48":1,"53":1,"54":1,"56":5,"65":1,"66":1,"106":1,"135":1,"140":2,"153":2,"171":4,"173":1}}],["architecture",{"2":{"135":2,"140":1}}],["architectures",{"2":{"83":1}}],["architectural",{"2":{"135":1}}],["args",{"2":{"58":2,"59":4,"65":2,"67":6,"98":1}}],["argumenterror",{"2":{"236":1}}],["argument",{"2":{"8":2,"29":1,"48":4,"52":1,"54":1,"69":1,"90":1,"99":1,"100":1,"127":1}}],["arguments",{"2":{"2":2,"4":2,"8":1,"13":2,"15":1,"16":1,"17":2,"18":1,"19":4,"22":2,"23":1,"28":3,"35":1,"38":1,"40":3,"41":1,"44":2,"46":2,"48":11,"49":4,"50":5,"51":9,"52":7,"53":8,"54":7,"55":9,"56":2,"58":3,"59":5,"62":3,"68":1,"69":2,"87":1,"135":1,"169":1,"220":3}}],["arbitrary",{"2":{"56":1,"217":1}}],["arxiv",{"2":{"19":4,"28":1,"54":1,"55":2}}],["around",{"2":{"14":1,"22":2,"44":1,"49":2,"51":3,"83":1,"136":1,"148":1}}],["aren",{"2":{"209":1}}],["are",{"2":{"2":1,"3":2,"6":1,"7":2,"8":1,"11":1,"13":1,"18":1,"19":3,"20":1,"23":2,"28":4,"30":1,"32":2,"37":2,"38":6,"40":4,"41":1,"46":2,"53":3,"54":2,"55":5,"56":2,"59":2,"62":6,"63":6,"64":2,"65":1,"67":2,"69":2,"70":1,"76":1,"78":1,"79":1,"82":2,"83":2,"90":1,"98":1,"100":1,"103":3,"104":5,"105":2,"106":2,"107":1,"108":1,"110":1,"116":1,"120":1,"123":1,"124":1,"126":1,"130":1,"135":3,"136":2,"137":2,"138":1,"144":2,"145":2,"146":1,"149":1,"152":3,"153":1,"155":2,"158":1,"159":2,"160":1,"163":2,"164":1,"166":1,"167":4,"171":1,"173":1,"175":1,"184":2,"202":1,"209":2,"210":2,"214":1,"225":1,"249":1,"250":1,"255":1,"256":1}}],["animation",{"2":{"250":1}}],["analytical",{"2":{"235":4,"237":1}}],["anticlockwise",{"2":{"181":1,"183":4}}],["anonymous",{"2":{"97":1}}],["another",{"2":{"53":1,"136":1}}],["answers",{"2":{"84":1}}],["anything",{"2":{"135":1}}],["anywhere",{"2":{"90":1}}],["any",{"2":{"8":2,"11":1,"16":1,"18":1,"37":1,"40":1,"46":1,"48":1,"52":3,"54":1,"59":1,"62":2,"63":3,"65":2,"69":2,"82":1,"93":2,"105":1,"106":3,"107":1,"109":1,"111":1,"116":7,"123":1,"133":2,"135":2,"137":1,"140":1,"144":1,"153":1,"167":1,"172":1,"173":2,"174":1,"175":1,"188":1,"210":1,"217":1,"220":15,"226":1,"244":1}}],["an",{"2":{"2":5,"3":2,"4":1,"5":1,"8":3,"13":1,"17":4,"19":3,"22":2,"28":19,"29":36,"31":2,"34":1,"35":1,"40":1,"48":10,"49":3,"50":3,"51":3,"52":2,"53":8,"54":2,"55":7,"60":3,"62":2,"63":2,"65":1,"67":1,"69":1,"77":1,"83":1,"90":1,"97":1,"99":2,"100":1,"101":1,"103":1,"105":1,"106":1,"107":1,"108":1,"110":1,"111":2,"114":1,"124":1,"131":1,"144":1,"145":2,"146":1,"148":1,"155":1,"159":1,"161":2,"171":3,"174":1,"190":1,"202":1,"209":1,"210":3,"248":1,"249":1,"250":1}}],["andrea",{"2":{"19":1,"55":1}}],["and",{"0":{"42":1,"160":1,"186":1,"215":1,"222":1,"226":1},"1":{"43":1,"44":1,"45":1,"46":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"1":1,"2":10,"3":3,"4":7,"5":4,"7":7,"8":7,"13":1,"14":1,"15":1,"16":4,"17":2,"18":3,"19":20,"23":1,"28":13,"32":4,"34":2,"36":2,"37":2,"38":4,"39":7,"40":5,"41":3,"44":5,"46":2,"48":10,"49":9,"51":3,"52":31,"53":8,"54":3,"55":22,"56":8,"57":2,"58":2,"60":2,"62":8,"63":19,"64":2,"65":6,"66":1,"67":4,"68":1,"69":5,"70":1,"76":1,"77":4,"78":5,"81":3,"82":2,"83":6,"84":2,"90":5,"95":2,"97":5,"98":3,"99":3,"100":4,"103":4,"104":2,"106":6,"107":2,"109":3,"110":6,"111":3,"118":1,"120":6,"121":2,"123":1,"124":1,"126":1,"130":1,"131":3,"132":1,"133":2,"135":9,"136":8,"137":3,"138":1,"140":10,"142":1,"144":2,"145":5,"147":1,"148":1,"149":2,"150":2,"152":1,"153":5,"155":3,"158":2,"159":1,"160":5,"164":2,"166":2,"167":4,"169":2,"171":7,"172":2,"173":1,"174":3,"175":3,"176":1,"178":1,"179":3,"181":2,"183":3,"184":10,"185":1,"188":3,"192":1,"193":1,"196":1,"202":3,"209":1,"210":3,"213":1,"214":2,"217":2,"220":1,"230":1,"233":1,"235":2,"241":1,"244":2,"245":1,"248":1,"249":12,"250":3,"254":3,"255":2,"256":6}}],["a",{"0":{"45":1,"126":1,"129":1,"181":1,"184":1,"222":1,"225":1,"230":1,"239":1,"252":1,"256":1},"1":{"46":1,"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"238":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"0":1,"1":2,"2":5,"3":5,"4":5,"5":3,"6":1,"7":10,"8":12,"10":1,"11":4,"14":1,"15":3,"16":2,"17":4,"18":2,"19":2,"22":2,"23":4,"25":1,"28":20,"29":13,"32":2,"34":3,"35":1,"37":1,"38":4,"39":1,"40":3,"41":4,"44":9,"46":9,"48":27,"49":21,"51":18,"52":43,"53":8,"54":10,"55":20,"56":6,"57":2,"58":1,"59":4,"60":5,"62":5,"63":4,"64":5,"65":5,"67":3,"68":4,"69":21,"72":1,"73":1,"76":1,"77":2,"78":3,"81":2,"82":1,"83":4,"84":1,"85":2,"90":9,"91":2,"92":2,"95":3,"106":4,"107":3,"108":1,"109":3,"110":5,"113":2,"114":1,"117":3,"118":1,"119":2,"120":1,"123":1,"124":1,"125":1,"126":1,"128":2,"130":1,"131":7,"133":3,"135":8,"136":8,"137":2,"140":17,"142":1,"143":1,"144":6,"145":7,"146":2,"147":3,"148":1,"149":10,"152":1,"153":4,"155":5,"160":1,"163":1,"164":1,"168":2,"169":2,"170":3,"171":12,"172":1,"174":3,"175":12,"176":1,"177":2,"179":4,"181":1,"183":2,"184":3,"185":1,"190":2,"196":1,"198":3,"201":1,"202":6,"209":1,"214":3,"217":2,"218":1,"219":1,"226":1,"230":2,"233":3,"234":1,"235":1,"239":2,"242":1,"245":2,"247":1,"248":3,"249":7,"250":5,"254":9,"256":3,"257":1,"258":1}}]],"serializationVersion":2}';export{e as default}; diff --git a/dev/assets/chunks/VPLocalSearchBox.BA0mQ21I.js b/dev/assets/chunks/VPLocalSearchBox.omfuhOSb.js similarity index 99% rename from dev/assets/chunks/VPLocalSearchBox.BA0mQ21I.js rename to dev/assets/chunks/VPLocalSearchBox.omfuhOSb.js index 4932e87152..942bf3fdc5 100644 --- a/dev/assets/chunks/VPLocalSearchBox.BA0mQ21I.js +++ b/dev/assets/chunks/VPLocalSearchBox.omfuhOSb.js @@ -1,4 +1,4 @@ -var Ft=Object.defineProperty;var Ot=(a,e,t)=>e in a?Ft(a,e,{enumerable:!0,configurable:!0,writable:!0,value:t}):a[e]=t;var Me=(a,e,t)=>Ot(a,typeof e!="symbol"?e+"":e,t);import{V as Rt,p as ie,h as me,aj as et,ak as Ct,al as Mt,q as $e,am as At,d as Lt,D as xe,an as tt,ao as Dt,ap as zt,s as Pt,aq as jt,v as Ae,P as he,O as Se,ar as Vt,as as $t,W as Bt,R as Wt,$ as Kt,o as H,b as Jt,j as S,a0 as Ut,k as L,at as qt,au as Gt,av as Ht,c as Z,n as st,e as _e,C as nt,F as it,a as fe,t as pe,aw as Qt,ax as rt,ay as Yt,a9 as Zt,af as Xt,az as es,_ as ts}from"./framework.CCjWn1F9.js";import{u as ss,d as ns}from"./theme.C74glUlb.js";const is={root:()=>Rt(()=>import("./@localSearchIndexroot.Bf_mqVD5.js"),[])};/*! +var Ft=Object.defineProperty;var Ot=(a,e,t)=>e in a?Ft(a,e,{enumerable:!0,configurable:!0,writable:!0,value:t}):a[e]=t;var Me=(a,e,t)=>Ot(a,typeof e!="symbol"?e+"":e,t);import{V as Rt,p as ie,h as me,aj as et,ak as Ct,al as Mt,q as $e,am as At,d as Lt,D as xe,an as tt,ao as Dt,ap as zt,s as Pt,aq as jt,v as Ae,P as he,O as Se,ar as Vt,as as $t,W as Bt,R as Wt,$ as Kt,o as H,b as Jt,j as S,a0 as Ut,k as L,at as qt,au as Gt,av as Ht,c as Z,n as st,e as _e,C as nt,F as it,a as fe,t as pe,aw as Qt,ax as rt,ay as Yt,a9 as Zt,af as Xt,az as es,_ as ts}from"./framework.CCjWn1F9.js";import{u as ss,d as ns}from"./theme.DgYiELuP.js";const is={root:()=>Rt(()=>import("./@localSearchIndexroot.CZmaQmit.js"),[])};/*! * tabbable 6.2.0 * @license MIT, https://github.com/focus-trap/tabbable/blob/master/LICENSE */var mt=["input:not([inert])","select:not([inert])","textarea:not([inert])","a[href]:not([inert])","button:not([inert])","[tabindex]:not(slot):not([inert])","audio[controls]:not([inert])","video[controls]:not([inert])",'[contenteditable]:not([contenteditable="false"]):not([inert])',"details>summary:first-of-type:not([inert])","details:not([inert])"],Ne=mt.join(","),gt=typeof Element>"u",ae=gt?function(){}:Element.prototype.matches||Element.prototype.msMatchesSelector||Element.prototype.webkitMatchesSelector,Fe=!gt&&Element.prototype.getRootNode?function(a){var e;return a==null||(e=a.getRootNode)===null||e===void 0?void 0:e.call(a)}:function(a){return a==null?void 0:a.ownerDocument},Oe=function a(e,t){var s;t===void 0&&(t=!0);var n=e==null||(s=e.getAttribute)===null||s===void 0?void 0:s.call(e,"inert"),r=n===""||n==="true",i=r||t&&e&&a(e.parentNode);return i},rs=function(e){var t,s=e==null||(t=e.getAttribute)===null||t===void 0?void 0:t.call(e,"contenteditable");return s===""||s==="true"},bt=function(e,t,s){if(Oe(e))return[];var n=Array.prototype.slice.apply(e.querySelectorAll(Ne));return t&&ae.call(e,Ne)&&n.unshift(e),n=n.filter(s),n},yt=function a(e,t,s){for(var n=[],r=Array.from(e);r.length;){var i=r.shift();if(!Oe(i,!1))if(i.tagName==="SLOT"){var o=i.assignedElements(),l=o.length?o:i.children,c=a(l,!0,s);s.flatten?n.push.apply(n,c):n.push({scopeParent:i,candidates:c})}else{var h=ae.call(i,Ne);h&&s.filter(i)&&(t||!e.includes(i))&&n.push(i);var m=i.shadowRoot||typeof s.getShadowRoot=="function"&&s.getShadowRoot(i),f=!Oe(m,!1)&&(!s.shadowRootFilter||s.shadowRootFilter(i));if(m&&f){var b=a(m===!0?i.children:m.children,!0,s);s.flatten?n.push.apply(n,b):n.push({scopeParent:i,candidates:b})}else r.unshift.apply(r,i.children)}}return n},wt=function(e){return!isNaN(parseInt(e.getAttribute("tabindex"),10))},re=function(e){if(!e)throw new Error("No node provided");return e.tabIndex<0&&(/^(AUDIO|VIDEO|DETAILS)$/.test(e.tagName)||rs(e))&&!wt(e)?0:e.tabIndex},as=function(e,t){var s=re(e);return s<0&&t&&!wt(e)?0:s},os=function(e,t){return e.tabIndex===t.tabIndex?e.documentOrder-t.documentOrder:e.tabIndex-t.tabIndex},xt=function(e){return e.tagName==="INPUT"},ls=function(e){return xt(e)&&e.type==="hidden"},cs=function(e){var t=e.tagName==="DETAILS"&&Array.prototype.slice.apply(e.children).some(function(s){return s.tagName==="SUMMARY"});return t},us=function(e,t){for(var s=0;ssummary:first-of-type"),i=r?e.parentElement:e;if(ae.call(i,"details:not([open]) *"))return!0;if(!s||s==="full"||s==="legacy-full"){if(typeof n=="function"){for(var o=e;e;){var l=e.parentElement,c=Fe(e);if(l&&!l.shadowRoot&&n(l)===!0)return at(e);e.assignedSlot?e=e.assignedSlot:!l&&c!==e.ownerDocument?e=c.host:e=l}e=o}if(ps(e))return!e.getClientRects().length;if(s!=="legacy-full")return!0}else if(s==="non-zero-area")return at(e);return!1},ms=function(e){if(/^(INPUT|BUTTON|SELECT|TEXTAREA)$/.test(e.tagName))for(var t=e.parentElement;t;){if(t.tagName==="FIELDSET"&&t.disabled){for(var s=0;s=0)},bs=function a(e){var t=[],s=[];return e.forEach(function(n,r){var i=!!n.scopeParent,o=i?n.scopeParent:n,l=as(o,i),c=i?a(n.candidates):o;l===0?i?t.push.apply(t,c):t.push(o):s.push({documentOrder:r,tabIndex:l,item:n,isScope:i,content:c})}),s.sort(os).reduce(function(n,r){return r.isScope?n.push.apply(n,r.content):n.push(r.content),n},[]).concat(t)},ys=function(e,t){t=t||{};var s;return t.getShadowRoot?s=yt([e],t.includeContainer,{filter:Be.bind(null,t),flatten:!1,getShadowRoot:t.getShadowRoot,shadowRootFilter:gs}):s=bt(e,t.includeContainer,Be.bind(null,t)),bs(s)},ws=function(e,t){t=t||{};var s;return t.getShadowRoot?s=yt([e],t.includeContainer,{filter:Re.bind(null,t),flatten:!0,getShadowRoot:t.getShadowRoot}):s=bt(e,t.includeContainer,Re.bind(null,t)),s},oe=function(e,t){if(t=t||{},!e)throw new Error("No node provided");return ae.call(e,Ne)===!1?!1:Be(t,e)},xs=mt.concat("iframe").join(","),Le=function(e,t){if(t=t||{},!e)throw new Error("No node provided");return ae.call(e,xs)===!1?!1:Re(t,e)};/*! diff --git a/dev/assets/chunks/theme.C74glUlb.js b/dev/assets/chunks/theme.DgYiELuP.js similarity index 99% rename from dev/assets/chunks/theme.C74glUlb.js rename to dev/assets/chunks/theme.DgYiELuP.js index ac5a80b3bd..f79551c752 100644 --- a/dev/assets/chunks/theme.C74glUlb.js +++ b/dev/assets/chunks/theme.DgYiELuP.js @@ -1,2 +1,2 @@ -const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=["assets/chunks/VPLocalSearchBox.BA0mQ21I.js","assets/chunks/framework.CCjWn1F9.js"])))=>i.map(i=>d[i]); -import{d as b,o as a,c as l,r as u,a as F,t as w,n as I,b as $,w as f,e as m,T as pe,_ as k,u as Le,i as Ke,f as qe,g as ve,h as y,j as p,k as i,l as G,m as ie,p as T,q as D,s as Z,v as R,x as fe,y as me,z as We,A as Je,B as K,F as M,C as B,D as Te,E as x,G as g,H as O,I as we,J as ee,K as j,L as W,M as Ye,N as Ne,O as le,P as Ie,Q as Me,R as te,S as Qe,U as Xe,V as Ze,W as Ce,X as he,Y as xe,Z as et,$ as tt,a0 as nt,a1 as Ae,a2 as st,a3 as ot,a4 as at,a5 as ye}from"./framework.CCjWn1F9.js";const rt=b({__name:"VPBadge",props:{text:{},type:{default:"tip"}},setup(o){return(e,t)=>(a(),l("span",{class:I(["VPBadge",e.type])},[u(e.$slots,"default",{},()=>[F(w(e.text),1)])],2))}}),it={key:0,class:"VPBackdrop"},lt=b({__name:"VPBackdrop",props:{show:{type:Boolean}},setup(o){return(e,t)=>(a(),$(pe,{name:"fade"},{default:f(()=>[e.show?(a(),l("div",it)):m("",!0)]),_:1}))}}),ct=k(lt,[["__scopeId","data-v-b06cdb19"]]),L=Le;function ut(o,e){let t,s=!1;return()=>{t&&clearTimeout(t),s?t=setTimeout(o,e):(o(),(s=!0)&&setTimeout(()=>s=!1,e))}}function ce(o){return/^\//.test(o)?o:`/${o}`}function _e(o){const{pathname:e,search:t,hash:s,protocol:n}=new URL(o,"http://a.com");if(Ke(o)||o.startsWith("#")||!n.startsWith("http")||!qe(e))return o;const{site:r}=L(),c=e.endsWith("/")||e.endsWith(".html")?o:o.replace(/(?:(^\.+)\/)?.*$/,`$1${e.replace(/(\.md)?$/,r.value.cleanUrls?"":".html")}${t}${s}`);return ve(c)}function Y({correspondingLink:o=!1}={}){const{site:e,localeIndex:t,page:s,theme:n,hash:r}=L(),c=y(()=>{var d,h;return{label:(d=e.value.locales[t.value])==null?void 0:d.label,link:((h=e.value.locales[t.value])==null?void 0:h.link)||(t.value==="root"?"/":`/${t.value}/`)}});return{localeLinks:y(()=>Object.entries(e.value.locales).flatMap(([d,h])=>c.value.label===h.label?[]:{text:h.label,link:dt(h.link||(d==="root"?"/":`/${d}/`),n.value.i18nRouting!==!1&&o,s.value.relativePath.slice(c.value.link.length-1),!e.value.cleanUrls)+r.value})),currentLang:c}}function dt(o,e,t,s){return e?o.replace(/\/$/,"")+ce(t.replace(/(^|\/)index\.md$/,"$1").replace(/\.md$/,s?".html":"")):o}const pt={class:"NotFound"},vt={class:"code"},ft={class:"title"},mt={class:"quote"},ht={class:"action"},_t=["href","aria-label"],bt=b({__name:"NotFound",setup(o){const{theme:e}=L(),{currentLang:t}=Y();return(s,n)=>{var r,c,v,d,h;return a(),l("div",pt,[p("p",vt,w(((r=i(e).notFound)==null?void 0:r.code)??"404"),1),p("h1",ft,w(((c=i(e).notFound)==null?void 0:c.title)??"PAGE NOT FOUND"),1),n[0]||(n[0]=p("div",{class:"divider"},null,-1)),p("blockquote",mt,w(((v=i(e).notFound)==null?void 0:v.quote)??"But if you don't change your direction, and if you keep looking, you may end up where you are heading."),1),p("div",ht,[p("a",{class:"link",href:i(ve)(i(t).link),"aria-label":((d=i(e).notFound)==null?void 0:d.linkLabel)??"go to home"},w(((h=i(e).notFound)==null?void 0:h.linkText)??"Take me home"),9,_t)])])}}}),kt=k(bt,[["__scopeId","data-v-951cab6c"]]);function Be(o,e){if(Array.isArray(o))return Q(o);if(o==null)return[];e=ce(e);const t=Object.keys(o).sort((n,r)=>r.split("/").length-n.split("/").length).find(n=>e.startsWith(ce(n))),s=t?o[t]:[];return Array.isArray(s)?Q(s):Q(s.items,s.base)}function gt(o){const e=[];let t=0;for(const s in o){const n=o[s];if(n.items){t=e.push(n);continue}e[t]||e.push({items:[]}),e[t].items.push(n)}return e}function $t(o){const e=[];function t(s){for(const n of s)n.text&&n.link&&e.push({text:n.text,link:n.link,docFooterText:n.docFooterText}),n.items&&t(n.items)}return t(o),e}function ue(o,e){return Array.isArray(e)?e.some(t=>ue(o,t)):G(o,e.link)?!0:e.items?ue(o,e.items):!1}function Q(o,e){return[...o].map(t=>{const s={...t},n=s.base||e;return n&&s.link&&(s.link=n+s.link),s.items&&(s.items=Q(s.items,n)),s})}function U(){const{frontmatter:o,page:e,theme:t}=L(),s=ie("(min-width: 960px)"),n=T(!1),r=y(()=>{const A=t.value.sidebar,N=e.value.relativePath;return A?Be(A,N):[]}),c=T(r.value);D(r,(A,N)=>{JSON.stringify(A)!==JSON.stringify(N)&&(c.value=r.value)});const v=y(()=>o.value.sidebar!==!1&&c.value.length>0&&o.value.layout!=="home"),d=y(()=>h?o.value.aside==null?t.value.aside==="left":o.value.aside==="left":!1),h=y(()=>o.value.layout==="home"?!1:o.value.aside!=null?!!o.value.aside:t.value.aside!==!1),P=y(()=>v.value&&s.value),_=y(()=>v.value?gt(c.value):[]);function V(){n.value=!0}function S(){n.value=!1}function C(){n.value?S():V()}return{isOpen:n,sidebar:c,sidebarGroups:_,hasSidebar:v,hasAside:h,leftAside:d,isSidebarEnabled:P,open:V,close:S,toggle:C}}function yt(o,e){let t;Z(()=>{t=o.value?document.activeElement:void 0}),R(()=>{window.addEventListener("keyup",s)}),fe(()=>{window.removeEventListener("keyup",s)});function s(n){n.key==="Escape"&&o.value&&(e(),t==null||t.focus())}}function Pt(o){const{page:e,hash:t}=L(),s=T(!1),n=y(()=>o.value.collapsed!=null),r=y(()=>!!o.value.link),c=T(!1),v=()=>{c.value=G(e.value.relativePath,o.value.link)};D([e,o,t],v),R(v);const d=y(()=>c.value?!0:o.value.items?ue(e.value.relativePath,o.value.items):!1),h=y(()=>!!(o.value.items&&o.value.items.length));Z(()=>{s.value=!!(n.value&&o.value.collapsed)}),me(()=>{(c.value||d.value)&&(s.value=!1)});function P(){n.value&&(s.value=!s.value)}return{collapsed:s,collapsible:n,isLink:r,isActiveLink:c,hasActiveLink:d,hasChildren:h,toggle:P}}function Vt(){const{hasSidebar:o}=U(),e=ie("(min-width: 960px)"),t=ie("(min-width: 1280px)");return{isAsideEnabled:y(()=>!t.value&&!e.value?!1:o.value?t.value:e.value)}}const de=[];function Ee(o){return typeof o.outline=="object"&&!Array.isArray(o.outline)&&o.outline.label||o.outlineTitle||"On this page"}function be(o){const e=[...document.querySelectorAll(".VPDoc :where(h1,h2,h3,h4,h5,h6)")].filter(t=>t.id&&t.hasChildNodes()).map(t=>{const s=Number(t.tagName[1]);return{element:t,title:St(t),link:"#"+t.id,level:s}});return Lt(e,o)}function St(o){let e="";for(const t of o.childNodes)if(t.nodeType===1){if(t.classList.contains("VPBadge")||t.classList.contains("header-anchor")||t.classList.contains("ignore-header"))continue;e+=t.textContent}else t.nodeType===3&&(e+=t.textContent);return e.trim()}function Lt(o,e){if(e===!1)return[];const t=(typeof e=="object"&&!Array.isArray(e)?e.level:e)||2,[s,n]=typeof t=="number"?[t,t]:t==="deep"?[2,6]:t;return Nt(o,s,n)}function Tt(o,e){const{isAsideEnabled:t}=Vt(),s=ut(r,100);let n=null;R(()=>{requestAnimationFrame(r),window.addEventListener("scroll",s)}),We(()=>{c(location.hash)}),fe(()=>{window.removeEventListener("scroll",s)});function r(){if(!t.value)return;const v=window.scrollY,d=window.innerHeight,h=document.body.offsetHeight,P=Math.abs(v+d-h)<1,_=de.map(({element:S,link:C})=>({link:C,top:wt(S)})).filter(({top:S})=>!Number.isNaN(S)).sort((S,C)=>S.top-C.top);if(!_.length){c(null);return}if(v<1){c(null);return}if(P){c(_[_.length-1].link);return}let V=null;for(const{link:S,top:C}of _){if(C>v+Je()+4)break;V=S}c(V)}function c(v){n&&n.classList.remove("active"),v==null?n=null:n=o.value.querySelector(`a[href="${decodeURIComponent(v)}"]`);const d=n;d?(d.classList.add("active"),e.value.style.top=d.offsetTop+39+"px",e.value.style.opacity="1"):(e.value.style.top="33px",e.value.style.opacity="0")}}function wt(o){let e=0;for(;o!==document.body;){if(o===null)return NaN;e+=o.offsetTop,o=o.offsetParent}return e}function Nt(o,e,t){de.length=0;const s=[],n=[];return o.forEach(r=>{const c={...r,children:[]};let v=n[n.length-1];for(;v&&v.level>=c.level;)n.pop(),v=n[n.length-1];if(c.element.classList.contains("ignore-header")||v&&"shouldIgnore"in v){n.push({level:c.level,shouldIgnore:!0});return}c.level>t||c.level{const n=K("VPDocOutlineItem",!0);return a(),l("ul",{class:I(["VPDocOutlineItem",t.root?"root":"nested"])},[(a(!0),l(M,null,B(t.headers,({children:r,link:c,title:v})=>(a(),l("li",null,[p("a",{class:"outline-link",href:c,onClick:e,title:v},w(v),9,It),r!=null&&r.length?(a(),$(n,{key:0,headers:r},null,8,["headers"])):m("",!0)]))),256))],2)}}}),He=k(Mt,[["__scopeId","data-v-3f927ebe"]]),Ct={class:"content"},At={"aria-level":"2",class:"outline-title",id:"doc-outline-aria-label",role:"heading"},Bt=b({__name:"VPDocAsideOutline",setup(o){const{frontmatter:e,theme:t}=L(),s=Te([]);x(()=>{s.value=be(e.value.outline??t.value.outline)});const n=T(),r=T();return Tt(n,r),(c,v)=>(a(),l("nav",{"aria-labelledby":"doc-outline-aria-label",class:I(["VPDocAsideOutline",{"has-outline":s.value.length>0}]),ref_key:"container",ref:n},[p("div",Ct,[p("div",{class:"outline-marker",ref_key:"marker",ref:r},null,512),p("div",At,w(i(Ee)(i(t))),1),g(He,{headers:s.value,root:!0},null,8,["headers"])])],2))}}),Et=k(Bt,[["__scopeId","data-v-b38bf2ff"]]),Ht={class:"VPDocAsideCarbonAds"},Ot=b({__name:"VPDocAsideCarbonAds",props:{carbonAds:{}},setup(o){const e=()=>null;return(t,s)=>(a(),l("div",Ht,[g(i(e),{"carbon-ads":t.carbonAds},null,8,["carbon-ads"])]))}}),Dt={class:"VPDocAside"},Ft=b({__name:"VPDocAside",setup(o){const{theme:e}=L();return(t,s)=>(a(),l("div",Dt,[u(t.$slots,"aside-top",{},void 0,!0),u(t.$slots,"aside-outline-before",{},void 0,!0),g(Et),u(t.$slots,"aside-outline-after",{},void 0,!0),s[0]||(s[0]=p("div",{class:"spacer"},null,-1)),u(t.$slots,"aside-ads-before",{},void 0,!0),i(e).carbonAds?(a(),$(Ot,{key:0,"carbon-ads":i(e).carbonAds},null,8,["carbon-ads"])):m("",!0),u(t.$slots,"aside-ads-after",{},void 0,!0),u(t.$slots,"aside-bottom",{},void 0,!0)]))}}),Rt=k(Ft,[["__scopeId","data-v-6d7b3c46"]]);function Ut(){const{theme:o,page:e}=L();return y(()=>{const{text:t="Edit this page",pattern:s=""}=o.value.editLink||{};let n;return typeof s=="function"?n=s(e.value):n=s.replace(/:path/g,e.value.filePath),{url:n,text:t}})}function zt(){const{page:o,theme:e,frontmatter:t}=L();return y(()=>{var h,P,_,V,S,C,A,N;const s=Be(e.value.sidebar,o.value.relativePath),n=$t(s),r=jt(n,E=>E.link.replace(/[?#].*$/,"")),c=r.findIndex(E=>G(o.value.relativePath,E.link)),v=((h=e.value.docFooter)==null?void 0:h.prev)===!1&&!t.value.prev||t.value.prev===!1,d=((P=e.value.docFooter)==null?void 0:P.next)===!1&&!t.value.next||t.value.next===!1;return{prev:v?void 0:{text:(typeof t.value.prev=="string"?t.value.prev:typeof t.value.prev=="object"?t.value.prev.text:void 0)??((_=r[c-1])==null?void 0:_.docFooterText)??((V=r[c-1])==null?void 0:V.text),link:(typeof t.value.prev=="object"?t.value.prev.link:void 0)??((S=r[c-1])==null?void 0:S.link)},next:d?void 0:{text:(typeof t.value.next=="string"?t.value.next:typeof t.value.next=="object"?t.value.next.text:void 0)??((C=r[c+1])==null?void 0:C.docFooterText)??((A=r[c+1])==null?void 0:A.text),link:(typeof t.value.next=="object"?t.value.next.link:void 0)??((N=r[c+1])==null?void 0:N.link)}}})}function jt(o,e){const t=new Set;return o.filter(s=>{const n=e(s);return t.has(n)?!1:t.add(n)})}const H=b({__name:"VPLink",props:{tag:{},href:{},noIcon:{type:Boolean},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.tag??(e.href?"a":"span")),s=y(()=>e.href&&we.test(e.href)||e.target==="_blank");return(n,r)=>(a(),$(O(t.value),{class:I(["VPLink",{link:n.href,"vp-external-link-icon":s.value,"no-icon":n.noIcon}]),href:n.href?i(_e)(n.href):void 0,target:n.target??(s.value?"_blank":void 0),rel:n.rel??(s.value?"noreferrer":void 0)},{default:f(()=>[u(n.$slots,"default")]),_:3},8,["class","href","target","rel"]))}}),Gt={class:"VPLastUpdated"},Kt=["datetime"],qt=b({__name:"VPDocFooterLastUpdated",setup(o){const{theme:e,page:t,lang:s}=L(),n=y(()=>new Date(t.value.lastUpdated)),r=y(()=>n.value.toISOString()),c=T("");return R(()=>{Z(()=>{var v,d,h;c.value=new Intl.DateTimeFormat((d=(v=e.value.lastUpdated)==null?void 0:v.formatOptions)!=null&&d.forceLocale?s.value:void 0,((h=e.value.lastUpdated)==null?void 0:h.formatOptions)??{dateStyle:"short",timeStyle:"short"}).format(n.value)})}),(v,d)=>{var h;return a(),l("p",Gt,[F(w(((h=i(e).lastUpdated)==null?void 0:h.text)||i(e).lastUpdatedText||"Last updated")+": ",1),p("time",{datetime:r.value},w(c.value),9,Kt)])}}}),Wt=k(qt,[["__scopeId","data-v-475f71b8"]]),Jt={key:0,class:"VPDocFooter"},Yt={key:0,class:"edit-info"},Qt={key:0,class:"edit-link"},Xt={key:1,class:"last-updated"},Zt={key:1,class:"prev-next","aria-labelledby":"doc-footer-aria-label"},xt={class:"pager"},en=["innerHTML"],tn=["innerHTML"],nn={class:"pager"},sn=["innerHTML"],on=["innerHTML"],an=b({__name:"VPDocFooter",setup(o){const{theme:e,page:t,frontmatter:s}=L(),n=Ut(),r=zt(),c=y(()=>e.value.editLink&&s.value.editLink!==!1),v=y(()=>t.value.lastUpdated),d=y(()=>c.value||v.value||r.value.prev||r.value.next);return(h,P)=>{var _,V,S,C;return d.value?(a(),l("footer",Jt,[u(h.$slots,"doc-footer-before",{},void 0,!0),c.value||v.value?(a(),l("div",Yt,[c.value?(a(),l("div",Qt,[g(H,{class:"edit-link-button",href:i(n).url,"no-icon":!0},{default:f(()=>[P[0]||(P[0]=p("span",{class:"vpi-square-pen edit-link-icon"},null,-1)),F(" "+w(i(n).text),1)]),_:1},8,["href"])])):m("",!0),v.value?(a(),l("div",Xt,[g(Wt)])):m("",!0)])):m("",!0),(_=i(r).prev)!=null&&_.link||(V=i(r).next)!=null&&V.link?(a(),l("nav",Zt,[P[1]||(P[1]=p("span",{class:"visually-hidden",id:"doc-footer-aria-label"},"Pager",-1)),p("div",xt,[(S=i(r).prev)!=null&&S.link?(a(),$(H,{key:0,class:"pager-link prev",href:i(r).prev.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.prev)||"Previous page"},null,8,en),p("span",{class:"title",innerHTML:i(r).prev.text},null,8,tn)]}),_:1},8,["href"])):m("",!0)]),p("div",nn,[(C=i(r).next)!=null&&C.link?(a(),$(H,{key:0,class:"pager-link next",href:i(r).next.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.next)||"Next page"},null,8,sn),p("span",{class:"title",innerHTML:i(r).next.text},null,8,on)]}),_:1},8,["href"])):m("",!0)])])):m("",!0)])):m("",!0)}}}),rn=k(an,[["__scopeId","data-v-4f9813fa"]]),ln={class:"container"},cn={class:"aside-container"},un={class:"aside-content"},dn={class:"content"},pn={class:"content-container"},vn={class:"main"},fn=b({__name:"VPDoc",setup(o){const{theme:e}=L(),t=ee(),{hasSidebar:s,hasAside:n,leftAside:r}=U(),c=y(()=>t.path.replace(/[./]+/g,"_").replace(/_html$/,""));return(v,d)=>{const h=K("Content");return a(),l("div",{class:I(["VPDoc",{"has-sidebar":i(s),"has-aside":i(n)}])},[u(v.$slots,"doc-top",{},void 0,!0),p("div",ln,[i(n)?(a(),l("div",{key:0,class:I(["aside",{"left-aside":i(r)}])},[d[0]||(d[0]=p("div",{class:"aside-curtain"},null,-1)),p("div",cn,[p("div",un,[g(Rt,null,{"aside-top":f(()=>[u(v.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(v.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(v.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(v.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(v.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(v.$slots,"aside-ads-after",{},void 0,!0)]),_:3})])])],2)):m("",!0),p("div",dn,[p("div",pn,[u(v.$slots,"doc-before",{},void 0,!0),p("main",vn,[g(h,{class:I(["vp-doc",[c.value,i(e).externalLinkIcon&&"external-link-icon-enabled"]])},null,8,["class"])]),g(rn,null,{"doc-footer-before":f(()=>[u(v.$slots,"doc-footer-before",{},void 0,!0)]),_:3}),u(v.$slots,"doc-after",{},void 0,!0)])])]),u(v.$slots,"doc-bottom",{},void 0,!0)],2)}}}),mn=k(fn,[["__scopeId","data-v-83890dd9"]]),hn=b({__name:"VPButton",props:{tag:{},size:{default:"medium"},theme:{default:"brand"},text:{},href:{},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.href&&we.test(e.href)),s=y(()=>e.tag||(e.href?"a":"button"));return(n,r)=>(a(),$(O(s.value),{class:I(["VPButton",[n.size,n.theme]]),href:n.href?i(_e)(n.href):void 0,target:e.target??(t.value?"_blank":void 0),rel:e.rel??(t.value?"noreferrer":void 0)},{default:f(()=>[F(w(n.text),1)]),_:1},8,["class","href","target","rel"]))}}),_n=k(hn,[["__scopeId","data-v-906d7fb4"]]),bn=["src","alt"],kn=b({inheritAttrs:!1,__name:"VPImage",props:{image:{},alt:{}},setup(o){return(e,t)=>{const s=K("VPImage",!0);return e.image?(a(),l(M,{key:0},[typeof e.image=="string"||"src"in e.image?(a(),l("img",j({key:0,class:"VPImage"},typeof e.image=="string"?e.$attrs:{...e.image,...e.$attrs},{src:i(ve)(typeof e.image=="string"?e.image:e.image.src),alt:e.alt??(typeof e.image=="string"?"":e.image.alt||"")}),null,16,bn)):(a(),l(M,{key:1},[g(s,j({class:"dark",image:e.image.dark,alt:e.image.alt},e.$attrs),null,16,["image","alt"]),g(s,j({class:"light",image:e.image.light,alt:e.image.alt},e.$attrs),null,16,["image","alt"])],64))],64)):m("",!0)}}}),X=k(kn,[["__scopeId","data-v-35a7d0b8"]]),gn={class:"container"},$n={class:"main"},yn={key:0,class:"name"},Pn=["innerHTML"],Vn=["innerHTML"],Sn=["innerHTML"],Ln={key:0,class:"actions"},Tn={key:0,class:"image"},wn={class:"image-container"},Nn=b({__name:"VPHero",props:{name:{},text:{},tagline:{},image:{},actions:{}},setup(o){const e=W("hero-image-slot-exists");return(t,s)=>(a(),l("div",{class:I(["VPHero",{"has-image":t.image||i(e)}])},[p("div",gn,[p("div",$n,[u(t.$slots,"home-hero-info-before",{},void 0,!0),u(t.$slots,"home-hero-info",{},()=>[t.name?(a(),l("h1",yn,[p("span",{innerHTML:t.name,class:"clip"},null,8,Pn)])):m("",!0),t.text?(a(),l("p",{key:1,innerHTML:t.text,class:"text"},null,8,Vn)):m("",!0),t.tagline?(a(),l("p",{key:2,innerHTML:t.tagline,class:"tagline"},null,8,Sn)):m("",!0)],!0),u(t.$slots,"home-hero-info-after",{},void 0,!0),t.actions?(a(),l("div",Ln,[(a(!0),l(M,null,B(t.actions,n=>(a(),l("div",{key:n.link,class:"action"},[g(_n,{tag:"a",size:"medium",theme:n.theme,text:n.text,href:n.link,target:n.target,rel:n.rel},null,8,["theme","text","href","target","rel"])]))),128))])):m("",!0),u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),t.image||i(e)?(a(),l("div",Tn,[p("div",wn,[s[0]||(s[0]=p("div",{class:"image-bg"},null,-1)),u(t.$slots,"home-hero-image",{},()=>[t.image?(a(),$(X,{key:0,class:"image-src",image:t.image},null,8,["image"])):m("",!0)],!0)])])):m("",!0)])],2))}}),In=k(Nn,[["__scopeId","data-v-955009fc"]]),Mn=b({__name:"VPHomeHero",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).hero?(a(),$(In,{key:0,class:"VPHomeHero",name:i(e).hero.name,text:i(e).hero.text,tagline:i(e).hero.tagline,image:i(e).hero.image,actions:i(e).hero.actions},{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before")]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info")]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after")]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after")]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image")]),_:3},8,["name","text","tagline","image","actions"])):m("",!0)}}),Cn={class:"box"},An={key:0,class:"icon"},Bn=["innerHTML"],En=["innerHTML"],Hn=["innerHTML"],On={key:4,class:"link-text"},Dn={class:"link-text-value"},Fn=b({__name:"VPFeature",props:{icon:{},title:{},details:{},link:{},linkText:{},rel:{},target:{}},setup(o){return(e,t)=>(a(),$(H,{class:"VPFeature",href:e.link,rel:e.rel,target:e.target,"no-icon":!0,tag:e.link?"a":"div"},{default:f(()=>[p("article",Cn,[typeof e.icon=="object"&&e.icon.wrap?(a(),l("div",An,[g(X,{image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])])):typeof e.icon=="object"?(a(),$(X,{key:1,image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])):e.icon?(a(),l("div",{key:2,class:"icon",innerHTML:e.icon},null,8,Bn)):m("",!0),p("h2",{class:"title",innerHTML:e.title},null,8,En),e.details?(a(),l("p",{key:3,class:"details",innerHTML:e.details},null,8,Hn)):m("",!0),e.linkText?(a(),l("div",On,[p("p",Dn,[F(w(e.linkText)+" ",1),t[0]||(t[0]=p("span",{class:"vpi-arrow-right link-text-icon"},null,-1))])])):m("",!0)])]),_:1},8,["href","rel","target","tag"]))}}),Rn=k(Fn,[["__scopeId","data-v-f5e9645b"]]),Un={key:0,class:"VPFeatures"},zn={class:"container"},jn={class:"items"},Gn=b({__name:"VPFeatures",props:{features:{}},setup(o){const e=o,t=y(()=>{const s=e.features.length;if(s){if(s===2)return"grid-2";if(s===3)return"grid-3";if(s%3===0)return"grid-6";if(s>3)return"grid-4"}else return});return(s,n)=>s.features?(a(),l("div",Un,[p("div",zn,[p("div",jn,[(a(!0),l(M,null,B(s.features,r=>(a(),l("div",{key:r.title,class:I(["item",[t.value]])},[g(Rn,{icon:r.icon,title:r.title,details:r.details,link:r.link,"link-text":r.linkText,rel:r.rel,target:r.target},null,8,["icon","title","details","link","link-text","rel","target"])],2))),128))])])])):m("",!0)}}),Kn=k(Gn,[["__scopeId","data-v-d0a190d7"]]),qn=b({__name:"VPHomeFeatures",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).features?(a(),$(Kn,{key:0,class:"VPHomeFeatures",features:i(e).features},null,8,["features"])):m("",!0)}}),Wn=b({__name:"VPHomeContent",setup(o){const{width:e}=Ye({initialWidth:0,includeScrollbar:!1});return(t,s)=>(a(),l("div",{class:"vp-doc container",style:Ne(i(e)?{"--vp-offset":`calc(50% - ${i(e)/2}px)`}:{})},[u(t.$slots,"default",{},void 0,!0)],4))}}),Jn=k(Wn,[["__scopeId","data-v-7a48a447"]]),Yn={class:"VPHome"},Qn=b({__name:"VPHome",setup(o){const{frontmatter:e}=L();return(t,s)=>{const n=K("Content");return a(),l("div",Yn,[u(t.$slots,"home-hero-before",{},void 0,!0),g(Mn,null,{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image",{},void 0,!0)]),_:3}),u(t.$slots,"home-hero-after",{},void 0,!0),u(t.$slots,"home-features-before",{},void 0,!0),g(qn),u(t.$slots,"home-features-after",{},void 0,!0),i(e).markdownStyles!==!1?(a(),$(Jn,{key:0},{default:f(()=>[g(n)]),_:1})):(a(),$(n,{key:1}))])}}}),Xn=k(Qn,[["__scopeId","data-v-cbb6ec48"]]),Zn={},xn={class:"VPPage"};function es(o,e){const t=K("Content");return a(),l("div",xn,[u(o.$slots,"page-top"),g(t),u(o.$slots,"page-bottom")])}const ts=k(Zn,[["render",es]]),ns=b({__name:"VPContent",setup(o){const{page:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>(a(),l("div",{class:I(["VPContent",{"has-sidebar":i(s),"is-home":i(t).layout==="home"}]),id:"VPContent"},[i(e).isNotFound?u(n.$slots,"not-found",{key:0},()=>[g(kt)],!0):i(t).layout==="page"?(a(),$(ts,{key:1},{"page-top":f(()=>[u(n.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(n.$slots,"page-bottom",{},void 0,!0)]),_:3})):i(t).layout==="home"?(a(),$(Xn,{key:2},{"home-hero-before":f(()=>[u(n.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(n.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(n.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(n.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(n.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(n.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(n.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(n.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(n.$slots,"home-features-after",{},void 0,!0)]),_:3})):i(t).layout&&i(t).layout!=="doc"?(a(),$(O(i(t).layout),{key:3})):(a(),$(mn,{key:4},{"doc-top":f(()=>[u(n.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(n.$slots,"doc-bottom",{},void 0,!0)]),"doc-footer-before":f(()=>[u(n.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(n.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(n.$slots,"doc-after",{},void 0,!0)]),"aside-top":f(()=>[u(n.$slots,"aside-top",{},void 0,!0)]),"aside-outline-before":f(()=>[u(n.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(n.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(n.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(n.$slots,"aside-ads-after",{},void 0,!0)]),"aside-bottom":f(()=>[u(n.$slots,"aside-bottom",{},void 0,!0)]),_:3}))],2))}}),ss=k(ns,[["__scopeId","data-v-91765379"]]),os={class:"container"},as=["innerHTML"],rs=["innerHTML"],is=b({__name:"VPFooter",setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>i(e).footer&&i(t).footer!==!1?(a(),l("footer",{key:0,class:I(["VPFooter",{"has-sidebar":i(s)}])},[p("div",os,[i(e).footer.message?(a(),l("p",{key:0,class:"message",innerHTML:i(e).footer.message},null,8,as)):m("",!0),i(e).footer.copyright?(a(),l("p",{key:1,class:"copyright",innerHTML:i(e).footer.copyright},null,8,rs)):m("",!0)])],2)):m("",!0)}}),ls=k(is,[["__scopeId","data-v-c970a860"]]);function cs(){const{theme:o,frontmatter:e}=L(),t=Te([]),s=y(()=>t.value.length>0);return x(()=>{t.value=be(e.value.outline??o.value.outline)}),{headers:t,hasLocalNav:s}}const us={class:"menu-text"},ds={class:"header"},ps={class:"outline"},vs=b({__name:"VPLocalNavOutlineDropdown",props:{headers:{},navHeight:{}},setup(o){const e=o,{theme:t}=L(),s=T(!1),n=T(0),r=T(),c=T();function v(_){var V;(V=r.value)!=null&&V.contains(_.target)||(s.value=!1)}D(s,_=>{if(_){document.addEventListener("click",v);return}document.removeEventListener("click",v)}),le("Escape",()=>{s.value=!1}),x(()=>{s.value=!1});function d(){s.value=!s.value,n.value=window.innerHeight+Math.min(window.scrollY-e.navHeight,0)}function h(_){_.target.classList.contains("outline-link")&&(c.value&&(c.value.style.transition="none"),Ie(()=>{s.value=!1}))}function P(){s.value=!1,window.scrollTo({top:0,left:0,behavior:"smooth"})}return(_,V)=>(a(),l("div",{class:"VPLocalNavOutlineDropdown",style:Ne({"--vp-vh":n.value+"px"}),ref_key:"main",ref:r},[_.headers.length>0?(a(),l("button",{key:0,onClick:d,class:I({open:s.value})},[p("span",us,w(i(Ee)(i(t))),1),V[0]||(V[0]=p("span",{class:"vpi-chevron-right icon"},null,-1))],2)):(a(),l("button",{key:1,onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)),g(pe,{name:"flyout"},{default:f(()=>[s.value?(a(),l("div",{key:0,ref_key:"items",ref:c,class:"items",onClick:h},[p("div",ds,[p("a",{class:"top-link",href:"#",onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)]),p("div",ps,[g(He,{headers:_.headers},null,8,["headers"])])],512)):m("",!0)]),_:1})],4))}}),fs=k(vs,[["__scopeId","data-v-bc9dc845"]]),ms={class:"container"},hs=["aria-expanded"],_s={class:"menu-text"},bs=b({__name:"VPLocalNav",props:{open:{type:Boolean}},emits:["open-menu"],setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U(),{headers:n}=cs(),{y:r}=Me(),c=T(0);R(()=>{c.value=parseInt(getComputedStyle(document.documentElement).getPropertyValue("--vp-nav-height"))}),x(()=>{n.value=be(t.value.outline??e.value.outline)});const v=y(()=>n.value.length===0),d=y(()=>v.value&&!s.value),h=y(()=>({VPLocalNav:!0,"has-sidebar":s.value,empty:v.value,fixed:d.value}));return(P,_)=>i(t).layout!=="home"&&(!d.value||i(r)>=c.value)?(a(),l("div",{key:0,class:I(h.value)},[p("div",ms,[i(s)?(a(),l("button",{key:0,class:"menu","aria-expanded":P.open,"aria-controls":"VPSidebarNav",onClick:_[0]||(_[0]=V=>P.$emit("open-menu"))},[_[1]||(_[1]=p("span",{class:"vpi-align-left menu-icon"},null,-1)),p("span",_s,w(i(e).sidebarMenuLabel||"Menu"),1)],8,hs)):m("",!0),g(fs,{headers:i(n),navHeight:c.value},null,8,["headers","navHeight"])])],2)):m("",!0)}}),ks=k(bs,[["__scopeId","data-v-070ab83d"]]);function gs(){const o=T(!1);function e(){o.value=!0,window.addEventListener("resize",n)}function t(){o.value=!1,window.removeEventListener("resize",n)}function s(){o.value?t():e()}function n(){window.outerWidth>=768&&t()}const r=ee();return D(()=>r.path,t),{isScreenOpen:o,openScreen:e,closeScreen:t,toggleScreen:s}}const $s={},ys={class:"VPSwitch",type:"button",role:"switch"},Ps={class:"check"},Vs={key:0,class:"icon"};function Ss(o,e){return a(),l("button",ys,[p("span",Ps,[o.$slots.default?(a(),l("span",Vs,[u(o.$slots,"default",{},void 0,!0)])):m("",!0)])])}const Ls=k($s,[["render",Ss],["__scopeId","data-v-4a1c76db"]]),Ts=b({__name:"VPSwitchAppearance",setup(o){const{isDark:e,theme:t}=L(),s=W("toggle-appearance",()=>{e.value=!e.value}),n=T("");return me(()=>{n.value=e.value?t.value.lightModeSwitchTitle||"Switch to light theme":t.value.darkModeSwitchTitle||"Switch to dark theme"}),(r,c)=>(a(),$(Ls,{title:n.value,class:"VPSwitchAppearance","aria-checked":i(e),onClick:i(s)},{default:f(()=>c[0]||(c[0]=[p("span",{class:"vpi-sun sun"},null,-1),p("span",{class:"vpi-moon moon"},null,-1)])),_:1},8,["title","aria-checked","onClick"]))}}),ke=k(Ts,[["__scopeId","data-v-e40a8bb6"]]),ws={key:0,class:"VPNavBarAppearance"},Ns=b({__name:"VPNavBarAppearance",setup(o){const{site:e}=L();return(t,s)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",ws,[g(ke)])):m("",!0)}}),Is=k(Ns,[["__scopeId","data-v-af096f4a"]]),ge=T();let Oe=!1,re=0;function Ms(o){const e=T(!1);if(te){!Oe&&Cs(),re++;const t=D(ge,s=>{var n,r,c;s===o.el.value||(n=o.el.value)!=null&&n.contains(s)?(e.value=!0,(r=o.onFocus)==null||r.call(o)):(e.value=!1,(c=o.onBlur)==null||c.call(o))});fe(()=>{t(),re--,re||As()})}return Qe(e)}function Cs(){document.addEventListener("focusin",De),Oe=!0,ge.value=document.activeElement}function As(){document.removeEventListener("focusin",De)}function De(){ge.value=document.activeElement}const Bs={class:"VPMenuLink"},Es=["innerHTML"],Hs=b({__name:"VPMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),l("div",Bs,[g(H,{class:I({active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Es)]),_:1},8,["class","href","target","rel","no-icon"])]))}}),ne=k(Hs,[["__scopeId","data-v-acbfed09"]]),Os={class:"VPMenuGroup"},Ds={key:0,class:"title"},Fs=b({__name:"VPMenuGroup",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",Os,[e.text?(a(),l("p",Ds,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),l(M,null,["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):m("",!0)],64))),256))]))}}),Rs=k(Fs,[["__scopeId","data-v-48c802d0"]]),Us={class:"VPMenu"},zs={key:0,class:"items"},js=b({__name:"VPMenu",props:{items:{}},setup(o){return(e,t)=>(a(),l("div",Us,[e.items?(a(),l("div",zs,[(a(!0),l(M,null,B(e.items,s=>(a(),l(M,{key:JSON.stringify(s)},["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):"component"in s?(a(),$(O(s.component),j({key:1,ref_for:!0},s.props),null,16)):(a(),$(Rs,{key:2,text:s.text,items:s.items},null,8,["text","items"]))],64))),128))])):m("",!0),u(e.$slots,"default",{},void 0,!0)]))}}),Gs=k(js,[["__scopeId","data-v-7dd3104a"]]),Ks=["aria-expanded","aria-label"],qs={key:0,class:"text"},Ws=["innerHTML"],Js={key:1,class:"vpi-more-horizontal icon"},Ys={class:"menu"},Qs=b({__name:"VPFlyout",props:{icon:{},button:{},label:{},items:{}},setup(o){const e=T(!1),t=T();Ms({el:t,onBlur:s});function s(){e.value=!1}return(n,r)=>(a(),l("div",{class:"VPFlyout",ref_key:"el",ref:t,onMouseenter:r[1]||(r[1]=c=>e.value=!0),onMouseleave:r[2]||(r[2]=c=>e.value=!1)},[p("button",{type:"button",class:"button","aria-haspopup":"true","aria-expanded":e.value,"aria-label":n.label,onClick:r[0]||(r[0]=c=>e.value=!e.value)},[n.button||n.icon?(a(),l("span",qs,[n.icon?(a(),l("span",{key:0,class:I([n.icon,"option-icon"])},null,2)):m("",!0),n.button?(a(),l("span",{key:1,innerHTML:n.button},null,8,Ws)):m("",!0),r[3]||(r[3]=p("span",{class:"vpi-chevron-down text-icon"},null,-1))])):(a(),l("span",Js))],8,Ks),p("div",Ys,[g(Gs,{items:n.items},{default:f(()=>[u(n.$slots,"default",{},void 0,!0)]),_:3},8,["items"])])],544))}}),$e=k(Qs,[["__scopeId","data-v-04f5c5e9"]]),Xs=["href","aria-label","innerHTML"],Zs=b({__name:"VPSocialLink",props:{icon:{},link:{},ariaLabel:{}},setup(o){const e=o,t=y(()=>typeof e.icon=="object"?e.icon.svg:``);return(s,n)=>(a(),l("a",{class:"VPSocialLink no-icon",href:s.link,"aria-label":s.ariaLabel??(typeof s.icon=="string"?s.icon:""),target:"_blank",rel:"noopener",innerHTML:t.value},null,8,Xs))}}),xs=k(Zs,[["__scopeId","data-v-717b8b75"]]),eo={class:"VPSocialLinks"},to=b({__name:"VPSocialLinks",props:{links:{}},setup(o){return(e,t)=>(a(),l("div",eo,[(a(!0),l(M,null,B(e.links,({link:s,icon:n,ariaLabel:r})=>(a(),$(xs,{key:s,icon:n,link:s,ariaLabel:r},null,8,["icon","link","ariaLabel"]))),128))]))}}),se=k(to,[["__scopeId","data-v-ee7a9424"]]),no={key:0,class:"group translations"},so={class:"trans-title"},oo={key:1,class:"group"},ao={class:"item appearance"},ro={class:"label"},io={class:"appearance-action"},lo={key:2,class:"group"},co={class:"item social-links"},uo=b({__name:"VPNavBarExtra",setup(o){const{site:e,theme:t}=L(),{localeLinks:s,currentLang:n}=Y({correspondingLink:!0}),r=y(()=>s.value.length&&n.value.label||e.value.appearance||t.value.socialLinks);return(c,v)=>r.value?(a(),$($e,{key:0,class:"VPNavBarExtra",label:"extra navigation"},{default:f(()=>[i(s).length&&i(n).label?(a(),l("div",no,[p("p",so,w(i(n).label),1),(a(!0),l(M,null,B(i(s),d=>(a(),$(ne,{key:d.link,item:d},null,8,["item"]))),128))])):m("",!0),i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",oo,[p("div",ao,[p("p",ro,w(i(t).darkModeSwitchLabel||"Appearance"),1),p("div",io,[g(ke)])])])):m("",!0),i(t).socialLinks?(a(),l("div",lo,[p("div",co,[g(se,{class:"social-links-list",links:i(t).socialLinks},null,8,["links"])])])):m("",!0)]),_:1})):m("",!0)}}),po=k(uo,[["__scopeId","data-v-925effce"]]),vo=["aria-expanded"],fo=b({__name:"VPNavBarHamburger",props:{active:{type:Boolean}},emits:["click"],setup(o){return(e,t)=>(a(),l("button",{type:"button",class:I(["VPNavBarHamburger",{active:e.active}]),"aria-label":"mobile navigation","aria-expanded":e.active,"aria-controls":"VPNavScreen",onClick:t[0]||(t[0]=s=>e.$emit("click"))},t[1]||(t[1]=[p("span",{class:"container"},[p("span",{class:"top"}),p("span",{class:"middle"}),p("span",{class:"bottom"})],-1)]),10,vo))}}),mo=k(fo,[["__scopeId","data-v-5dea55bf"]]),ho=["innerHTML"],_o=b({__name:"VPNavBarMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),$(H,{class:I({VPNavBarMenuLink:!0,active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,tabindex:"0"},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ho)]),_:1},8,["class","href","target","rel","no-icon"]))}}),bo=k(_o,[["__scopeId","data-v-956ec74c"]]),Fe=b({__name:"VPNavBarMenuGroup",props:{item:{}},setup(o){const e=o,{page:t}=L(),s=r=>"component"in r?!1:"link"in r?G(t.value.relativePath,r.link,!!e.item.activeMatch):r.items.some(s),n=y(()=>s(e.item));return(r,c)=>(a(),$($e,{class:I({VPNavBarMenuGroup:!0,active:i(G)(i(t).relativePath,r.item.activeMatch,!!r.item.activeMatch)||n.value}),button:r.item.text,items:r.item.items},null,8,["class","button","items"]))}}),ko={key:0,"aria-labelledby":"main-nav-aria-label",class:"VPNavBarMenu"},go=b({__name:"VPNavBarMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",ko,[s[0]||(s[0]=p("span",{id:"main-nav-aria-label",class:"visually-hidden"}," Main Navigation ",-1)),(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(bo,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props),null,16)):(a(),$(Fe,{key:2,item:n},null,8,["item"]))],64))),128))])):m("",!0)}}),$o=k(go,[["__scopeId","data-v-e6d46098"]]);function yo(o){const{localeIndex:e,theme:t}=L();function s(n){var C,A,N;const r=n.split("."),c=(C=t.value.search)==null?void 0:C.options,v=c&&typeof c=="object",d=v&&((N=(A=c.locales)==null?void 0:A[e.value])==null?void 0:N.translations)||null,h=v&&c.translations||null;let P=d,_=h,V=o;const S=r.pop();for(const E of r){let z=null;const q=V==null?void 0:V[E];q&&(z=V=q);const oe=_==null?void 0:_[E];oe&&(z=_=oe);const ae=P==null?void 0:P[E];ae&&(z=P=ae),q||(V=z),oe||(_=z),ae||(P=z)}return(P==null?void 0:P[S])??(_==null?void 0:_[S])??(V==null?void 0:V[S])??""}return s}const Po=["aria-label"],Vo={class:"DocSearch-Button-Container"},So={class:"DocSearch-Button-Placeholder"},Pe=b({__name:"VPNavBarSearchButton",setup(o){const t=yo({button:{buttonText:"Search",buttonAriaLabel:"Search"}});return(s,n)=>(a(),l("button",{type:"button",class:"DocSearch DocSearch-Button","aria-label":i(t)("button.buttonAriaLabel")},[p("span",Vo,[n[0]||(n[0]=p("span",{class:"vp-icon DocSearch-Search-Icon"},null,-1)),p("span",So,w(i(t)("button.buttonText")),1)]),n[1]||(n[1]=p("span",{class:"DocSearch-Button-Keys"},[p("kbd",{class:"DocSearch-Button-Key"}),p("kbd",{class:"DocSearch-Button-Key"},"K")],-1))],8,Po))}}),Lo={class:"VPNavBarSearch"},To={id:"local-search"},wo={key:1,id:"docsearch"},No=b({__name:"VPNavBarSearch",setup(o){const e=Xe(()=>Ze(()=>import("./VPLocalSearchBox.BA0mQ21I.js"),__vite__mapDeps([0,1]))),t=()=>null,{theme:s}=L(),n=T(!1),r=T(!1);R(()=>{});function c(){n.value||(n.value=!0,setTimeout(v,16))}function v(){const _=new Event("keydown");_.key="k",_.metaKey=!0,window.dispatchEvent(_),setTimeout(()=>{document.querySelector(".DocSearch-Modal")||v()},16)}function d(_){const V=_.target,S=V.tagName;return V.isContentEditable||S==="INPUT"||S==="SELECT"||S==="TEXTAREA"}const h=T(!1);le("k",_=>{(_.ctrlKey||_.metaKey)&&(_.preventDefault(),h.value=!0)}),le("/",_=>{d(_)||(_.preventDefault(),h.value=!0)});const P="local";return(_,V)=>{var S;return a(),l("div",Lo,[i(P)==="local"?(a(),l(M,{key:0},[h.value?(a(),$(i(e),{key:0,onClose:V[0]||(V[0]=C=>h.value=!1)})):m("",!0),p("div",To,[g(Pe,{onClick:V[1]||(V[1]=C=>h.value=!0)})])],64)):i(P)==="algolia"?(a(),l(M,{key:1},[n.value?(a(),$(i(t),{key:0,algolia:((S=i(s).search)==null?void 0:S.options)??i(s).algolia,onVnodeBeforeMount:V[2]||(V[2]=C=>r.value=!0)},null,8,["algolia"])):m("",!0),r.value?m("",!0):(a(),l("div",wo,[g(Pe,{onClick:c})]))],64)):m("",!0)])}}}),Io=b({__name:"VPNavBarSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavBarSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),Mo=k(Io,[["__scopeId","data-v-164c457f"]]),Co=["href","rel","target"],Ao=["innerHTML"],Bo={key:2},Eo=b({__name:"VPNavBarTitle",setup(o){const{site:e,theme:t}=L(),{hasSidebar:s}=U(),{currentLang:n}=Y(),r=y(()=>{var d;return typeof t.value.logoLink=="string"?t.value.logoLink:(d=t.value.logoLink)==null?void 0:d.link}),c=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.rel}),v=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.target});return(d,h)=>(a(),l("div",{class:I(["VPNavBarTitle",{"has-sidebar":i(s)}])},[p("a",{class:"title",href:r.value??i(_e)(i(n).link),rel:c.value,target:v.value},[u(d.$slots,"nav-bar-title-before",{},void 0,!0),i(t).logo?(a(),$(X,{key:0,class:"logo",image:i(t).logo},null,8,["image"])):m("",!0),i(t).siteTitle?(a(),l("span",{key:1,innerHTML:i(t).siteTitle},null,8,Ao)):i(t).siteTitle===void 0?(a(),l("span",Bo,w(i(e).title),1)):m("",!0),u(d.$slots,"nav-bar-title-after",{},void 0,!0)],8,Co)],2))}}),Ho=k(Eo,[["__scopeId","data-v-0f4f798b"]]),Oo={class:"items"},Do={class:"title"},Fo=b({__name:"VPNavBarTranslations",setup(o){const{theme:e}=L(),{localeLinks:t,currentLang:s}=Y({correspondingLink:!0});return(n,r)=>i(t).length&&i(s).label?(a(),$($e,{key:0,class:"VPNavBarTranslations",icon:"vpi-languages",label:i(e).langMenuLabel||"Change language"},{default:f(()=>[p("div",Oo,[p("p",Do,w(i(s).label),1),(a(!0),l(M,null,B(i(t),c=>(a(),$(ne,{key:c.link,item:c},null,8,["item"]))),128))])]),_:1},8,["label"])):m("",!0)}}),Ro=k(Fo,[["__scopeId","data-v-c80d9ad0"]]),Uo={class:"wrapper"},zo={class:"container"},jo={class:"title"},Go={class:"content"},Ko={class:"content-body"},qo=b({__name:"VPNavBar",props:{isScreenOpen:{type:Boolean}},emits:["toggle-screen"],setup(o){const e=o,{y:t}=Me(),{hasSidebar:s}=U(),{frontmatter:n}=L(),r=T({});return me(()=>{r.value={"has-sidebar":s.value,home:n.value.layout==="home",top:t.value===0,"screen-open":e.isScreenOpen}}),(c,v)=>(a(),l("div",{class:I(["VPNavBar",r.value])},[p("div",Uo,[p("div",zo,[p("div",jo,[g(Ho,null,{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),_:3})]),p("div",Go,[p("div",Ko,[u(c.$slots,"nav-bar-content-before",{},void 0,!0),g(No,{class:"search"}),g($o,{class:"menu"}),g(Ro,{class:"translations"}),g(Is,{class:"appearance"}),g(Mo,{class:"social-links"}),g(po,{class:"extra"}),u(c.$slots,"nav-bar-content-after",{},void 0,!0),g(mo,{class:"hamburger",active:c.isScreenOpen,onClick:v[0]||(v[0]=d=>c.$emit("toggle-screen"))},null,8,["active"])])])])]),v[1]||(v[1]=p("div",{class:"divider"},[p("div",{class:"divider-line"})],-1))],2))}}),Wo=k(qo,[["__scopeId","data-v-822684d1"]]),Jo={key:0,class:"VPNavScreenAppearance"},Yo={class:"text"},Qo=b({__name:"VPNavScreenAppearance",setup(o){const{site:e,theme:t}=L();return(s,n)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",Jo,[p("p",Yo,w(i(t).darkModeSwitchLabel||"Appearance"),1),g(ke)])):m("",!0)}}),Xo=k(Qo,[["__scopeId","data-v-ffb44008"]]),Zo=["innerHTML"],xo=b({__name:"VPNavScreenMenuLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Zo)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),ea=k(xo,[["__scopeId","data-v-735512b8"]]),ta=["innerHTML"],na=b({__name:"VPNavScreenMenuGroupLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuGroupLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ta)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),Re=k(na,[["__scopeId","data-v-372ae7c0"]]),sa={class:"VPNavScreenMenuGroupSection"},oa={key:0,class:"title"},aa=b({__name:"VPNavScreenMenuGroupSection",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",sa,[e.text?(a(),l("p",oa,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),$(Re,{key:s.text,item:s},null,8,["item"]))),128))]))}}),ra=k(aa,[["__scopeId","data-v-4b8941ac"]]),ia=["aria-controls","aria-expanded"],la=["innerHTML"],ca=["id"],ua={key:0,class:"item"},da={key:1,class:"item"},pa={key:2,class:"group"},va=b({__name:"VPNavScreenMenuGroup",props:{text:{},items:{}},setup(o){const e=o,t=T(!1),s=y(()=>`NavScreenGroup-${e.text.replace(" ","-").toLowerCase()}`);function n(){t.value=!t.value}return(r,c)=>(a(),l("div",{class:I(["VPNavScreenMenuGroup",{open:t.value}])},[p("button",{class:"button","aria-controls":s.value,"aria-expanded":t.value,onClick:n},[p("span",{class:"button-text",innerHTML:r.text},null,8,la),c[0]||(c[0]=p("span",{class:"vpi-plus button-icon"},null,-1))],8,ia),p("div",{id:s.value,class:"items"},[(a(!0),l(M,null,B(r.items,v=>(a(),l(M,{key:JSON.stringify(v)},["link"in v?(a(),l("div",ua,[g(Re,{item:v},null,8,["item"])])):"component"in v?(a(),l("div",da,[(a(),$(O(v.component),j({ref_for:!0},v.props,{"screen-menu":""}),null,16))])):(a(),l("div",pa,[g(ra,{text:v.text,items:v.items},null,8,["text","items"])]))],64))),128))],8,ca)],2))}}),Ue=k(va,[["__scopeId","data-v-875057a5"]]),fa={key:0,class:"VPNavScreenMenu"},ma=b({__name:"VPNavScreenMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",fa,[(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(ea,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props,{"screen-menu":""}),null,16)):(a(),$(Ue,{key:2,text:n.text||"",items:n.items},null,8,["text","items"]))],64))),128))])):m("",!0)}}),ha=b({__name:"VPNavScreenSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavScreenSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),_a={class:"list"},ba=b({__name:"VPNavScreenTranslations",setup(o){const{localeLinks:e,currentLang:t}=Y({correspondingLink:!0}),s=T(!1);function n(){s.value=!s.value}return(r,c)=>i(e).length&&i(t).label?(a(),l("div",{key:0,class:I(["VPNavScreenTranslations",{open:s.value}])},[p("button",{class:"title",onClick:n},[c[0]||(c[0]=p("span",{class:"vpi-languages icon lang"},null,-1)),F(" "+w(i(t).label)+" ",1),c[1]||(c[1]=p("span",{class:"vpi-chevron-down icon chevron"},null,-1))]),p("ul",_a,[(a(!0),l(M,null,B(i(e),v=>(a(),l("li",{key:v.link,class:"item"},[g(H,{class:"link",href:v.link},{default:f(()=>[F(w(v.text),1)]),_:2},1032,["href"])]))),128))])],2)):m("",!0)}}),ka=k(ba,[["__scopeId","data-v-362991c2"]]),ga={class:"container"},$a=b({__name:"VPNavScreen",props:{open:{type:Boolean}},setup(o){const e=T(null),t=Ce(te?document.body:null);return(s,n)=>(a(),$(pe,{name:"fade",onEnter:n[0]||(n[0]=r=>t.value=!0),onAfterLeave:n[1]||(n[1]=r=>t.value=!1)},{default:f(()=>[s.open?(a(),l("div",{key:0,class:"VPNavScreen",ref_key:"screen",ref:e,id:"VPNavScreen"},[p("div",ga,[u(s.$slots,"nav-screen-content-before",{},void 0,!0),g(ma,{class:"menu"}),g(ka,{class:"translations"}),g(Xo,{class:"appearance"}),g(ha,{class:"social-links"}),u(s.$slots,"nav-screen-content-after",{},void 0,!0)])],512)):m("",!0)]),_:3}))}}),ya=k($a,[["__scopeId","data-v-833aabba"]]),Pa={key:0,class:"VPNav"},Va=b({__name:"VPNav",setup(o){const{isScreenOpen:e,closeScreen:t,toggleScreen:s}=gs(),{frontmatter:n}=L(),r=y(()=>n.value.navbar!==!1);return he("close-screen",t),Z(()=>{te&&document.documentElement.classList.toggle("hide-nav",!r.value)}),(c,v)=>r.value?(a(),l("header",Pa,[g(Wo,{"is-screen-open":i(e),onToggleScreen:i(s)},{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(c.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(c.$slots,"nav-bar-content-after",{},void 0,!0)]),_:3},8,["is-screen-open","onToggleScreen"]),g(ya,{open:i(e)},{"nav-screen-content-before":f(()=>[u(c.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(c.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3},8,["open"])])):m("",!0)}}),Sa=k(Va,[["__scopeId","data-v-f1e365da"]]),La=["role","tabindex"],Ta={key:1,class:"items"},wa=b({__name:"VPSidebarItem",props:{item:{},depth:{}},setup(o){const e=o,{collapsed:t,collapsible:s,isLink:n,isActiveLink:r,hasActiveLink:c,hasChildren:v,toggle:d}=Pt(y(()=>e.item)),h=y(()=>v.value?"section":"div"),P=y(()=>n.value?"a":"div"),_=y(()=>v.value?e.depth+2===7?"p":`h${e.depth+2}`:"p"),V=y(()=>n.value?void 0:"button"),S=y(()=>[[`level-${e.depth}`],{collapsible:s.value},{collapsed:t.value},{"is-link":n.value},{"is-active":r.value},{"has-active":c.value}]);function C(N){"key"in N&&N.key!=="Enter"||!e.item.link&&d()}function A(){e.item.link&&d()}return(N,E)=>{const z=K("VPSidebarItem",!0);return a(),$(O(h.value),{class:I(["VPSidebarItem",S.value])},{default:f(()=>[N.item.text?(a(),l("div",j({key:0,class:"item",role:V.value},et(N.item.items?{click:C,keydown:C}:{},!0),{tabindex:N.item.items&&0}),[E[1]||(E[1]=p("div",{class:"indicator"},null,-1)),N.item.link?(a(),$(H,{key:0,tag:P.value,class:"link",href:N.item.link,rel:N.item.rel,target:N.item.target},{default:f(()=>[(a(),$(O(_.value),{class:"text",innerHTML:N.item.text},null,8,["innerHTML"]))]),_:1},8,["tag","href","rel","target"])):(a(),$(O(_.value),{key:1,class:"text",innerHTML:N.item.text},null,8,["innerHTML"])),N.item.collapsed!=null&&N.item.items&&N.item.items.length?(a(),l("div",{key:2,class:"caret",role:"button","aria-label":"toggle section",onClick:A,onKeydown:xe(A,["enter"]),tabindex:"0"},E[0]||(E[0]=[p("span",{class:"vpi-chevron-right caret-icon"},null,-1)]),32)):m("",!0)],16,La)):m("",!0),N.item.items&&N.item.items.length?(a(),l("div",Ta,[N.depth<5?(a(!0),l(M,{key:0},B(N.item.items,q=>(a(),$(z,{key:q.text,item:q,depth:N.depth+1},null,8,["item","depth"]))),128)):m("",!0)])):m("",!0)]),_:1},8,["class"])}}}),Na=k(wa,[["__scopeId","data-v-196b2e5f"]]),Ia=b({__name:"VPSidebarGroup",props:{items:{}},setup(o){const e=T(!0);let t=null;return R(()=>{t=setTimeout(()=>{t=null,e.value=!1},300)}),tt(()=>{t!=null&&(clearTimeout(t),t=null)}),(s,n)=>(a(!0),l(M,null,B(s.items,r=>(a(),l("div",{key:r.text,class:I(["group",{"no-transition":e.value}])},[g(Na,{item:r,depth:0},null,8,["item"])],2))),128))}}),Ma=k(Ia,[["__scopeId","data-v-9e426adc"]]),Ca={class:"nav",id:"VPSidebarNav","aria-labelledby":"sidebar-aria-label",tabindex:"-1"},Aa=b({__name:"VPSidebar",props:{open:{type:Boolean}},setup(o){const{sidebarGroups:e,hasSidebar:t}=U(),s=o,n=T(null),r=Ce(te?document.body:null);D([s,n],()=>{var v;s.open?(r.value=!0,(v=n.value)==null||v.focus()):r.value=!1},{immediate:!0,flush:"post"});const c=T(0);return D(e,()=>{c.value+=1},{deep:!0}),(v,d)=>i(t)?(a(),l("aside",{key:0,class:I(["VPSidebar",{open:v.open}]),ref_key:"navEl",ref:n,onClick:d[0]||(d[0]=nt(()=>{},["stop"]))},[d[2]||(d[2]=p("div",{class:"curtain"},null,-1)),p("nav",Ca,[d[1]||(d[1]=p("span",{class:"visually-hidden",id:"sidebar-aria-label"}," Sidebar Navigation ",-1)),u(v.$slots,"sidebar-nav-before",{},void 0,!0),(a(),$(Ma,{items:i(e),key:c.value},null,8,["items"])),u(v.$slots,"sidebar-nav-after",{},void 0,!0)])],2)):m("",!0)}}),Ba=k(Aa,[["__scopeId","data-v-18756405"]]),Ea=b({__name:"VPSkipLink",setup(o){const e=ee(),t=T();D(()=>e.path,()=>t.value.focus());function s({target:n}){const r=document.getElementById(decodeURIComponent(n.hash).slice(1));if(r){const c=()=>{r.removeAttribute("tabindex"),r.removeEventListener("blur",c)};r.setAttribute("tabindex","-1"),r.addEventListener("blur",c),r.focus(),window.scrollTo(0,0)}}return(n,r)=>(a(),l(M,null,[p("span",{ref_key:"backToTop",ref:t,tabindex:"-1"},null,512),p("a",{href:"#VPContent",class:"VPSkipLink visually-hidden",onClick:s}," Skip to content ")],64))}}),Ha=k(Ea,[["__scopeId","data-v-c3508ec8"]]),Oa=b({__name:"Layout",setup(o){const{isOpen:e,open:t,close:s}=U(),n=ee();D(()=>n.path,s),yt(e,s);const{frontmatter:r}=L(),c=Ae(),v=y(()=>!!c["home-hero-image"]);return he("hero-image-slot-exists",v),(d,h)=>{const P=K("Content");return i(r).layout!==!1?(a(),l("div",{key:0,class:I(["Layout",i(r).pageClass])},[u(d.$slots,"layout-top",{},void 0,!0),g(Ha),g(ct,{class:"backdrop",show:i(e),onClick:i(s)},null,8,["show","onClick"]),g(Sa,null,{"nav-bar-title-before":f(()=>[u(d.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(d.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(d.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(d.$slots,"nav-bar-content-after",{},void 0,!0)]),"nav-screen-content-before":f(()=>[u(d.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(d.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3}),g(ks,{open:i(e),onOpenMenu:i(t)},null,8,["open","onOpenMenu"]),g(Ba,{open:i(e)},{"sidebar-nav-before":f(()=>[u(d.$slots,"sidebar-nav-before",{},void 0,!0)]),"sidebar-nav-after":f(()=>[u(d.$slots,"sidebar-nav-after",{},void 0,!0)]),_:3},8,["open"]),g(ss,null,{"page-top":f(()=>[u(d.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(d.$slots,"page-bottom",{},void 0,!0)]),"not-found":f(()=>[u(d.$slots,"not-found",{},void 0,!0)]),"home-hero-before":f(()=>[u(d.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(d.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(d.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(d.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(d.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(d.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(d.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(d.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(d.$slots,"home-features-after",{},void 0,!0)]),"doc-footer-before":f(()=>[u(d.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(d.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(d.$slots,"doc-after",{},void 0,!0)]),"doc-top":f(()=>[u(d.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(d.$slots,"doc-bottom",{},void 0,!0)]),"aside-top":f(()=>[u(d.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(d.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(d.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(d.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(d.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(d.$slots,"aside-ads-after",{},void 0,!0)]),_:3}),g(ls),u(d.$slots,"layout-bottom",{},void 0,!0)],2)):(a(),$(P,{key:1}))}}}),Da=k(Oa,[["__scopeId","data-v-a9a9e638"]]),Fa={},Ra={class:"VPTeamPage"};function Ua(o,e){return a(),l("div",Ra,[u(o.$slots,"default")])}const Or=k(Fa,[["render",Ua],["__scopeId","data-v-c2f8e101"]]),za={},ja={class:"VPTeamPageTitle"},Ga={key:0,class:"title"},Ka={key:1,class:"lead"};function qa(o,e){return a(),l("div",ja,[o.$slots.title?(a(),l("h1",Ga,[u(o.$slots,"title",{},void 0,!0)])):m("",!0),o.$slots.lead?(a(),l("p",Ka,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0)])}const Dr=k(za,[["render",qa],["__scopeId","data-v-e277e15c"]]),Wa={},Ja={class:"VPTeamPageSection"},Ya={class:"title"},Qa={key:0,class:"title-text"},Xa={key:0,class:"lead"},Za={key:1,class:"members"};function xa(o,e){return a(),l("section",Ja,[p("div",Ya,[e[0]||(e[0]=p("div",{class:"title-line"},null,-1)),o.$slots.title?(a(),l("h2",Qa,[u(o.$slots,"title",{},void 0,!0)])):m("",!0)]),o.$slots.lead?(a(),l("p",Xa,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0),o.$slots.members?(a(),l("div",Za,[u(o.$slots,"members",{},void 0,!0)])):m("",!0)])}const Fr=k(Wa,[["render",xa],["__scopeId","data-v-d43bc49d"]]),er={class:"profile"},tr={class:"avatar"},nr=["src","alt"],sr={class:"data"},or={class:"name"},ar={key:0,class:"affiliation"},rr={key:0,class:"title"},ir={key:1,class:"at"},lr=["innerHTML"],cr={key:2,class:"links"},ur={key:0,class:"sp"},dr=b({__name:"VPTeamMembersItem",props:{size:{default:"medium"},member:{}},setup(o){return(e,t)=>(a(),l("article",{class:I(["VPTeamMembersItem",[e.size]])},[p("div",er,[p("figure",tr,[p("img",{class:"avatar-img",src:e.member.avatar,alt:e.member.name},null,8,nr)]),p("div",sr,[p("h1",or,w(e.member.name),1),e.member.title||e.member.org?(a(),l("p",ar,[e.member.title?(a(),l("span",rr,w(e.member.title),1)):m("",!0),e.member.title&&e.member.org?(a(),l("span",ir," @ ")):m("",!0),e.member.org?(a(),$(H,{key:2,class:I(["org",{link:e.member.orgLink}]),href:e.member.orgLink,"no-icon":""},{default:f(()=>[F(w(e.member.org),1)]),_:1},8,["class","href"])):m("",!0)])):m("",!0),e.member.desc?(a(),l("p",{key:1,class:"desc",innerHTML:e.member.desc},null,8,lr)):m("",!0),e.member.links?(a(),l("div",cr,[g(se,{links:e.member.links},null,8,["links"])])):m("",!0)])]),e.member.sponsor?(a(),l("div",ur,[g(H,{class:"sp-link",href:e.member.sponsor,"no-icon":""},{default:f(()=>[t[0]||(t[0]=p("span",{class:"vpi-heart sp-icon"},null,-1)),F(" "+w(e.member.actionText||"Sponsor"),1)]),_:1},8,["href"])])):m("",!0)],2))}}),pr=k(dr,[["__scopeId","data-v-f9987cb6"]]),vr={class:"container"},fr=b({__name:"VPTeamMembers",props:{size:{default:"medium"},members:{}},setup(o){const e=o,t=y(()=>[e.size,`count-${e.members.length}`]);return(s,n)=>(a(),l("div",{class:I(["VPTeamMembers",t.value])},[p("div",vr,[(a(!0),l(M,null,B(s.members,r=>(a(),l("div",{key:r.name,class:"item"},[g(pr,{size:s.size,member:r},null,8,["size","member"])]))),128))])],2))}}),Rr=k(fr,[["__scopeId","data-v-fba19bad"]]),Ve={Layout:Da,enhanceApp:({app:o})=>{o.component("Badge",rt)}},mr={};function hr(o,e){return e[0]||(e[0]=st('

    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    ',3))}const _r=k(mr,[["render",hr]]),br=b({__name:"VersionPicker",props:{screenMenu:{type:Boolean}},setup(o){const e=T([]),t=T("Versions"),s=T(!1);Le();const n=()=>typeof window<"u"&&(window.location.hostname==="localhost"||window.location.hostname==="127.0.0.1"),r=()=>{if(typeof window>"u")return"";const{origin:d,pathname:h}=window.location;if(d.includes("github.io")){const P=h.split("/").filter(Boolean),_=P.length>0?`/${P[0]}/`:"/";return`${d}${_}`}else return d},c=()=>new Promise(d=>{if(n()){d(!1);return}const h=setInterval(()=>{window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION&&(clearInterval(h),d(!0))},100);setTimeout(()=>{clearInterval(h),d(!1)},5e3)});return R(async()=>{if(!(typeof window>"u")){try{if(n()){const d=["dev"];e.value=d.map(h=>({text:h,link:"/"})),t.value="dev"}else{const d=await c(),h=y(()=>r());if(d&&window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION)e.value=window.DOC_VERSIONS.map(P=>({text:P,link:`${h.value}/${P}/`})),t.value=window.DOCUMENTER_CURRENT_VERSION;else{const P=["dev"];e.value=P.map(_=>({text:_,link:`${h.value}/${_}/`})),t.value="dev"}}}catch(d){console.warn("Error loading versions:",d);const h=["dev"],P=y(()=>r());e.value=h.map(_=>({text:_,link:`${P.value}/${_}/`})),t.value="dev"}s.value=!0}}),(d,h)=>s.value?(a(),l(M,{key:0},[!d.screenMenu&&e.value.length>0?(a(),$(Fe,{key:0,item:{text:t.value,items:e.value},class:"VPVersionPicker"},null,8,["item"])):d.screenMenu&&e.value.length>0?(a(),$(Ue,{key:1,text:t.value,items:e.value,class:"VPVersionPicker"},null,8,["text","items"])):m("",!0)],64)):m("",!0)}}),kr=k(br,[["__scopeId","data-v-d483b3a6"]]),gr=o=>{if(typeof document>"u")return{stabilizeScrollPosition:n=>async(...r)=>n(...r)};const e=document.documentElement;return{stabilizeScrollPosition:s=>async(...n)=>{const r=s(...n),c=o.value;if(!c)return r;const v=c.offsetTop-e.scrollTop;return await Ie(),e.scrollTop=c.offsetTop-v,r}}},ze="vitepress:tabSharedState",J=typeof localStorage<"u"?localStorage:null,je="vitepress:tabsSharedState",$r=()=>{const o=J==null?void 0:J.getItem(je);if(o)try{return JSON.parse(o)}catch{}return{}},yr=o=>{J&&J.setItem(je,JSON.stringify(o))},Pr=o=>{const e=ot({});D(()=>e.content,(t,s)=>{t&&s&&yr(t)},{deep:!0}),o.provide(ze,e)},Vr=(o,e)=>{const t=W(ze);if(!t)throw new Error("[vitepress-plugin-tabs] TabsSharedState should be injected");R(()=>{t.content||(t.content=$r())});const s=T(),n=y({get(){var d;const c=e.value,v=o.value;if(c){const h=(d=t.content)==null?void 0:d[c];if(h&&v.includes(h))return h}else{const h=s.value;if(h)return h}return v[0]},set(c){const v=e.value;v?t.content&&(t.content[v]=c):s.value=c}});return{selected:n,select:c=>{n.value=c}}};let Se=0;const Sr=()=>(Se++,""+Se);function Lr(){const o=Ae();return y(()=>{var s;const t=(s=o.default)==null?void 0:s.call(o);return t?t.filter(n=>typeof n.type=="object"&&"__name"in n.type&&n.type.__name==="PluginTabsTab"&&n.props).map(n=>{var r;return(r=n.props)==null?void 0:r.label}):[]})}const Ge="vitepress:tabSingleState",Tr=o=>{he(Ge,o)},wr=()=>{const o=W(Ge);if(!o)throw new Error("[vitepress-plugin-tabs] TabsSingleState should be injected");return o},Nr={class:"plugin-tabs"},Ir=["id","aria-selected","aria-controls","tabindex","onClick"],Mr=b({__name:"PluginTabs",props:{sharedStateKey:{}},setup(o){const e=o,t=Lr(),{selected:s,select:n}=Vr(t,at(e,"sharedStateKey")),r=T(),{stabilizeScrollPosition:c}=gr(r),v=c(n),d=T([]),h=_=>{var C;const V=t.value.indexOf(s.value);let S;_.key==="ArrowLeft"?S=V>=1?V-1:t.value.length-1:_.key==="ArrowRight"&&(S=V(a(),l("div",Nr,[p("div",{ref_key:"tablist",ref:r,class:"plugin-tabs--tab-list",role:"tablist",onKeydown:h},[(a(!0),l(M,null,B(i(t),S=>(a(),l("button",{id:`tab-${S}-${i(P)}`,ref_for:!0,ref_key:"buttonRefs",ref:d,key:S,role:"tab",class:"plugin-tabs--tab","aria-selected":S===i(s),"aria-controls":`panel-${S}-${i(P)}`,tabindex:S===i(s)?0:-1,onClick:()=>i(v)(S)},w(S),9,Ir))),128))],544),u(_.$slots,"default")]))}}),Cr=["id","aria-labelledby"],Ar=b({__name:"PluginTabsTab",props:{label:{}},setup(o){const{uid:e,selected:t}=wr();return(s,n)=>i(t)===s.label?(a(),l("div",{key:0,id:`panel-${s.label}-${i(e)}`,class:"plugin-tabs--content",role:"tabpanel",tabindex:"0","aria-labelledby":`tab-${s.label}-${i(e)}`},[u(s.$slots,"default",{},void 0,!0)],8,Cr)):m("",!0)}}),Br=k(Ar,[["__scopeId","data-v-9b0d03d2"]]),Er=o=>{Pr(o),o.component("PluginTabs",Mr),o.component("PluginTabsTab",Br)},Ur={extends:Ve,Layout(){return ye(Ve.Layout,null,{"aside-ads-before":()=>ye(_r)})},enhanceApp({app:o}){Er(o),o.component("VersionPicker",kr)}};export{Ur as R,Dr as V,Fr as a,Rr as b,Or as c,yo as d,L as u}; +const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=["assets/chunks/VPLocalSearchBox.omfuhOSb.js","assets/chunks/framework.CCjWn1F9.js"])))=>i.map(i=>d[i]); +import{d as b,o as a,c as l,r as u,a as F,t as w,n as I,b as $,w as f,e as m,T as pe,_ as k,u as Le,i as Ke,f as qe,g as ve,h as y,j as p,k as i,l as G,m as ie,p as T,q as D,s as Z,v as R,x as fe,y as me,z as We,A as Je,B as K,F as M,C as B,D as Te,E as x,G as g,H as O,I as we,J as ee,K as j,L as W,M as Ye,N as Ne,O as le,P as Ie,Q as Me,R as te,S as Qe,U as Xe,V as Ze,W as Ce,X as he,Y as xe,Z as et,$ as tt,a0 as nt,a1 as Ae,a2 as st,a3 as ot,a4 as at,a5 as ye}from"./framework.CCjWn1F9.js";const rt=b({__name:"VPBadge",props:{text:{},type:{default:"tip"}},setup(o){return(e,t)=>(a(),l("span",{class:I(["VPBadge",e.type])},[u(e.$slots,"default",{},()=>[F(w(e.text),1)])],2))}}),it={key:0,class:"VPBackdrop"},lt=b({__name:"VPBackdrop",props:{show:{type:Boolean}},setup(o){return(e,t)=>(a(),$(pe,{name:"fade"},{default:f(()=>[e.show?(a(),l("div",it)):m("",!0)]),_:1}))}}),ct=k(lt,[["__scopeId","data-v-b06cdb19"]]),L=Le;function ut(o,e){let t,s=!1;return()=>{t&&clearTimeout(t),s?t=setTimeout(o,e):(o(),(s=!0)&&setTimeout(()=>s=!1,e))}}function ce(o){return/^\//.test(o)?o:`/${o}`}function _e(o){const{pathname:e,search:t,hash:s,protocol:n}=new URL(o,"http://a.com");if(Ke(o)||o.startsWith("#")||!n.startsWith("http")||!qe(e))return o;const{site:r}=L(),c=e.endsWith("/")||e.endsWith(".html")?o:o.replace(/(?:(^\.+)\/)?.*$/,`$1${e.replace(/(\.md)?$/,r.value.cleanUrls?"":".html")}${t}${s}`);return ve(c)}function Y({correspondingLink:o=!1}={}){const{site:e,localeIndex:t,page:s,theme:n,hash:r}=L(),c=y(()=>{var d,h;return{label:(d=e.value.locales[t.value])==null?void 0:d.label,link:((h=e.value.locales[t.value])==null?void 0:h.link)||(t.value==="root"?"/":`/${t.value}/`)}});return{localeLinks:y(()=>Object.entries(e.value.locales).flatMap(([d,h])=>c.value.label===h.label?[]:{text:h.label,link:dt(h.link||(d==="root"?"/":`/${d}/`),n.value.i18nRouting!==!1&&o,s.value.relativePath.slice(c.value.link.length-1),!e.value.cleanUrls)+r.value})),currentLang:c}}function dt(o,e,t,s){return e?o.replace(/\/$/,"")+ce(t.replace(/(^|\/)index\.md$/,"$1").replace(/\.md$/,s?".html":"")):o}const pt={class:"NotFound"},vt={class:"code"},ft={class:"title"},mt={class:"quote"},ht={class:"action"},_t=["href","aria-label"],bt=b({__name:"NotFound",setup(o){const{theme:e}=L(),{currentLang:t}=Y();return(s,n)=>{var r,c,v,d,h;return a(),l("div",pt,[p("p",vt,w(((r=i(e).notFound)==null?void 0:r.code)??"404"),1),p("h1",ft,w(((c=i(e).notFound)==null?void 0:c.title)??"PAGE NOT FOUND"),1),n[0]||(n[0]=p("div",{class:"divider"},null,-1)),p("blockquote",mt,w(((v=i(e).notFound)==null?void 0:v.quote)??"But if you don't change your direction, and if you keep looking, you may end up where you are heading."),1),p("div",ht,[p("a",{class:"link",href:i(ve)(i(t).link),"aria-label":((d=i(e).notFound)==null?void 0:d.linkLabel)??"go to home"},w(((h=i(e).notFound)==null?void 0:h.linkText)??"Take me home"),9,_t)])])}}}),kt=k(bt,[["__scopeId","data-v-951cab6c"]]);function Be(o,e){if(Array.isArray(o))return Q(o);if(o==null)return[];e=ce(e);const t=Object.keys(o).sort((n,r)=>r.split("/").length-n.split("/").length).find(n=>e.startsWith(ce(n))),s=t?o[t]:[];return Array.isArray(s)?Q(s):Q(s.items,s.base)}function gt(o){const e=[];let t=0;for(const s in o){const n=o[s];if(n.items){t=e.push(n);continue}e[t]||e.push({items:[]}),e[t].items.push(n)}return e}function $t(o){const e=[];function t(s){for(const n of s)n.text&&n.link&&e.push({text:n.text,link:n.link,docFooterText:n.docFooterText}),n.items&&t(n.items)}return t(o),e}function ue(o,e){return Array.isArray(e)?e.some(t=>ue(o,t)):G(o,e.link)?!0:e.items?ue(o,e.items):!1}function Q(o,e){return[...o].map(t=>{const s={...t},n=s.base||e;return n&&s.link&&(s.link=n+s.link),s.items&&(s.items=Q(s.items,n)),s})}function U(){const{frontmatter:o,page:e,theme:t}=L(),s=ie("(min-width: 960px)"),n=T(!1),r=y(()=>{const A=t.value.sidebar,N=e.value.relativePath;return A?Be(A,N):[]}),c=T(r.value);D(r,(A,N)=>{JSON.stringify(A)!==JSON.stringify(N)&&(c.value=r.value)});const v=y(()=>o.value.sidebar!==!1&&c.value.length>0&&o.value.layout!=="home"),d=y(()=>h?o.value.aside==null?t.value.aside==="left":o.value.aside==="left":!1),h=y(()=>o.value.layout==="home"?!1:o.value.aside!=null?!!o.value.aside:t.value.aside!==!1),P=y(()=>v.value&&s.value),_=y(()=>v.value?gt(c.value):[]);function V(){n.value=!0}function S(){n.value=!1}function C(){n.value?S():V()}return{isOpen:n,sidebar:c,sidebarGroups:_,hasSidebar:v,hasAside:h,leftAside:d,isSidebarEnabled:P,open:V,close:S,toggle:C}}function yt(o,e){let t;Z(()=>{t=o.value?document.activeElement:void 0}),R(()=>{window.addEventListener("keyup",s)}),fe(()=>{window.removeEventListener("keyup",s)});function s(n){n.key==="Escape"&&o.value&&(e(),t==null||t.focus())}}function Pt(o){const{page:e,hash:t}=L(),s=T(!1),n=y(()=>o.value.collapsed!=null),r=y(()=>!!o.value.link),c=T(!1),v=()=>{c.value=G(e.value.relativePath,o.value.link)};D([e,o,t],v),R(v);const d=y(()=>c.value?!0:o.value.items?ue(e.value.relativePath,o.value.items):!1),h=y(()=>!!(o.value.items&&o.value.items.length));Z(()=>{s.value=!!(n.value&&o.value.collapsed)}),me(()=>{(c.value||d.value)&&(s.value=!1)});function P(){n.value&&(s.value=!s.value)}return{collapsed:s,collapsible:n,isLink:r,isActiveLink:c,hasActiveLink:d,hasChildren:h,toggle:P}}function Vt(){const{hasSidebar:o}=U(),e=ie("(min-width: 960px)"),t=ie("(min-width: 1280px)");return{isAsideEnabled:y(()=>!t.value&&!e.value?!1:o.value?t.value:e.value)}}const de=[];function Ee(o){return typeof o.outline=="object"&&!Array.isArray(o.outline)&&o.outline.label||o.outlineTitle||"On this page"}function be(o){const e=[...document.querySelectorAll(".VPDoc :where(h1,h2,h3,h4,h5,h6)")].filter(t=>t.id&&t.hasChildNodes()).map(t=>{const s=Number(t.tagName[1]);return{element:t,title:St(t),link:"#"+t.id,level:s}});return Lt(e,o)}function St(o){let e="";for(const t of o.childNodes)if(t.nodeType===1){if(t.classList.contains("VPBadge")||t.classList.contains("header-anchor")||t.classList.contains("ignore-header"))continue;e+=t.textContent}else t.nodeType===3&&(e+=t.textContent);return e.trim()}function Lt(o,e){if(e===!1)return[];const t=(typeof e=="object"&&!Array.isArray(e)?e.level:e)||2,[s,n]=typeof t=="number"?[t,t]:t==="deep"?[2,6]:t;return Nt(o,s,n)}function Tt(o,e){const{isAsideEnabled:t}=Vt(),s=ut(r,100);let n=null;R(()=>{requestAnimationFrame(r),window.addEventListener("scroll",s)}),We(()=>{c(location.hash)}),fe(()=>{window.removeEventListener("scroll",s)});function r(){if(!t.value)return;const v=window.scrollY,d=window.innerHeight,h=document.body.offsetHeight,P=Math.abs(v+d-h)<1,_=de.map(({element:S,link:C})=>({link:C,top:wt(S)})).filter(({top:S})=>!Number.isNaN(S)).sort((S,C)=>S.top-C.top);if(!_.length){c(null);return}if(v<1){c(null);return}if(P){c(_[_.length-1].link);return}let V=null;for(const{link:S,top:C}of _){if(C>v+Je()+4)break;V=S}c(V)}function c(v){n&&n.classList.remove("active"),v==null?n=null:n=o.value.querySelector(`a[href="${decodeURIComponent(v)}"]`);const d=n;d?(d.classList.add("active"),e.value.style.top=d.offsetTop+39+"px",e.value.style.opacity="1"):(e.value.style.top="33px",e.value.style.opacity="0")}}function wt(o){let e=0;for(;o!==document.body;){if(o===null)return NaN;e+=o.offsetTop,o=o.offsetParent}return e}function Nt(o,e,t){de.length=0;const s=[],n=[];return o.forEach(r=>{const c={...r,children:[]};let v=n[n.length-1];for(;v&&v.level>=c.level;)n.pop(),v=n[n.length-1];if(c.element.classList.contains("ignore-header")||v&&"shouldIgnore"in v){n.push({level:c.level,shouldIgnore:!0});return}c.level>t||c.level{const n=K("VPDocOutlineItem",!0);return a(),l("ul",{class:I(["VPDocOutlineItem",t.root?"root":"nested"])},[(a(!0),l(M,null,B(t.headers,({children:r,link:c,title:v})=>(a(),l("li",null,[p("a",{class:"outline-link",href:c,onClick:e,title:v},w(v),9,It),r!=null&&r.length?(a(),$(n,{key:0,headers:r},null,8,["headers"])):m("",!0)]))),256))],2)}}}),He=k(Mt,[["__scopeId","data-v-3f927ebe"]]),Ct={class:"content"},At={"aria-level":"2",class:"outline-title",id:"doc-outline-aria-label",role:"heading"},Bt=b({__name:"VPDocAsideOutline",setup(o){const{frontmatter:e,theme:t}=L(),s=Te([]);x(()=>{s.value=be(e.value.outline??t.value.outline)});const n=T(),r=T();return Tt(n,r),(c,v)=>(a(),l("nav",{"aria-labelledby":"doc-outline-aria-label",class:I(["VPDocAsideOutline",{"has-outline":s.value.length>0}]),ref_key:"container",ref:n},[p("div",Ct,[p("div",{class:"outline-marker",ref_key:"marker",ref:r},null,512),p("div",At,w(i(Ee)(i(t))),1),g(He,{headers:s.value,root:!0},null,8,["headers"])])],2))}}),Et=k(Bt,[["__scopeId","data-v-b38bf2ff"]]),Ht={class:"VPDocAsideCarbonAds"},Ot=b({__name:"VPDocAsideCarbonAds",props:{carbonAds:{}},setup(o){const e=()=>null;return(t,s)=>(a(),l("div",Ht,[g(i(e),{"carbon-ads":t.carbonAds},null,8,["carbon-ads"])]))}}),Dt={class:"VPDocAside"},Ft=b({__name:"VPDocAside",setup(o){const{theme:e}=L();return(t,s)=>(a(),l("div",Dt,[u(t.$slots,"aside-top",{},void 0,!0),u(t.$slots,"aside-outline-before",{},void 0,!0),g(Et),u(t.$slots,"aside-outline-after",{},void 0,!0),s[0]||(s[0]=p("div",{class:"spacer"},null,-1)),u(t.$slots,"aside-ads-before",{},void 0,!0),i(e).carbonAds?(a(),$(Ot,{key:0,"carbon-ads":i(e).carbonAds},null,8,["carbon-ads"])):m("",!0),u(t.$slots,"aside-ads-after",{},void 0,!0),u(t.$slots,"aside-bottom",{},void 0,!0)]))}}),Rt=k(Ft,[["__scopeId","data-v-6d7b3c46"]]);function Ut(){const{theme:o,page:e}=L();return y(()=>{const{text:t="Edit this page",pattern:s=""}=o.value.editLink||{};let n;return typeof s=="function"?n=s(e.value):n=s.replace(/:path/g,e.value.filePath),{url:n,text:t}})}function zt(){const{page:o,theme:e,frontmatter:t}=L();return y(()=>{var h,P,_,V,S,C,A,N;const s=Be(e.value.sidebar,o.value.relativePath),n=$t(s),r=jt(n,E=>E.link.replace(/[?#].*$/,"")),c=r.findIndex(E=>G(o.value.relativePath,E.link)),v=((h=e.value.docFooter)==null?void 0:h.prev)===!1&&!t.value.prev||t.value.prev===!1,d=((P=e.value.docFooter)==null?void 0:P.next)===!1&&!t.value.next||t.value.next===!1;return{prev:v?void 0:{text:(typeof t.value.prev=="string"?t.value.prev:typeof t.value.prev=="object"?t.value.prev.text:void 0)??((_=r[c-1])==null?void 0:_.docFooterText)??((V=r[c-1])==null?void 0:V.text),link:(typeof t.value.prev=="object"?t.value.prev.link:void 0)??((S=r[c-1])==null?void 0:S.link)},next:d?void 0:{text:(typeof t.value.next=="string"?t.value.next:typeof t.value.next=="object"?t.value.next.text:void 0)??((C=r[c+1])==null?void 0:C.docFooterText)??((A=r[c+1])==null?void 0:A.text),link:(typeof t.value.next=="object"?t.value.next.link:void 0)??((N=r[c+1])==null?void 0:N.link)}}})}function jt(o,e){const t=new Set;return o.filter(s=>{const n=e(s);return t.has(n)?!1:t.add(n)})}const H=b({__name:"VPLink",props:{tag:{},href:{},noIcon:{type:Boolean},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.tag??(e.href?"a":"span")),s=y(()=>e.href&&we.test(e.href)||e.target==="_blank");return(n,r)=>(a(),$(O(t.value),{class:I(["VPLink",{link:n.href,"vp-external-link-icon":s.value,"no-icon":n.noIcon}]),href:n.href?i(_e)(n.href):void 0,target:n.target??(s.value?"_blank":void 0),rel:n.rel??(s.value?"noreferrer":void 0)},{default:f(()=>[u(n.$slots,"default")]),_:3},8,["class","href","target","rel"]))}}),Gt={class:"VPLastUpdated"},Kt=["datetime"],qt=b({__name:"VPDocFooterLastUpdated",setup(o){const{theme:e,page:t,lang:s}=L(),n=y(()=>new Date(t.value.lastUpdated)),r=y(()=>n.value.toISOString()),c=T("");return R(()=>{Z(()=>{var v,d,h;c.value=new Intl.DateTimeFormat((d=(v=e.value.lastUpdated)==null?void 0:v.formatOptions)!=null&&d.forceLocale?s.value:void 0,((h=e.value.lastUpdated)==null?void 0:h.formatOptions)??{dateStyle:"short",timeStyle:"short"}).format(n.value)})}),(v,d)=>{var h;return a(),l("p",Gt,[F(w(((h=i(e).lastUpdated)==null?void 0:h.text)||i(e).lastUpdatedText||"Last updated")+": ",1),p("time",{datetime:r.value},w(c.value),9,Kt)])}}}),Wt=k(qt,[["__scopeId","data-v-475f71b8"]]),Jt={key:0,class:"VPDocFooter"},Yt={key:0,class:"edit-info"},Qt={key:0,class:"edit-link"},Xt={key:1,class:"last-updated"},Zt={key:1,class:"prev-next","aria-labelledby":"doc-footer-aria-label"},xt={class:"pager"},en=["innerHTML"],tn=["innerHTML"],nn={class:"pager"},sn=["innerHTML"],on=["innerHTML"],an=b({__name:"VPDocFooter",setup(o){const{theme:e,page:t,frontmatter:s}=L(),n=Ut(),r=zt(),c=y(()=>e.value.editLink&&s.value.editLink!==!1),v=y(()=>t.value.lastUpdated),d=y(()=>c.value||v.value||r.value.prev||r.value.next);return(h,P)=>{var _,V,S,C;return d.value?(a(),l("footer",Jt,[u(h.$slots,"doc-footer-before",{},void 0,!0),c.value||v.value?(a(),l("div",Yt,[c.value?(a(),l("div",Qt,[g(H,{class:"edit-link-button",href:i(n).url,"no-icon":!0},{default:f(()=>[P[0]||(P[0]=p("span",{class:"vpi-square-pen edit-link-icon"},null,-1)),F(" "+w(i(n).text),1)]),_:1},8,["href"])])):m("",!0),v.value?(a(),l("div",Xt,[g(Wt)])):m("",!0)])):m("",!0),(_=i(r).prev)!=null&&_.link||(V=i(r).next)!=null&&V.link?(a(),l("nav",Zt,[P[1]||(P[1]=p("span",{class:"visually-hidden",id:"doc-footer-aria-label"},"Pager",-1)),p("div",xt,[(S=i(r).prev)!=null&&S.link?(a(),$(H,{key:0,class:"pager-link prev",href:i(r).prev.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.prev)||"Previous page"},null,8,en),p("span",{class:"title",innerHTML:i(r).prev.text},null,8,tn)]}),_:1},8,["href"])):m("",!0)]),p("div",nn,[(C=i(r).next)!=null&&C.link?(a(),$(H,{key:0,class:"pager-link next",href:i(r).next.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.next)||"Next page"},null,8,sn),p("span",{class:"title",innerHTML:i(r).next.text},null,8,on)]}),_:1},8,["href"])):m("",!0)])])):m("",!0)])):m("",!0)}}}),rn=k(an,[["__scopeId","data-v-4f9813fa"]]),ln={class:"container"},cn={class:"aside-container"},un={class:"aside-content"},dn={class:"content"},pn={class:"content-container"},vn={class:"main"},fn=b({__name:"VPDoc",setup(o){const{theme:e}=L(),t=ee(),{hasSidebar:s,hasAside:n,leftAside:r}=U(),c=y(()=>t.path.replace(/[./]+/g,"_").replace(/_html$/,""));return(v,d)=>{const h=K("Content");return a(),l("div",{class:I(["VPDoc",{"has-sidebar":i(s),"has-aside":i(n)}])},[u(v.$slots,"doc-top",{},void 0,!0),p("div",ln,[i(n)?(a(),l("div",{key:0,class:I(["aside",{"left-aside":i(r)}])},[d[0]||(d[0]=p("div",{class:"aside-curtain"},null,-1)),p("div",cn,[p("div",un,[g(Rt,null,{"aside-top":f(()=>[u(v.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(v.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(v.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(v.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(v.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(v.$slots,"aside-ads-after",{},void 0,!0)]),_:3})])])],2)):m("",!0),p("div",dn,[p("div",pn,[u(v.$slots,"doc-before",{},void 0,!0),p("main",vn,[g(h,{class:I(["vp-doc",[c.value,i(e).externalLinkIcon&&"external-link-icon-enabled"]])},null,8,["class"])]),g(rn,null,{"doc-footer-before":f(()=>[u(v.$slots,"doc-footer-before",{},void 0,!0)]),_:3}),u(v.$slots,"doc-after",{},void 0,!0)])])]),u(v.$slots,"doc-bottom",{},void 0,!0)],2)}}}),mn=k(fn,[["__scopeId","data-v-83890dd9"]]),hn=b({__name:"VPButton",props:{tag:{},size:{default:"medium"},theme:{default:"brand"},text:{},href:{},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.href&&we.test(e.href)),s=y(()=>e.tag||(e.href?"a":"button"));return(n,r)=>(a(),$(O(s.value),{class:I(["VPButton",[n.size,n.theme]]),href:n.href?i(_e)(n.href):void 0,target:e.target??(t.value?"_blank":void 0),rel:e.rel??(t.value?"noreferrer":void 0)},{default:f(()=>[F(w(n.text),1)]),_:1},8,["class","href","target","rel"]))}}),_n=k(hn,[["__scopeId","data-v-906d7fb4"]]),bn=["src","alt"],kn=b({inheritAttrs:!1,__name:"VPImage",props:{image:{},alt:{}},setup(o){return(e,t)=>{const s=K("VPImage",!0);return e.image?(a(),l(M,{key:0},[typeof e.image=="string"||"src"in e.image?(a(),l("img",j({key:0,class:"VPImage"},typeof e.image=="string"?e.$attrs:{...e.image,...e.$attrs},{src:i(ve)(typeof e.image=="string"?e.image:e.image.src),alt:e.alt??(typeof e.image=="string"?"":e.image.alt||"")}),null,16,bn)):(a(),l(M,{key:1},[g(s,j({class:"dark",image:e.image.dark,alt:e.image.alt},e.$attrs),null,16,["image","alt"]),g(s,j({class:"light",image:e.image.light,alt:e.image.alt},e.$attrs),null,16,["image","alt"])],64))],64)):m("",!0)}}}),X=k(kn,[["__scopeId","data-v-35a7d0b8"]]),gn={class:"container"},$n={class:"main"},yn={key:0,class:"name"},Pn=["innerHTML"],Vn=["innerHTML"],Sn=["innerHTML"],Ln={key:0,class:"actions"},Tn={key:0,class:"image"},wn={class:"image-container"},Nn=b({__name:"VPHero",props:{name:{},text:{},tagline:{},image:{},actions:{}},setup(o){const e=W("hero-image-slot-exists");return(t,s)=>(a(),l("div",{class:I(["VPHero",{"has-image":t.image||i(e)}])},[p("div",gn,[p("div",$n,[u(t.$slots,"home-hero-info-before",{},void 0,!0),u(t.$slots,"home-hero-info",{},()=>[t.name?(a(),l("h1",yn,[p("span",{innerHTML:t.name,class:"clip"},null,8,Pn)])):m("",!0),t.text?(a(),l("p",{key:1,innerHTML:t.text,class:"text"},null,8,Vn)):m("",!0),t.tagline?(a(),l("p",{key:2,innerHTML:t.tagline,class:"tagline"},null,8,Sn)):m("",!0)],!0),u(t.$slots,"home-hero-info-after",{},void 0,!0),t.actions?(a(),l("div",Ln,[(a(!0),l(M,null,B(t.actions,n=>(a(),l("div",{key:n.link,class:"action"},[g(_n,{tag:"a",size:"medium",theme:n.theme,text:n.text,href:n.link,target:n.target,rel:n.rel},null,8,["theme","text","href","target","rel"])]))),128))])):m("",!0),u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),t.image||i(e)?(a(),l("div",Tn,[p("div",wn,[s[0]||(s[0]=p("div",{class:"image-bg"},null,-1)),u(t.$slots,"home-hero-image",{},()=>[t.image?(a(),$(X,{key:0,class:"image-src",image:t.image},null,8,["image"])):m("",!0)],!0)])])):m("",!0)])],2))}}),In=k(Nn,[["__scopeId","data-v-955009fc"]]),Mn=b({__name:"VPHomeHero",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).hero?(a(),$(In,{key:0,class:"VPHomeHero",name:i(e).hero.name,text:i(e).hero.text,tagline:i(e).hero.tagline,image:i(e).hero.image,actions:i(e).hero.actions},{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before")]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info")]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after")]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after")]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image")]),_:3},8,["name","text","tagline","image","actions"])):m("",!0)}}),Cn={class:"box"},An={key:0,class:"icon"},Bn=["innerHTML"],En=["innerHTML"],Hn=["innerHTML"],On={key:4,class:"link-text"},Dn={class:"link-text-value"},Fn=b({__name:"VPFeature",props:{icon:{},title:{},details:{},link:{},linkText:{},rel:{},target:{}},setup(o){return(e,t)=>(a(),$(H,{class:"VPFeature",href:e.link,rel:e.rel,target:e.target,"no-icon":!0,tag:e.link?"a":"div"},{default:f(()=>[p("article",Cn,[typeof e.icon=="object"&&e.icon.wrap?(a(),l("div",An,[g(X,{image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])])):typeof e.icon=="object"?(a(),$(X,{key:1,image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])):e.icon?(a(),l("div",{key:2,class:"icon",innerHTML:e.icon},null,8,Bn)):m("",!0),p("h2",{class:"title",innerHTML:e.title},null,8,En),e.details?(a(),l("p",{key:3,class:"details",innerHTML:e.details},null,8,Hn)):m("",!0),e.linkText?(a(),l("div",On,[p("p",Dn,[F(w(e.linkText)+" ",1),t[0]||(t[0]=p("span",{class:"vpi-arrow-right link-text-icon"},null,-1))])])):m("",!0)])]),_:1},8,["href","rel","target","tag"]))}}),Rn=k(Fn,[["__scopeId","data-v-f5e9645b"]]),Un={key:0,class:"VPFeatures"},zn={class:"container"},jn={class:"items"},Gn=b({__name:"VPFeatures",props:{features:{}},setup(o){const e=o,t=y(()=>{const s=e.features.length;if(s){if(s===2)return"grid-2";if(s===3)return"grid-3";if(s%3===0)return"grid-6";if(s>3)return"grid-4"}else return});return(s,n)=>s.features?(a(),l("div",Un,[p("div",zn,[p("div",jn,[(a(!0),l(M,null,B(s.features,r=>(a(),l("div",{key:r.title,class:I(["item",[t.value]])},[g(Rn,{icon:r.icon,title:r.title,details:r.details,link:r.link,"link-text":r.linkText,rel:r.rel,target:r.target},null,8,["icon","title","details","link","link-text","rel","target"])],2))),128))])])])):m("",!0)}}),Kn=k(Gn,[["__scopeId","data-v-d0a190d7"]]),qn=b({__name:"VPHomeFeatures",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).features?(a(),$(Kn,{key:0,class:"VPHomeFeatures",features:i(e).features},null,8,["features"])):m("",!0)}}),Wn=b({__name:"VPHomeContent",setup(o){const{width:e}=Ye({initialWidth:0,includeScrollbar:!1});return(t,s)=>(a(),l("div",{class:"vp-doc container",style:Ne(i(e)?{"--vp-offset":`calc(50% - ${i(e)/2}px)`}:{})},[u(t.$slots,"default",{},void 0,!0)],4))}}),Jn=k(Wn,[["__scopeId","data-v-7a48a447"]]),Yn={class:"VPHome"},Qn=b({__name:"VPHome",setup(o){const{frontmatter:e}=L();return(t,s)=>{const n=K("Content");return a(),l("div",Yn,[u(t.$slots,"home-hero-before",{},void 0,!0),g(Mn,null,{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image",{},void 0,!0)]),_:3}),u(t.$slots,"home-hero-after",{},void 0,!0),u(t.$slots,"home-features-before",{},void 0,!0),g(qn),u(t.$slots,"home-features-after",{},void 0,!0),i(e).markdownStyles!==!1?(a(),$(Jn,{key:0},{default:f(()=>[g(n)]),_:1})):(a(),$(n,{key:1}))])}}}),Xn=k(Qn,[["__scopeId","data-v-cbb6ec48"]]),Zn={},xn={class:"VPPage"};function es(o,e){const t=K("Content");return a(),l("div",xn,[u(o.$slots,"page-top"),g(t),u(o.$slots,"page-bottom")])}const ts=k(Zn,[["render",es]]),ns=b({__name:"VPContent",setup(o){const{page:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>(a(),l("div",{class:I(["VPContent",{"has-sidebar":i(s),"is-home":i(t).layout==="home"}]),id:"VPContent"},[i(e).isNotFound?u(n.$slots,"not-found",{key:0},()=>[g(kt)],!0):i(t).layout==="page"?(a(),$(ts,{key:1},{"page-top":f(()=>[u(n.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(n.$slots,"page-bottom",{},void 0,!0)]),_:3})):i(t).layout==="home"?(a(),$(Xn,{key:2},{"home-hero-before":f(()=>[u(n.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(n.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(n.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(n.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(n.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(n.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(n.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(n.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(n.$slots,"home-features-after",{},void 0,!0)]),_:3})):i(t).layout&&i(t).layout!=="doc"?(a(),$(O(i(t).layout),{key:3})):(a(),$(mn,{key:4},{"doc-top":f(()=>[u(n.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(n.$slots,"doc-bottom",{},void 0,!0)]),"doc-footer-before":f(()=>[u(n.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(n.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(n.$slots,"doc-after",{},void 0,!0)]),"aside-top":f(()=>[u(n.$slots,"aside-top",{},void 0,!0)]),"aside-outline-before":f(()=>[u(n.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(n.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(n.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(n.$slots,"aside-ads-after",{},void 0,!0)]),"aside-bottom":f(()=>[u(n.$slots,"aside-bottom",{},void 0,!0)]),_:3}))],2))}}),ss=k(ns,[["__scopeId","data-v-91765379"]]),os={class:"container"},as=["innerHTML"],rs=["innerHTML"],is=b({__name:"VPFooter",setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>i(e).footer&&i(t).footer!==!1?(a(),l("footer",{key:0,class:I(["VPFooter",{"has-sidebar":i(s)}])},[p("div",os,[i(e).footer.message?(a(),l("p",{key:0,class:"message",innerHTML:i(e).footer.message},null,8,as)):m("",!0),i(e).footer.copyright?(a(),l("p",{key:1,class:"copyright",innerHTML:i(e).footer.copyright},null,8,rs)):m("",!0)])],2)):m("",!0)}}),ls=k(is,[["__scopeId","data-v-c970a860"]]);function cs(){const{theme:o,frontmatter:e}=L(),t=Te([]),s=y(()=>t.value.length>0);return x(()=>{t.value=be(e.value.outline??o.value.outline)}),{headers:t,hasLocalNav:s}}const us={class:"menu-text"},ds={class:"header"},ps={class:"outline"},vs=b({__name:"VPLocalNavOutlineDropdown",props:{headers:{},navHeight:{}},setup(o){const e=o,{theme:t}=L(),s=T(!1),n=T(0),r=T(),c=T();function v(_){var V;(V=r.value)!=null&&V.contains(_.target)||(s.value=!1)}D(s,_=>{if(_){document.addEventListener("click",v);return}document.removeEventListener("click",v)}),le("Escape",()=>{s.value=!1}),x(()=>{s.value=!1});function d(){s.value=!s.value,n.value=window.innerHeight+Math.min(window.scrollY-e.navHeight,0)}function h(_){_.target.classList.contains("outline-link")&&(c.value&&(c.value.style.transition="none"),Ie(()=>{s.value=!1}))}function P(){s.value=!1,window.scrollTo({top:0,left:0,behavior:"smooth"})}return(_,V)=>(a(),l("div",{class:"VPLocalNavOutlineDropdown",style:Ne({"--vp-vh":n.value+"px"}),ref_key:"main",ref:r},[_.headers.length>0?(a(),l("button",{key:0,onClick:d,class:I({open:s.value})},[p("span",us,w(i(Ee)(i(t))),1),V[0]||(V[0]=p("span",{class:"vpi-chevron-right icon"},null,-1))],2)):(a(),l("button",{key:1,onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)),g(pe,{name:"flyout"},{default:f(()=>[s.value?(a(),l("div",{key:0,ref_key:"items",ref:c,class:"items",onClick:h},[p("div",ds,[p("a",{class:"top-link",href:"#",onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)]),p("div",ps,[g(He,{headers:_.headers},null,8,["headers"])])],512)):m("",!0)]),_:1})],4))}}),fs=k(vs,[["__scopeId","data-v-bc9dc845"]]),ms={class:"container"},hs=["aria-expanded"],_s={class:"menu-text"},bs=b({__name:"VPLocalNav",props:{open:{type:Boolean}},emits:["open-menu"],setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U(),{headers:n}=cs(),{y:r}=Me(),c=T(0);R(()=>{c.value=parseInt(getComputedStyle(document.documentElement).getPropertyValue("--vp-nav-height"))}),x(()=>{n.value=be(t.value.outline??e.value.outline)});const v=y(()=>n.value.length===0),d=y(()=>v.value&&!s.value),h=y(()=>({VPLocalNav:!0,"has-sidebar":s.value,empty:v.value,fixed:d.value}));return(P,_)=>i(t).layout!=="home"&&(!d.value||i(r)>=c.value)?(a(),l("div",{key:0,class:I(h.value)},[p("div",ms,[i(s)?(a(),l("button",{key:0,class:"menu","aria-expanded":P.open,"aria-controls":"VPSidebarNav",onClick:_[0]||(_[0]=V=>P.$emit("open-menu"))},[_[1]||(_[1]=p("span",{class:"vpi-align-left menu-icon"},null,-1)),p("span",_s,w(i(e).sidebarMenuLabel||"Menu"),1)],8,hs)):m("",!0),g(fs,{headers:i(n),navHeight:c.value},null,8,["headers","navHeight"])])],2)):m("",!0)}}),ks=k(bs,[["__scopeId","data-v-070ab83d"]]);function gs(){const o=T(!1);function e(){o.value=!0,window.addEventListener("resize",n)}function t(){o.value=!1,window.removeEventListener("resize",n)}function s(){o.value?t():e()}function n(){window.outerWidth>=768&&t()}const r=ee();return D(()=>r.path,t),{isScreenOpen:o,openScreen:e,closeScreen:t,toggleScreen:s}}const $s={},ys={class:"VPSwitch",type:"button",role:"switch"},Ps={class:"check"},Vs={key:0,class:"icon"};function Ss(o,e){return a(),l("button",ys,[p("span",Ps,[o.$slots.default?(a(),l("span",Vs,[u(o.$slots,"default",{},void 0,!0)])):m("",!0)])])}const Ls=k($s,[["render",Ss],["__scopeId","data-v-4a1c76db"]]),Ts=b({__name:"VPSwitchAppearance",setup(o){const{isDark:e,theme:t}=L(),s=W("toggle-appearance",()=>{e.value=!e.value}),n=T("");return me(()=>{n.value=e.value?t.value.lightModeSwitchTitle||"Switch to light theme":t.value.darkModeSwitchTitle||"Switch to dark theme"}),(r,c)=>(a(),$(Ls,{title:n.value,class:"VPSwitchAppearance","aria-checked":i(e),onClick:i(s)},{default:f(()=>c[0]||(c[0]=[p("span",{class:"vpi-sun sun"},null,-1),p("span",{class:"vpi-moon moon"},null,-1)])),_:1},8,["title","aria-checked","onClick"]))}}),ke=k(Ts,[["__scopeId","data-v-e40a8bb6"]]),ws={key:0,class:"VPNavBarAppearance"},Ns=b({__name:"VPNavBarAppearance",setup(o){const{site:e}=L();return(t,s)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",ws,[g(ke)])):m("",!0)}}),Is=k(Ns,[["__scopeId","data-v-af096f4a"]]),ge=T();let Oe=!1,re=0;function Ms(o){const e=T(!1);if(te){!Oe&&Cs(),re++;const t=D(ge,s=>{var n,r,c;s===o.el.value||(n=o.el.value)!=null&&n.contains(s)?(e.value=!0,(r=o.onFocus)==null||r.call(o)):(e.value=!1,(c=o.onBlur)==null||c.call(o))});fe(()=>{t(),re--,re||As()})}return Qe(e)}function Cs(){document.addEventListener("focusin",De),Oe=!0,ge.value=document.activeElement}function As(){document.removeEventListener("focusin",De)}function De(){ge.value=document.activeElement}const Bs={class:"VPMenuLink"},Es=["innerHTML"],Hs=b({__name:"VPMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),l("div",Bs,[g(H,{class:I({active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Es)]),_:1},8,["class","href","target","rel","no-icon"])]))}}),ne=k(Hs,[["__scopeId","data-v-acbfed09"]]),Os={class:"VPMenuGroup"},Ds={key:0,class:"title"},Fs=b({__name:"VPMenuGroup",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",Os,[e.text?(a(),l("p",Ds,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),l(M,null,["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):m("",!0)],64))),256))]))}}),Rs=k(Fs,[["__scopeId","data-v-48c802d0"]]),Us={class:"VPMenu"},zs={key:0,class:"items"},js=b({__name:"VPMenu",props:{items:{}},setup(o){return(e,t)=>(a(),l("div",Us,[e.items?(a(),l("div",zs,[(a(!0),l(M,null,B(e.items,s=>(a(),l(M,{key:JSON.stringify(s)},["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):"component"in s?(a(),$(O(s.component),j({key:1,ref_for:!0},s.props),null,16)):(a(),$(Rs,{key:2,text:s.text,items:s.items},null,8,["text","items"]))],64))),128))])):m("",!0),u(e.$slots,"default",{},void 0,!0)]))}}),Gs=k(js,[["__scopeId","data-v-7dd3104a"]]),Ks=["aria-expanded","aria-label"],qs={key:0,class:"text"},Ws=["innerHTML"],Js={key:1,class:"vpi-more-horizontal icon"},Ys={class:"menu"},Qs=b({__name:"VPFlyout",props:{icon:{},button:{},label:{},items:{}},setup(o){const e=T(!1),t=T();Ms({el:t,onBlur:s});function s(){e.value=!1}return(n,r)=>(a(),l("div",{class:"VPFlyout",ref_key:"el",ref:t,onMouseenter:r[1]||(r[1]=c=>e.value=!0),onMouseleave:r[2]||(r[2]=c=>e.value=!1)},[p("button",{type:"button",class:"button","aria-haspopup":"true","aria-expanded":e.value,"aria-label":n.label,onClick:r[0]||(r[0]=c=>e.value=!e.value)},[n.button||n.icon?(a(),l("span",qs,[n.icon?(a(),l("span",{key:0,class:I([n.icon,"option-icon"])},null,2)):m("",!0),n.button?(a(),l("span",{key:1,innerHTML:n.button},null,8,Ws)):m("",!0),r[3]||(r[3]=p("span",{class:"vpi-chevron-down text-icon"},null,-1))])):(a(),l("span",Js))],8,Ks),p("div",Ys,[g(Gs,{items:n.items},{default:f(()=>[u(n.$slots,"default",{},void 0,!0)]),_:3},8,["items"])])],544))}}),$e=k(Qs,[["__scopeId","data-v-04f5c5e9"]]),Xs=["href","aria-label","innerHTML"],Zs=b({__name:"VPSocialLink",props:{icon:{},link:{},ariaLabel:{}},setup(o){const e=o,t=y(()=>typeof e.icon=="object"?e.icon.svg:``);return(s,n)=>(a(),l("a",{class:"VPSocialLink no-icon",href:s.link,"aria-label":s.ariaLabel??(typeof s.icon=="string"?s.icon:""),target:"_blank",rel:"noopener",innerHTML:t.value},null,8,Xs))}}),xs=k(Zs,[["__scopeId","data-v-717b8b75"]]),eo={class:"VPSocialLinks"},to=b({__name:"VPSocialLinks",props:{links:{}},setup(o){return(e,t)=>(a(),l("div",eo,[(a(!0),l(M,null,B(e.links,({link:s,icon:n,ariaLabel:r})=>(a(),$(xs,{key:s,icon:n,link:s,ariaLabel:r},null,8,["icon","link","ariaLabel"]))),128))]))}}),se=k(to,[["__scopeId","data-v-ee7a9424"]]),no={key:0,class:"group translations"},so={class:"trans-title"},oo={key:1,class:"group"},ao={class:"item appearance"},ro={class:"label"},io={class:"appearance-action"},lo={key:2,class:"group"},co={class:"item social-links"},uo=b({__name:"VPNavBarExtra",setup(o){const{site:e,theme:t}=L(),{localeLinks:s,currentLang:n}=Y({correspondingLink:!0}),r=y(()=>s.value.length&&n.value.label||e.value.appearance||t.value.socialLinks);return(c,v)=>r.value?(a(),$($e,{key:0,class:"VPNavBarExtra",label:"extra navigation"},{default:f(()=>[i(s).length&&i(n).label?(a(),l("div",no,[p("p",so,w(i(n).label),1),(a(!0),l(M,null,B(i(s),d=>(a(),$(ne,{key:d.link,item:d},null,8,["item"]))),128))])):m("",!0),i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",oo,[p("div",ao,[p("p",ro,w(i(t).darkModeSwitchLabel||"Appearance"),1),p("div",io,[g(ke)])])])):m("",!0),i(t).socialLinks?(a(),l("div",lo,[p("div",co,[g(se,{class:"social-links-list",links:i(t).socialLinks},null,8,["links"])])])):m("",!0)]),_:1})):m("",!0)}}),po=k(uo,[["__scopeId","data-v-925effce"]]),vo=["aria-expanded"],fo=b({__name:"VPNavBarHamburger",props:{active:{type:Boolean}},emits:["click"],setup(o){return(e,t)=>(a(),l("button",{type:"button",class:I(["VPNavBarHamburger",{active:e.active}]),"aria-label":"mobile navigation","aria-expanded":e.active,"aria-controls":"VPNavScreen",onClick:t[0]||(t[0]=s=>e.$emit("click"))},t[1]||(t[1]=[p("span",{class:"container"},[p("span",{class:"top"}),p("span",{class:"middle"}),p("span",{class:"bottom"})],-1)]),10,vo))}}),mo=k(fo,[["__scopeId","data-v-5dea55bf"]]),ho=["innerHTML"],_o=b({__name:"VPNavBarMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),$(H,{class:I({VPNavBarMenuLink:!0,active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,tabindex:"0"},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ho)]),_:1},8,["class","href","target","rel","no-icon"]))}}),bo=k(_o,[["__scopeId","data-v-956ec74c"]]),Fe=b({__name:"VPNavBarMenuGroup",props:{item:{}},setup(o){const e=o,{page:t}=L(),s=r=>"component"in r?!1:"link"in r?G(t.value.relativePath,r.link,!!e.item.activeMatch):r.items.some(s),n=y(()=>s(e.item));return(r,c)=>(a(),$($e,{class:I({VPNavBarMenuGroup:!0,active:i(G)(i(t).relativePath,r.item.activeMatch,!!r.item.activeMatch)||n.value}),button:r.item.text,items:r.item.items},null,8,["class","button","items"]))}}),ko={key:0,"aria-labelledby":"main-nav-aria-label",class:"VPNavBarMenu"},go=b({__name:"VPNavBarMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",ko,[s[0]||(s[0]=p("span",{id:"main-nav-aria-label",class:"visually-hidden"}," Main Navigation ",-1)),(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(bo,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props),null,16)):(a(),$(Fe,{key:2,item:n},null,8,["item"]))],64))),128))])):m("",!0)}}),$o=k(go,[["__scopeId","data-v-e6d46098"]]);function yo(o){const{localeIndex:e,theme:t}=L();function s(n){var C,A,N;const r=n.split("."),c=(C=t.value.search)==null?void 0:C.options,v=c&&typeof c=="object",d=v&&((N=(A=c.locales)==null?void 0:A[e.value])==null?void 0:N.translations)||null,h=v&&c.translations||null;let P=d,_=h,V=o;const S=r.pop();for(const E of r){let z=null;const q=V==null?void 0:V[E];q&&(z=V=q);const oe=_==null?void 0:_[E];oe&&(z=_=oe);const ae=P==null?void 0:P[E];ae&&(z=P=ae),q||(V=z),oe||(_=z),ae||(P=z)}return(P==null?void 0:P[S])??(_==null?void 0:_[S])??(V==null?void 0:V[S])??""}return s}const Po=["aria-label"],Vo={class:"DocSearch-Button-Container"},So={class:"DocSearch-Button-Placeholder"},Pe=b({__name:"VPNavBarSearchButton",setup(o){const t=yo({button:{buttonText:"Search",buttonAriaLabel:"Search"}});return(s,n)=>(a(),l("button",{type:"button",class:"DocSearch DocSearch-Button","aria-label":i(t)("button.buttonAriaLabel")},[p("span",Vo,[n[0]||(n[0]=p("span",{class:"vp-icon DocSearch-Search-Icon"},null,-1)),p("span",So,w(i(t)("button.buttonText")),1)]),n[1]||(n[1]=p("span",{class:"DocSearch-Button-Keys"},[p("kbd",{class:"DocSearch-Button-Key"}),p("kbd",{class:"DocSearch-Button-Key"},"K")],-1))],8,Po))}}),Lo={class:"VPNavBarSearch"},To={id:"local-search"},wo={key:1,id:"docsearch"},No=b({__name:"VPNavBarSearch",setup(o){const e=Xe(()=>Ze(()=>import("./VPLocalSearchBox.omfuhOSb.js"),__vite__mapDeps([0,1]))),t=()=>null,{theme:s}=L(),n=T(!1),r=T(!1);R(()=>{});function c(){n.value||(n.value=!0,setTimeout(v,16))}function v(){const _=new Event("keydown");_.key="k",_.metaKey=!0,window.dispatchEvent(_),setTimeout(()=>{document.querySelector(".DocSearch-Modal")||v()},16)}function d(_){const V=_.target,S=V.tagName;return V.isContentEditable||S==="INPUT"||S==="SELECT"||S==="TEXTAREA"}const h=T(!1);le("k",_=>{(_.ctrlKey||_.metaKey)&&(_.preventDefault(),h.value=!0)}),le("/",_=>{d(_)||(_.preventDefault(),h.value=!0)});const P="local";return(_,V)=>{var S;return a(),l("div",Lo,[i(P)==="local"?(a(),l(M,{key:0},[h.value?(a(),$(i(e),{key:0,onClose:V[0]||(V[0]=C=>h.value=!1)})):m("",!0),p("div",To,[g(Pe,{onClick:V[1]||(V[1]=C=>h.value=!0)})])],64)):i(P)==="algolia"?(a(),l(M,{key:1},[n.value?(a(),$(i(t),{key:0,algolia:((S=i(s).search)==null?void 0:S.options)??i(s).algolia,onVnodeBeforeMount:V[2]||(V[2]=C=>r.value=!0)},null,8,["algolia"])):m("",!0),r.value?m("",!0):(a(),l("div",wo,[g(Pe,{onClick:c})]))],64)):m("",!0)])}}}),Io=b({__name:"VPNavBarSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavBarSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),Mo=k(Io,[["__scopeId","data-v-164c457f"]]),Co=["href","rel","target"],Ao=["innerHTML"],Bo={key:2},Eo=b({__name:"VPNavBarTitle",setup(o){const{site:e,theme:t}=L(),{hasSidebar:s}=U(),{currentLang:n}=Y(),r=y(()=>{var d;return typeof t.value.logoLink=="string"?t.value.logoLink:(d=t.value.logoLink)==null?void 0:d.link}),c=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.rel}),v=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.target});return(d,h)=>(a(),l("div",{class:I(["VPNavBarTitle",{"has-sidebar":i(s)}])},[p("a",{class:"title",href:r.value??i(_e)(i(n).link),rel:c.value,target:v.value},[u(d.$slots,"nav-bar-title-before",{},void 0,!0),i(t).logo?(a(),$(X,{key:0,class:"logo",image:i(t).logo},null,8,["image"])):m("",!0),i(t).siteTitle?(a(),l("span",{key:1,innerHTML:i(t).siteTitle},null,8,Ao)):i(t).siteTitle===void 0?(a(),l("span",Bo,w(i(e).title),1)):m("",!0),u(d.$slots,"nav-bar-title-after",{},void 0,!0)],8,Co)],2))}}),Ho=k(Eo,[["__scopeId","data-v-0f4f798b"]]),Oo={class:"items"},Do={class:"title"},Fo=b({__name:"VPNavBarTranslations",setup(o){const{theme:e}=L(),{localeLinks:t,currentLang:s}=Y({correspondingLink:!0});return(n,r)=>i(t).length&&i(s).label?(a(),$($e,{key:0,class:"VPNavBarTranslations",icon:"vpi-languages",label:i(e).langMenuLabel||"Change language"},{default:f(()=>[p("div",Oo,[p("p",Do,w(i(s).label),1),(a(!0),l(M,null,B(i(t),c=>(a(),$(ne,{key:c.link,item:c},null,8,["item"]))),128))])]),_:1},8,["label"])):m("",!0)}}),Ro=k(Fo,[["__scopeId","data-v-c80d9ad0"]]),Uo={class:"wrapper"},zo={class:"container"},jo={class:"title"},Go={class:"content"},Ko={class:"content-body"},qo=b({__name:"VPNavBar",props:{isScreenOpen:{type:Boolean}},emits:["toggle-screen"],setup(o){const e=o,{y:t}=Me(),{hasSidebar:s}=U(),{frontmatter:n}=L(),r=T({});return me(()=>{r.value={"has-sidebar":s.value,home:n.value.layout==="home",top:t.value===0,"screen-open":e.isScreenOpen}}),(c,v)=>(a(),l("div",{class:I(["VPNavBar",r.value])},[p("div",Uo,[p("div",zo,[p("div",jo,[g(Ho,null,{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),_:3})]),p("div",Go,[p("div",Ko,[u(c.$slots,"nav-bar-content-before",{},void 0,!0),g(No,{class:"search"}),g($o,{class:"menu"}),g(Ro,{class:"translations"}),g(Is,{class:"appearance"}),g(Mo,{class:"social-links"}),g(po,{class:"extra"}),u(c.$slots,"nav-bar-content-after",{},void 0,!0),g(mo,{class:"hamburger",active:c.isScreenOpen,onClick:v[0]||(v[0]=d=>c.$emit("toggle-screen"))},null,8,["active"])])])])]),v[1]||(v[1]=p("div",{class:"divider"},[p("div",{class:"divider-line"})],-1))],2))}}),Wo=k(qo,[["__scopeId","data-v-822684d1"]]),Jo={key:0,class:"VPNavScreenAppearance"},Yo={class:"text"},Qo=b({__name:"VPNavScreenAppearance",setup(o){const{site:e,theme:t}=L();return(s,n)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",Jo,[p("p",Yo,w(i(t).darkModeSwitchLabel||"Appearance"),1),g(ke)])):m("",!0)}}),Xo=k(Qo,[["__scopeId","data-v-ffb44008"]]),Zo=["innerHTML"],xo=b({__name:"VPNavScreenMenuLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Zo)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),ea=k(xo,[["__scopeId","data-v-735512b8"]]),ta=["innerHTML"],na=b({__name:"VPNavScreenMenuGroupLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuGroupLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ta)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),Re=k(na,[["__scopeId","data-v-372ae7c0"]]),sa={class:"VPNavScreenMenuGroupSection"},oa={key:0,class:"title"},aa=b({__name:"VPNavScreenMenuGroupSection",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",sa,[e.text?(a(),l("p",oa,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),$(Re,{key:s.text,item:s},null,8,["item"]))),128))]))}}),ra=k(aa,[["__scopeId","data-v-4b8941ac"]]),ia=["aria-controls","aria-expanded"],la=["innerHTML"],ca=["id"],ua={key:0,class:"item"},da={key:1,class:"item"},pa={key:2,class:"group"},va=b({__name:"VPNavScreenMenuGroup",props:{text:{},items:{}},setup(o){const e=o,t=T(!1),s=y(()=>`NavScreenGroup-${e.text.replace(" ","-").toLowerCase()}`);function n(){t.value=!t.value}return(r,c)=>(a(),l("div",{class:I(["VPNavScreenMenuGroup",{open:t.value}])},[p("button",{class:"button","aria-controls":s.value,"aria-expanded":t.value,onClick:n},[p("span",{class:"button-text",innerHTML:r.text},null,8,la),c[0]||(c[0]=p("span",{class:"vpi-plus button-icon"},null,-1))],8,ia),p("div",{id:s.value,class:"items"},[(a(!0),l(M,null,B(r.items,v=>(a(),l(M,{key:JSON.stringify(v)},["link"in v?(a(),l("div",ua,[g(Re,{item:v},null,8,["item"])])):"component"in v?(a(),l("div",da,[(a(),$(O(v.component),j({ref_for:!0},v.props,{"screen-menu":""}),null,16))])):(a(),l("div",pa,[g(ra,{text:v.text,items:v.items},null,8,["text","items"])]))],64))),128))],8,ca)],2))}}),Ue=k(va,[["__scopeId","data-v-875057a5"]]),fa={key:0,class:"VPNavScreenMenu"},ma=b({__name:"VPNavScreenMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",fa,[(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(ea,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props,{"screen-menu":""}),null,16)):(a(),$(Ue,{key:2,text:n.text||"",items:n.items},null,8,["text","items"]))],64))),128))])):m("",!0)}}),ha=b({__name:"VPNavScreenSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavScreenSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),_a={class:"list"},ba=b({__name:"VPNavScreenTranslations",setup(o){const{localeLinks:e,currentLang:t}=Y({correspondingLink:!0}),s=T(!1);function n(){s.value=!s.value}return(r,c)=>i(e).length&&i(t).label?(a(),l("div",{key:0,class:I(["VPNavScreenTranslations",{open:s.value}])},[p("button",{class:"title",onClick:n},[c[0]||(c[0]=p("span",{class:"vpi-languages icon lang"},null,-1)),F(" "+w(i(t).label)+" ",1),c[1]||(c[1]=p("span",{class:"vpi-chevron-down icon chevron"},null,-1))]),p("ul",_a,[(a(!0),l(M,null,B(i(e),v=>(a(),l("li",{key:v.link,class:"item"},[g(H,{class:"link",href:v.link},{default:f(()=>[F(w(v.text),1)]),_:2},1032,["href"])]))),128))])],2)):m("",!0)}}),ka=k(ba,[["__scopeId","data-v-362991c2"]]),ga={class:"container"},$a=b({__name:"VPNavScreen",props:{open:{type:Boolean}},setup(o){const e=T(null),t=Ce(te?document.body:null);return(s,n)=>(a(),$(pe,{name:"fade",onEnter:n[0]||(n[0]=r=>t.value=!0),onAfterLeave:n[1]||(n[1]=r=>t.value=!1)},{default:f(()=>[s.open?(a(),l("div",{key:0,class:"VPNavScreen",ref_key:"screen",ref:e,id:"VPNavScreen"},[p("div",ga,[u(s.$slots,"nav-screen-content-before",{},void 0,!0),g(ma,{class:"menu"}),g(ka,{class:"translations"}),g(Xo,{class:"appearance"}),g(ha,{class:"social-links"}),u(s.$slots,"nav-screen-content-after",{},void 0,!0)])],512)):m("",!0)]),_:3}))}}),ya=k($a,[["__scopeId","data-v-833aabba"]]),Pa={key:0,class:"VPNav"},Va=b({__name:"VPNav",setup(o){const{isScreenOpen:e,closeScreen:t,toggleScreen:s}=gs(),{frontmatter:n}=L(),r=y(()=>n.value.navbar!==!1);return he("close-screen",t),Z(()=>{te&&document.documentElement.classList.toggle("hide-nav",!r.value)}),(c,v)=>r.value?(a(),l("header",Pa,[g(Wo,{"is-screen-open":i(e),onToggleScreen:i(s)},{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(c.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(c.$slots,"nav-bar-content-after",{},void 0,!0)]),_:3},8,["is-screen-open","onToggleScreen"]),g(ya,{open:i(e)},{"nav-screen-content-before":f(()=>[u(c.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(c.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3},8,["open"])])):m("",!0)}}),Sa=k(Va,[["__scopeId","data-v-f1e365da"]]),La=["role","tabindex"],Ta={key:1,class:"items"},wa=b({__name:"VPSidebarItem",props:{item:{},depth:{}},setup(o){const e=o,{collapsed:t,collapsible:s,isLink:n,isActiveLink:r,hasActiveLink:c,hasChildren:v,toggle:d}=Pt(y(()=>e.item)),h=y(()=>v.value?"section":"div"),P=y(()=>n.value?"a":"div"),_=y(()=>v.value?e.depth+2===7?"p":`h${e.depth+2}`:"p"),V=y(()=>n.value?void 0:"button"),S=y(()=>[[`level-${e.depth}`],{collapsible:s.value},{collapsed:t.value},{"is-link":n.value},{"is-active":r.value},{"has-active":c.value}]);function C(N){"key"in N&&N.key!=="Enter"||!e.item.link&&d()}function A(){e.item.link&&d()}return(N,E)=>{const z=K("VPSidebarItem",!0);return a(),$(O(h.value),{class:I(["VPSidebarItem",S.value])},{default:f(()=>[N.item.text?(a(),l("div",j({key:0,class:"item",role:V.value},et(N.item.items?{click:C,keydown:C}:{},!0),{tabindex:N.item.items&&0}),[E[1]||(E[1]=p("div",{class:"indicator"},null,-1)),N.item.link?(a(),$(H,{key:0,tag:P.value,class:"link",href:N.item.link,rel:N.item.rel,target:N.item.target},{default:f(()=>[(a(),$(O(_.value),{class:"text",innerHTML:N.item.text},null,8,["innerHTML"]))]),_:1},8,["tag","href","rel","target"])):(a(),$(O(_.value),{key:1,class:"text",innerHTML:N.item.text},null,8,["innerHTML"])),N.item.collapsed!=null&&N.item.items&&N.item.items.length?(a(),l("div",{key:2,class:"caret",role:"button","aria-label":"toggle section",onClick:A,onKeydown:xe(A,["enter"]),tabindex:"0"},E[0]||(E[0]=[p("span",{class:"vpi-chevron-right caret-icon"},null,-1)]),32)):m("",!0)],16,La)):m("",!0),N.item.items&&N.item.items.length?(a(),l("div",Ta,[N.depth<5?(a(!0),l(M,{key:0},B(N.item.items,q=>(a(),$(z,{key:q.text,item:q,depth:N.depth+1},null,8,["item","depth"]))),128)):m("",!0)])):m("",!0)]),_:1},8,["class"])}}}),Na=k(wa,[["__scopeId","data-v-196b2e5f"]]),Ia=b({__name:"VPSidebarGroup",props:{items:{}},setup(o){const e=T(!0);let t=null;return R(()=>{t=setTimeout(()=>{t=null,e.value=!1},300)}),tt(()=>{t!=null&&(clearTimeout(t),t=null)}),(s,n)=>(a(!0),l(M,null,B(s.items,r=>(a(),l("div",{key:r.text,class:I(["group",{"no-transition":e.value}])},[g(Na,{item:r,depth:0},null,8,["item"])],2))),128))}}),Ma=k(Ia,[["__scopeId","data-v-9e426adc"]]),Ca={class:"nav",id:"VPSidebarNav","aria-labelledby":"sidebar-aria-label",tabindex:"-1"},Aa=b({__name:"VPSidebar",props:{open:{type:Boolean}},setup(o){const{sidebarGroups:e,hasSidebar:t}=U(),s=o,n=T(null),r=Ce(te?document.body:null);D([s,n],()=>{var v;s.open?(r.value=!0,(v=n.value)==null||v.focus()):r.value=!1},{immediate:!0,flush:"post"});const c=T(0);return D(e,()=>{c.value+=1},{deep:!0}),(v,d)=>i(t)?(a(),l("aside",{key:0,class:I(["VPSidebar",{open:v.open}]),ref_key:"navEl",ref:n,onClick:d[0]||(d[0]=nt(()=>{},["stop"]))},[d[2]||(d[2]=p("div",{class:"curtain"},null,-1)),p("nav",Ca,[d[1]||(d[1]=p("span",{class:"visually-hidden",id:"sidebar-aria-label"}," Sidebar Navigation ",-1)),u(v.$slots,"sidebar-nav-before",{},void 0,!0),(a(),$(Ma,{items:i(e),key:c.value},null,8,["items"])),u(v.$slots,"sidebar-nav-after",{},void 0,!0)])],2)):m("",!0)}}),Ba=k(Aa,[["__scopeId","data-v-18756405"]]),Ea=b({__name:"VPSkipLink",setup(o){const e=ee(),t=T();D(()=>e.path,()=>t.value.focus());function s({target:n}){const r=document.getElementById(decodeURIComponent(n.hash).slice(1));if(r){const c=()=>{r.removeAttribute("tabindex"),r.removeEventListener("blur",c)};r.setAttribute("tabindex","-1"),r.addEventListener("blur",c),r.focus(),window.scrollTo(0,0)}}return(n,r)=>(a(),l(M,null,[p("span",{ref_key:"backToTop",ref:t,tabindex:"-1"},null,512),p("a",{href:"#VPContent",class:"VPSkipLink visually-hidden",onClick:s}," Skip to content ")],64))}}),Ha=k(Ea,[["__scopeId","data-v-c3508ec8"]]),Oa=b({__name:"Layout",setup(o){const{isOpen:e,open:t,close:s}=U(),n=ee();D(()=>n.path,s),yt(e,s);const{frontmatter:r}=L(),c=Ae(),v=y(()=>!!c["home-hero-image"]);return he("hero-image-slot-exists",v),(d,h)=>{const P=K("Content");return i(r).layout!==!1?(a(),l("div",{key:0,class:I(["Layout",i(r).pageClass])},[u(d.$slots,"layout-top",{},void 0,!0),g(Ha),g(ct,{class:"backdrop",show:i(e),onClick:i(s)},null,8,["show","onClick"]),g(Sa,null,{"nav-bar-title-before":f(()=>[u(d.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(d.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(d.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(d.$slots,"nav-bar-content-after",{},void 0,!0)]),"nav-screen-content-before":f(()=>[u(d.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(d.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3}),g(ks,{open:i(e),onOpenMenu:i(t)},null,8,["open","onOpenMenu"]),g(Ba,{open:i(e)},{"sidebar-nav-before":f(()=>[u(d.$slots,"sidebar-nav-before",{},void 0,!0)]),"sidebar-nav-after":f(()=>[u(d.$slots,"sidebar-nav-after",{},void 0,!0)]),_:3},8,["open"]),g(ss,null,{"page-top":f(()=>[u(d.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(d.$slots,"page-bottom",{},void 0,!0)]),"not-found":f(()=>[u(d.$slots,"not-found",{},void 0,!0)]),"home-hero-before":f(()=>[u(d.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(d.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(d.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(d.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(d.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(d.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(d.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(d.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(d.$slots,"home-features-after",{},void 0,!0)]),"doc-footer-before":f(()=>[u(d.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(d.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(d.$slots,"doc-after",{},void 0,!0)]),"doc-top":f(()=>[u(d.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(d.$slots,"doc-bottom",{},void 0,!0)]),"aside-top":f(()=>[u(d.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(d.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(d.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(d.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(d.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(d.$slots,"aside-ads-after",{},void 0,!0)]),_:3}),g(ls),u(d.$slots,"layout-bottom",{},void 0,!0)],2)):(a(),$(P,{key:1}))}}}),Da=k(Oa,[["__scopeId","data-v-a9a9e638"]]),Fa={},Ra={class:"VPTeamPage"};function Ua(o,e){return a(),l("div",Ra,[u(o.$slots,"default")])}const Or=k(Fa,[["render",Ua],["__scopeId","data-v-c2f8e101"]]),za={},ja={class:"VPTeamPageTitle"},Ga={key:0,class:"title"},Ka={key:1,class:"lead"};function qa(o,e){return a(),l("div",ja,[o.$slots.title?(a(),l("h1",Ga,[u(o.$slots,"title",{},void 0,!0)])):m("",!0),o.$slots.lead?(a(),l("p",Ka,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0)])}const Dr=k(za,[["render",qa],["__scopeId","data-v-e277e15c"]]),Wa={},Ja={class:"VPTeamPageSection"},Ya={class:"title"},Qa={key:0,class:"title-text"},Xa={key:0,class:"lead"},Za={key:1,class:"members"};function xa(o,e){return a(),l("section",Ja,[p("div",Ya,[e[0]||(e[0]=p("div",{class:"title-line"},null,-1)),o.$slots.title?(a(),l("h2",Qa,[u(o.$slots,"title",{},void 0,!0)])):m("",!0)]),o.$slots.lead?(a(),l("p",Xa,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0),o.$slots.members?(a(),l("div",Za,[u(o.$slots,"members",{},void 0,!0)])):m("",!0)])}const Fr=k(Wa,[["render",xa],["__scopeId","data-v-d43bc49d"]]),er={class:"profile"},tr={class:"avatar"},nr=["src","alt"],sr={class:"data"},or={class:"name"},ar={key:0,class:"affiliation"},rr={key:0,class:"title"},ir={key:1,class:"at"},lr=["innerHTML"],cr={key:2,class:"links"},ur={key:0,class:"sp"},dr=b({__name:"VPTeamMembersItem",props:{size:{default:"medium"},member:{}},setup(o){return(e,t)=>(a(),l("article",{class:I(["VPTeamMembersItem",[e.size]])},[p("div",er,[p("figure",tr,[p("img",{class:"avatar-img",src:e.member.avatar,alt:e.member.name},null,8,nr)]),p("div",sr,[p("h1",or,w(e.member.name),1),e.member.title||e.member.org?(a(),l("p",ar,[e.member.title?(a(),l("span",rr,w(e.member.title),1)):m("",!0),e.member.title&&e.member.org?(a(),l("span",ir," @ ")):m("",!0),e.member.org?(a(),$(H,{key:2,class:I(["org",{link:e.member.orgLink}]),href:e.member.orgLink,"no-icon":""},{default:f(()=>[F(w(e.member.org),1)]),_:1},8,["class","href"])):m("",!0)])):m("",!0),e.member.desc?(a(),l("p",{key:1,class:"desc",innerHTML:e.member.desc},null,8,lr)):m("",!0),e.member.links?(a(),l("div",cr,[g(se,{links:e.member.links},null,8,["links"])])):m("",!0)])]),e.member.sponsor?(a(),l("div",ur,[g(H,{class:"sp-link",href:e.member.sponsor,"no-icon":""},{default:f(()=>[t[0]||(t[0]=p("span",{class:"vpi-heart sp-icon"},null,-1)),F(" "+w(e.member.actionText||"Sponsor"),1)]),_:1},8,["href"])])):m("",!0)],2))}}),pr=k(dr,[["__scopeId","data-v-f9987cb6"]]),vr={class:"container"},fr=b({__name:"VPTeamMembers",props:{size:{default:"medium"},members:{}},setup(o){const e=o,t=y(()=>[e.size,`count-${e.members.length}`]);return(s,n)=>(a(),l("div",{class:I(["VPTeamMembers",t.value])},[p("div",vr,[(a(!0),l(M,null,B(s.members,r=>(a(),l("div",{key:r.name,class:"item"},[g(pr,{size:s.size,member:r},null,8,["size","member"])]))),128))])],2))}}),Rr=k(fr,[["__scopeId","data-v-fba19bad"]]),Ve={Layout:Da,enhanceApp:({app:o})=>{o.component("Badge",rt)}},mr={};function hr(o,e){return e[0]||(e[0]=st('

    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    ',3))}const _r=k(mr,[["render",hr]]),br=b({__name:"VersionPicker",props:{screenMenu:{type:Boolean}},setup(o){const e=T([]),t=T("Versions"),s=T(!1);Le();const n=()=>typeof window<"u"&&(window.location.hostname==="localhost"||window.location.hostname==="127.0.0.1"),r=()=>{if(typeof window>"u")return"";const{origin:d,pathname:h}=window.location;if(d.includes("github.io")){const P=h.split("/").filter(Boolean),_=P.length>0?`/${P[0]}/`:"/";return`${d}${_}`}else return d},c=()=>new Promise(d=>{if(n()){d(!1);return}const h=setInterval(()=>{window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION&&(clearInterval(h),d(!0))},100);setTimeout(()=>{clearInterval(h),d(!1)},5e3)});return R(async()=>{if(!(typeof window>"u")){try{if(n()){const d=["dev"];e.value=d.map(h=>({text:h,link:"/"})),t.value="dev"}else{const d=await c(),h=y(()=>r());if(d&&window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION)e.value=window.DOC_VERSIONS.map(P=>({text:P,link:`${h.value}/${P}/`})),t.value=window.DOCUMENTER_CURRENT_VERSION;else{const P=["dev"];e.value=P.map(_=>({text:_,link:`${h.value}/${_}/`})),t.value="dev"}}}catch(d){console.warn("Error loading versions:",d);const h=["dev"],P=y(()=>r());e.value=h.map(_=>({text:_,link:`${P.value}/${_}/`})),t.value="dev"}s.value=!0}}),(d,h)=>s.value?(a(),l(M,{key:0},[!d.screenMenu&&e.value.length>0?(a(),$(Fe,{key:0,item:{text:t.value,items:e.value},class:"VPVersionPicker"},null,8,["item"])):d.screenMenu&&e.value.length>0?(a(),$(Ue,{key:1,text:t.value,items:e.value,class:"VPVersionPicker"},null,8,["text","items"])):m("",!0)],64)):m("",!0)}}),kr=k(br,[["__scopeId","data-v-d483b3a6"]]),gr=o=>{if(typeof document>"u")return{stabilizeScrollPosition:n=>async(...r)=>n(...r)};const e=document.documentElement;return{stabilizeScrollPosition:s=>async(...n)=>{const r=s(...n),c=o.value;if(!c)return r;const v=c.offsetTop-e.scrollTop;return await Ie(),e.scrollTop=c.offsetTop-v,r}}},ze="vitepress:tabSharedState",J=typeof localStorage<"u"?localStorage:null,je="vitepress:tabsSharedState",$r=()=>{const o=J==null?void 0:J.getItem(je);if(o)try{return JSON.parse(o)}catch{}return{}},yr=o=>{J&&J.setItem(je,JSON.stringify(o))},Pr=o=>{const e=ot({});D(()=>e.content,(t,s)=>{t&&s&&yr(t)},{deep:!0}),o.provide(ze,e)},Vr=(o,e)=>{const t=W(ze);if(!t)throw new Error("[vitepress-plugin-tabs] TabsSharedState should be injected");R(()=>{t.content||(t.content=$r())});const s=T(),n=y({get(){var d;const c=e.value,v=o.value;if(c){const h=(d=t.content)==null?void 0:d[c];if(h&&v.includes(h))return h}else{const h=s.value;if(h)return h}return v[0]},set(c){const v=e.value;v?t.content&&(t.content[v]=c):s.value=c}});return{selected:n,select:c=>{n.value=c}}};let Se=0;const Sr=()=>(Se++,""+Se);function Lr(){const o=Ae();return y(()=>{var s;const t=(s=o.default)==null?void 0:s.call(o);return t?t.filter(n=>typeof n.type=="object"&&"__name"in n.type&&n.type.__name==="PluginTabsTab"&&n.props).map(n=>{var r;return(r=n.props)==null?void 0:r.label}):[]})}const Ge="vitepress:tabSingleState",Tr=o=>{he(Ge,o)},wr=()=>{const o=W(Ge);if(!o)throw new Error("[vitepress-plugin-tabs] TabsSingleState should be injected");return o},Nr={class:"plugin-tabs"},Ir=["id","aria-selected","aria-controls","tabindex","onClick"],Mr=b({__name:"PluginTabs",props:{sharedStateKey:{}},setup(o){const e=o,t=Lr(),{selected:s,select:n}=Vr(t,at(e,"sharedStateKey")),r=T(),{stabilizeScrollPosition:c}=gr(r),v=c(n),d=T([]),h=_=>{var C;const V=t.value.indexOf(s.value);let S;_.key==="ArrowLeft"?S=V>=1?V-1:t.value.length-1:_.key==="ArrowRight"&&(S=V(a(),l("div",Nr,[p("div",{ref_key:"tablist",ref:r,class:"plugin-tabs--tab-list",role:"tablist",onKeydown:h},[(a(!0),l(M,null,B(i(t),S=>(a(),l("button",{id:`tab-${S}-${i(P)}`,ref_for:!0,ref_key:"buttonRefs",ref:d,key:S,role:"tab",class:"plugin-tabs--tab","aria-selected":S===i(s),"aria-controls":`panel-${S}-${i(P)}`,tabindex:S===i(s)?0:-1,onClick:()=>i(v)(S)},w(S),9,Ir))),128))],544),u(_.$slots,"default")]))}}),Cr=["id","aria-labelledby"],Ar=b({__name:"PluginTabsTab",props:{label:{}},setup(o){const{uid:e,selected:t}=wr();return(s,n)=>i(t)===s.label?(a(),l("div",{key:0,id:`panel-${s.label}-${i(e)}`,class:"plugin-tabs--content",role:"tabpanel",tabindex:"0","aria-labelledby":`tab-${s.label}-${i(e)}`},[u(s.$slots,"default",{},void 0,!0)],8,Cr)):m("",!0)}}),Br=k(Ar,[["__scopeId","data-v-9b0d03d2"]]),Er=o=>{Pr(o),o.component("PluginTabs",Mr),o.component("PluginTabsTab",Br)},Ur={extends:Ve,Layout(){return ye(Ve.Layout,null,{"aside-ads-before":()=>ye(_r)})},enhanceApp({app:o}){Er(o),o.component("VersionPicker",kr)}};export{Ur as R,Dr as V,Fr as a,Rr as b,Or as c,yo as d,L as u}; diff --git a/dev/assets/ecosystem.md.BywRar4V.js b/dev/assets/ecosystem.md.BQKjQ0KY.js similarity index 99% rename from dev/assets/ecosystem.md.BywRar4V.js rename to dev/assets/ecosystem.md.BQKjQ0KY.js index 07d938d38d..326118c53e 100644 --- a/dev/assets/ecosystem.md.BywRar4V.js +++ b/dev/assets/ecosystem.md.BQKjQ0KY.js @@ -1 +1 @@ -import{V as d,a as s,b as l,c as f}from"./chunks/theme.C74glUlb.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/LuxTestUtils.jl"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; +import{V as d,a as s,b as l,c as f}from"./chunks/theme.DgYiELuP.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/LuxTestUtils.jl"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; diff --git a/dev/assets/ecosystem.md.BywRar4V.lean.js b/dev/assets/ecosystem.md.BQKjQ0KY.lean.js similarity index 99% rename from dev/assets/ecosystem.md.BywRar4V.lean.js rename to dev/assets/ecosystem.md.BQKjQ0KY.lean.js index 07d938d38d..326118c53e 100644 --- a/dev/assets/ecosystem.md.BywRar4V.lean.js +++ b/dev/assets/ecosystem.md.BQKjQ0KY.lean.js @@ -1 +1 @@ -import{V as d,a as s,b as l,c as f}from"./chunks/theme.C74glUlb.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/LuxTestUtils.jl"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; +import{V as d,a as s,b as l,c as f}from"./chunks/theme.DgYiELuP.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/LuxTestUtils.jl"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; diff --git a/dev/assets/index.md.B5rDYBAv.js b/dev/assets/index.md.kmP_n3ia.js similarity index 92% rename from dev/assets/index.md.B5rDYBAv.js rename to dev/assets/index.md.kmP_n3ia.js index ac753390f5..148b490ead 100644 --- a/dev/assets/index.md.B5rDYBAv.js +++ b/dev/assets/index.md.kmP_n3ia.js @@ -1,12 +1,12 @@ -import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const g=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),l={name:"index.md"};function e(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`

    How to Install Lux.jl? ​

    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:

    julia
    julia> using Pkg
+import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const o=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),l={name:"index.md"};function e(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`
    How to Install Lux.jl? ​
    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:
    julia
    julia> using Pkg
 julia> Pkg.add("Lux")
    If you want to use the latest unreleased version of Lux.jl, you can run the following command: (in most cases the released version will be same as the version on github)
    julia
    julia> using Pkg
-julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
+julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
 Pkg.add("LuxCUDA")
 # or
 Pkg.add(["CUDA", "cuDNN"])
    julia
    using Pkg
 Pkg.add("AMDGPU")
    julia
    using Pkg
 Pkg.add("Metal")
    julia
    using Pkg
-Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
+Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
 
 const dev = gpu_device()
    julia
    using Lux, AMDGPU
 
@@ -15,7 +15,7 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const g
 const dev = gpu_device()
    julia
    using Lux, oneAPI
 
 const dev = gpu_device()
    Want XLA Support? ​
    Install the following package:
    julia
    using Pkg;
-Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
+Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
 Reactant.set_default_backend("cpu") # default
 
 const dev = xla_device()
    julia
    using Reactant, Lux
@@ -24,4 +24,4 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const g
 const dev = xla_device()
    julia
    using Reactant, Lux
 Reactant.set_default_backend("tpu")
 
-const dev = xla_device()
    `,15)]))}const E=i(l,[["render",e]]);export{g as __pageData,E as default};
+const dev = xla_device()
    `,15)]))}const E=i(l,[["render",e]]);export{o as __pageData,E as default}; diff --git a/dev/assets/index.md.B5rDYBAv.lean.js b/dev/assets/index.md.kmP_n3ia.lean.js similarity index 92% rename from dev/assets/index.md.B5rDYBAv.lean.js rename to dev/assets/index.md.kmP_n3ia.lean.js index ac753390f5..148b490ead 100644 --- a/dev/assets/index.md.B5rDYBAv.lean.js +++ b/dev/assets/index.md.kmP_n3ia.lean.js @@ -1,12 +1,12 @@ -import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const g=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),l={name:"index.md"};function e(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`

    How to Install Lux.jl? ​

    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:

    julia
    julia> using Pkg
+import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const o=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),l={name:"index.md"};function e(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`
    How to Install Lux.jl? ​
    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:
    julia
    julia> using Pkg
 julia> Pkg.add("Lux")
    If you want to use the latest unreleased version of Lux.jl, you can run the following command: (in most cases the released version will be same as the version on github)
    julia
    julia> using Pkg
-julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
+julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
 Pkg.add("LuxCUDA")
 # or
 Pkg.add(["CUDA", "cuDNN"])
    julia
    using Pkg
 Pkg.add("AMDGPU")
    julia
    using Pkg
 Pkg.add("Metal")
    julia
    using Pkg
-Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
+Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
 
 const dev = gpu_device()
    julia
    using Lux, AMDGPU
 
@@ -15,7 +15,7 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const g
 const dev = gpu_device()
    julia
    using Lux, oneAPI
 
 const dev = gpu_device()
    Want XLA Support? ​
    Install the following package:
    julia
    using Pkg;
-Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
+Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
 Reactant.set_default_backend("cpu") # default
 
 const dev = xla_device()
    julia
    using Reactant, Lux
@@ -24,4 +24,4 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.CCjWn1F9.js";const g
 const dev = xla_device()
    julia
    using Reactant, Lux
 Reactant.set_default_backend("tpu")
 
-const dev = xla_device()
    `,15)]))}const E=i(l,[["render",e]]);export{g as __pageData,E as default};
+const dev = xla_device()
    `,15)]))}const E=i(l,[["render",e]]);export{o as __pageData,E as default}; diff --git a/dev/assets/manual_debugging.md.DrOcMYR1.js b/dev/assets/manual_debugging.md.C1VsyBGc.js similarity index 99% rename from dev/assets/manual_debugging.md.DrOcMYR1.js rename to dev/assets/manual_debugging.md.C1VsyBGc.js index b531ed970e..33f4f3767b 100644 --- a/dev/assets/manual_debugging.md.DrOcMYR1.js +++ b/dev/assets/manual_debugging.md.C1VsyBGc.js @@ -40,7 +40,7 @@ import{_ as a,c as i,a2 as n,o as e}from"./chunks/framework.CCjWn1F9.js";const g [ Info: Input Type: Matrix{Float32} | Input Structure: (3, 2). [ Info: Running Layer: Dense(1 => 1) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2)! ┌ Error: Layer Dense(1 => 1) failed!! This layer is present at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2). -└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/src/contrib/debug.jl:102 +└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-11/julialang/lux-dot-jl/src/contrib/debug.jl:102 DimensionMismatch("A has shape (1, 1) but B has shape (3, 2)")

    See now we know that model.layers.layer_2.layers.layer_2 is the problematic layer. Let us fix that layer and see what happens:

    julia
    model = Chain(Dense(1 => 16, relu),
     Chain(
         Dense(16 => 3),  
diff --git a/dev/assets/manual_debugging.md.DrOcMYR1.lean.js b/dev/assets/manual_debugging.md.C1VsyBGc.lean.js
similarity index 99%
rename from dev/assets/manual_debugging.md.DrOcMYR1.lean.js
rename to dev/assets/manual_debugging.md.C1VsyBGc.lean.js
index b531ed970e..33f4f3767b 100644
--- a/dev/assets/manual_debugging.md.DrOcMYR1.lean.js
+++ b/dev/assets/manual_debugging.md.C1VsyBGc.lean.js
@@ -40,7 +40,7 @@ import{_ as a,c as i,a2 as n,o as e}from"./chunks/framework.CCjWn1F9.js";const g
 [ Info: Input Type: Matrix{Float32} | Input Structure: (3, 2).
 [ Info: Running Layer: Dense(1 => 1) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2)!
 ┌ Error: Layer Dense(1 => 1) failed!! This layer is present at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2).
-└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/src/contrib/debug.jl:102
+└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-11/julialang/lux-dot-jl/src/contrib/debug.jl:102
 DimensionMismatch("A has shape (1, 1) but B has shape (3, 2)")

    See now we know that model.layers.layer_2.layers.layer_2 is the problematic layer. Let us fix that layer and see what happens:

    julia
    model = Chain(Dense(1 => 16, relu),
     Chain(
         Dense(16 => 3),  
diff --git a/dev/assets/manual_freezing_model_parameters.md.DzvhQn_f.js b/dev/assets/manual_freezing_model_parameters.md.v9KBWx0y.js
similarity index 97%
rename from dev/assets/manual_freezing_model_parameters.md.DzvhQn_f.js
rename to dev/assets/manual_freezing_model_parameters.md.v9KBWx0y.js
index 59aa5c1e26..2cc4f0ef2a 100644
--- a/dev/assets/manual_freezing_model_parameters.md.DzvhQn_f.js
+++ b/dev/assets/manual_freezing_model_parameters.md.v9KBWx0y.js
@@ -17,7 +17,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.CCjWn1F9.js";const g
 
 model_frozen, ps_frozen, st_frozen = Lux.Experimental.layer_map(freeze_dense, model, ps, st)
 
-model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
+model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
@@ -26,7 +26,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.CCjWn1F9.js";const g
 function freeze_dense(d::Dense, ps, st, _)
     return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
 end
-freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
+freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight,))
diff --git a/dev/assets/manual_freezing_model_parameters.md.DzvhQn_f.lean.js b/dev/assets/manual_freezing_model_parameters.md.v9KBWx0y.lean.js
similarity index 97%
rename from dev/assets/manual_freezing_model_parameters.md.DzvhQn_f.lean.js
rename to dev/assets/manual_freezing_model_parameters.md.v9KBWx0y.lean.js
index 59aa5c1e26..2cc4f0ef2a 100644
--- a/dev/assets/manual_freezing_model_parameters.md.DzvhQn_f.lean.js
+++ b/dev/assets/manual_freezing_model_parameters.md.v9KBWx0y.lean.js
@@ -17,7 +17,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.CCjWn1F9.js";const g
 
 model_frozen, ps_frozen, st_frozen = Lux.Experimental.layer_map(freeze_dense, model, ps, st)
 
-model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
+model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
@@ -26,7 +26,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.CCjWn1F9.js";const g
 function freeze_dense(d::Dense, ps, st, _)
     return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
 end
-freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
+freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight,))
diff --git a/dev/assets/manual_migrate_from_flux.md.8SF0xKdw.js b/dev/assets/manual_migrate_from_flux.md.Dgfpn4GK.js
similarity index 97%
rename from dev/assets/manual_migrate_from_flux.md.8SF0xKdw.js
rename to dev/assets/manual_migrate_from_flux.md.Dgfpn4GK.js
index 3205671d6b..a742bc1997 100644
--- a/dev/assets/manual_migrate_from_flux.md.8SF0xKdw.js
+++ b/dev/assets/manual_migrate_from_flux.md.Dgfpn4GK.js
@@ -1,4 +1,4 @@
-import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
+import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
 
 model = Chain(Dense(2 => 4), BatchNorm(4, relu), Dense(4 => 2))
 rng = Random.default_rng()
@@ -18,7 +18,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1
 
 model(x)
 
-gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
+gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
 
 struct LuxLinear <: Lux.AbstractLuxLayer
     init_A
@@ -56,7 +56,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1
 # Needed so that both \`A\` and \`B\` can be transferred between devices
 Flux.@functor FluxLinear
 
-(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
+(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
 model = LuxLinear(randn(rng, 2, 4), randn(rng, 4, 2))
 x = randn(rng, 2, 1)
 
diff --git a/dev/assets/manual_migrate_from_flux.md.8SF0xKdw.lean.js b/dev/assets/manual_migrate_from_flux.md.Dgfpn4GK.lean.js
similarity index 97%
rename from dev/assets/manual_migrate_from_flux.md.8SF0xKdw.lean.js
rename to dev/assets/manual_migrate_from_flux.md.Dgfpn4GK.lean.js
index 3205671d6b..a742bc1997 100644
--- a/dev/assets/manual_migrate_from_flux.md.8SF0xKdw.lean.js
+++ b/dev/assets/manual_migrate_from_flux.md.Dgfpn4GK.lean.js
@@ -1,4 +1,4 @@
-import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
+import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1F9.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
 
 model = Chain(Dense(2 => 4), BatchNorm(4, relu), Dense(4 => 2))
 rng = Random.default_rng()
@@ -18,7 +18,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1
 
 model(x)
 
-gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
+gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
 
 struct LuxLinear <: Lux.AbstractLuxLayer
     init_A
@@ -56,7 +56,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.CCjWn1
 # Needed so that both \`A\` and \`B\` can be transferred between devices
 Flux.@functor FluxLinear
 
-(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
+(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
 model = LuxLinear(randn(rng, 2, 4), randn(rng, 4, 2))
 x = randn(rng, 2, 1)
 
diff --git a/dev/assets/manual_weight_initializers.md.CwPu40d0.js b/dev/assets/manual_weight_initializers.md.BDecRyMi.js
similarity index 81%
rename from dev/assets/manual_weight_initializers.md.CwPu40d0.js
rename to dev/assets/manual_weight_initializers.md.BDecRyMi.js
index 72bff49acd..aa8b998ec4 100644
--- a/dev/assets/manual_weight_initializers.md.CwPu40d0.js
+++ b/dev/assets/manual_weight_initializers.md.BDecRyMi.js
@@ -18,10 +18,10 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const c
   0.486214   0.321506  -0.306641  0.145296   0.206476
    To generate weights directly on GPU, pass in a CUDA.RNG. For a complete list of supported RNG types, see Supported RNG Types.
    julia
    using LuxCUDA
 
 weights = kaiming_normal(CUDA.default_rng(), 2, 5)
    2×5 CuArray{Float32, 2, CUDA.DeviceMemory}:
- -0.298409  -0.0152843   0.0946174   0.33094   0.573845
-  1.2014    -0.637114   -0.302317   -0.791806  0.184668
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
- -0.0197287-0.5158im    -0.97285-0.00571707im  …  -0.279012-0.493142im
-  -0.957288+0.343211im   0.51363-0.130897im        0.438734+0.601441im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
+ -0.262623    0.690434  -0.74887    1.15873   -0.149648
+ -0.0928834  -0.241977   0.162357  -0.505488   0.222534
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
+ -0.139314+1.11577im  -0.107044+0.256751im  …   0.201611+0.76422im
+  0.900935-1.04194im   0.726115-0.348918im     -0.639297+0.205454im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
 weights = weights_init(rng, dims...)
 
 # Or
diff --git a/dev/assets/manual_weight_initializers.md.CwPu40d0.lean.js b/dev/assets/manual_weight_initializers.md.BDecRyMi.lean.js
similarity index 81%
rename from dev/assets/manual_weight_initializers.md.CwPu40d0.lean.js
rename to dev/assets/manual_weight_initializers.md.BDecRyMi.lean.js
index 72bff49acd..aa8b998ec4 100644
--- a/dev/assets/manual_weight_initializers.md.CwPu40d0.lean.js
+++ b/dev/assets/manual_weight_initializers.md.BDecRyMi.lean.js
@@ -18,10 +18,10 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const c
   0.486214   0.321506  -0.306641  0.145296   0.206476
    To generate weights directly on GPU, pass in a CUDA.RNG. For a complete list of supported RNG types, see Supported RNG Types.
    julia
    using LuxCUDA
 
 weights = kaiming_normal(CUDA.default_rng(), 2, 5)
    2×5 CuArray{Float32, 2, CUDA.DeviceMemory}:
- -0.298409  -0.0152843   0.0946174   0.33094   0.573845
-  1.2014    -0.637114   -0.302317   -0.791806  0.184668
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
- -0.0197287-0.5158im    -0.97285-0.00571707im  …  -0.279012-0.493142im
-  -0.957288+0.343211im   0.51363-0.130897im        0.438734+0.601441im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
+ -0.262623    0.690434  -0.74887    1.15873   -0.149648
+ -0.0928834  -0.241977   0.162357  -0.505488   0.222534
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
+ -0.139314+1.11577im  -0.107044+0.256751im  …   0.201611+0.76422im
+  0.900935-1.04194im   0.726115-0.348918im     -0.639297+0.205454im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
 weights = weights_init(rng, dims...)
 
 # Or
diff --git a/dev/assets/pinn_nested_ad.B__JnolW.gif b/dev/assets/pinn_nested_ad.B__JnolW.gif
deleted file mode 100644
index efb0cc6668..0000000000
Binary files a/dev/assets/pinn_nested_ad.B__JnolW.gif and /dev/null differ
diff --git a/dev/assets/pinn_nested_ad.Bwdus6h-.gif b/dev/assets/pinn_nested_ad.Bwdus6h-.gif
new file mode 100644
index 0000000000..35241dd01d
Binary files /dev/null and b/dev/assets/pinn_nested_ad.Bwdus6h-.gif differ
diff --git a/dev/assets/style.vsQccYa4.css b/dev/assets/style.jHWe5Zeq.css
similarity index 99%
rename from dev/assets/style.vsQccYa4.css
rename to dev/assets/style.jHWe5Zeq.css
index e5a212973d..07e805177a 100644
--- a/dev/assets/style.vsQccYa4.css
+++ b/dev/assets/style.jHWe5Zeq.css
@@ -1 +1 @@
-@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-cyrillic-ext.BBPuwvHQ.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C88,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-cyrillic.C5lxZ8CY.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-greek-ext.CqjqNYQ-.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-greek.BBVDIX6e.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-vietnamese.BjW4sHH5.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-latin-ext.4ZJIpNVo.woff2) format("woff2");unicode-range:U+0100-02AF,U+0304,U+0308,U+0329,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-latin.Di8DUHzh.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-cyrillic-ext.r48I6akx.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C88,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-cyrillic.By2_1cv3.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-greek-ext.1u6EdAuj.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-greek.DJ8dCoTZ.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-vietnamese.BSbpV94h.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-latin-ext.CN1xVJS-.woff2) format("woff2");unicode-range:U+0100-02AF,U+0304,U+0308,U+0329,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-latin.C2AdPX0b.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Punctuation SC;font-weight:400;src:local("PingFang SC Regular"),local("Noto Sans CJK SC"),local("Microsoft YaHei");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}@font-face{font-family:Punctuation SC;font-weight:500;src:local("PingFang SC Medium"),local("Noto Sans CJK SC"),local("Microsoft YaHei");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}@font-face{font-family:Punctuation SC;font-weight:600;src:local("PingFang SC Semibold"),local("Noto Sans CJK SC Bold"),local("Microsoft YaHei Bold");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}@font-face{font-family:Punctuation SC;font-weight:700;src:local("PingFang SC Semibold"),local("Noto Sans CJK SC Bold"),local("Microsoft YaHei Bold");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}:root{--vp-c-white: #ffffff;--vp-c-black: #000000;--vp-c-neutral: var(--vp-c-black);--vp-c-neutral-inverse: var(--vp-c-white)}.dark{--vp-c-neutral: var(--vp-c-white);--vp-c-neutral-inverse: var(--vp-c-black)}:root{--vp-c-gray-1: #dddde3;--vp-c-gray-2: #e4e4e9;--vp-c-gray-3: #ebebef;--vp-c-gray-soft: rgba(142, 150, 170, .14);--vp-c-indigo-1: #3451b2;--vp-c-indigo-2: #3a5ccc;--vp-c-indigo-3: #5672cd;--vp-c-indigo-soft: rgba(100, 108, 255, .14);--vp-c-purple-1: #6f42c1;--vp-c-purple-2: #7e4cc9;--vp-c-purple-3: #8e5cd9;--vp-c-purple-soft: rgba(159, 122, 234, .14);--vp-c-green-1: #18794e;--vp-c-green-2: #299764;--vp-c-green-3: #30a46c;--vp-c-green-soft: rgba(16, 185, 129, .14);--vp-c-yellow-1: #915930;--vp-c-yellow-2: #946300;--vp-c-yellow-3: #9f6a00;--vp-c-yellow-soft: rgba(234, 179, 8, .14);--vp-c-red-1: #b8272c;--vp-c-red-2: #d5393e;--vp-c-red-3: #e0575b;--vp-c-red-soft: rgba(244, 63, 94, .14);--vp-c-sponsor: #db2777}.dark{--vp-c-gray-1: #515c67;--vp-c-gray-2: #414853;--vp-c-gray-3: #32363f;--vp-c-gray-soft: rgba(101, 117, 133, .16);--vp-c-indigo-1: #a8b1ff;--vp-c-indigo-2: #5c73e7;--vp-c-indigo-3: #3e63dd;--vp-c-indigo-soft: rgba(100, 108, 255, .16);--vp-c-purple-1: #c8abfa;--vp-c-purple-2: #a879e6;--vp-c-purple-3: #8e5cd9;--vp-c-purple-soft: rgba(159, 122, 234, .16);--vp-c-green-1: #3dd68c;--vp-c-green-2: #30a46c;--vp-c-green-3: #298459;--vp-c-green-soft: rgba(16, 185, 129, .16);--vp-c-yellow-1: #f9b44e;--vp-c-yellow-2: #da8b17;--vp-c-yellow-3: #a46a0a;--vp-c-yellow-soft: rgba(234, 179, 8, .16);--vp-c-red-1: #f66f81;--vp-c-red-2: #f14158;--vp-c-red-3: #b62a3c;--vp-c-red-soft: rgba(244, 63, 94, .16)}:root{--vp-c-bg: #ffffff;--vp-c-bg-alt: #f6f6f7;--vp-c-bg-elv: #ffffff;--vp-c-bg-soft: #f6f6f7}.dark{--vp-c-bg: #1b1b1f;--vp-c-bg-alt: #161618;--vp-c-bg-elv: #202127;--vp-c-bg-soft: #202127}:root{--vp-c-border: #c2c2c4;--vp-c-divider: #e2e2e3;--vp-c-gutter: #e2e2e3}.dark{--vp-c-border: #3c3f44;--vp-c-divider: #2e2e32;--vp-c-gutter: #000000}:root{--vp-c-text-1: rgba(60, 60, 67);--vp-c-text-2: rgba(60, 60, 67, .78);--vp-c-text-3: rgba(60, 60, 67, .56)}.dark{--vp-c-text-1: rgba(255, 255, 245, .86);--vp-c-text-2: rgba(235, 235, 245, .6);--vp-c-text-3: rgba(235, 235, 245, .38)}:root{--vp-c-default-1: var(--vp-c-gray-1);--vp-c-default-2: var(--vp-c-gray-2);--vp-c-default-3: var(--vp-c-gray-3);--vp-c-default-soft: var(--vp-c-gray-soft);--vp-c-brand-1: var(--vp-c-indigo-1);--vp-c-brand-2: var(--vp-c-indigo-2);--vp-c-brand-3: var(--vp-c-indigo-3);--vp-c-brand-soft: var(--vp-c-indigo-soft);--vp-c-brand: var(--vp-c-brand-1);--vp-c-tip-1: var(--vp-c-brand-1);--vp-c-tip-2: var(--vp-c-brand-2);--vp-c-tip-3: var(--vp-c-brand-3);--vp-c-tip-soft: var(--vp-c-brand-soft);--vp-c-note-1: var(--vp-c-brand-1);--vp-c-note-2: var(--vp-c-brand-2);--vp-c-note-3: var(--vp-c-brand-3);--vp-c-note-soft: var(--vp-c-brand-soft);--vp-c-success-1: var(--vp-c-green-1);--vp-c-success-2: var(--vp-c-green-2);--vp-c-success-3: var(--vp-c-green-3);--vp-c-success-soft: var(--vp-c-green-soft);--vp-c-important-1: var(--vp-c-purple-1);--vp-c-important-2: var(--vp-c-purple-2);--vp-c-important-3: var(--vp-c-purple-3);--vp-c-important-soft: var(--vp-c-purple-soft);--vp-c-warning-1: var(--vp-c-yellow-1);--vp-c-warning-2: var(--vp-c-yellow-2);--vp-c-warning-3: var(--vp-c-yellow-3);--vp-c-warning-soft: var(--vp-c-yellow-soft);--vp-c-danger-1: var(--vp-c-red-1);--vp-c-danger-2: var(--vp-c-red-2);--vp-c-danger-3: var(--vp-c-red-3);--vp-c-danger-soft: var(--vp-c-red-soft);--vp-c-caution-1: var(--vp-c-red-1);--vp-c-caution-2: var(--vp-c-red-2);--vp-c-caution-3: var(--vp-c-red-3);--vp-c-caution-soft: var(--vp-c-red-soft)}:root{--vp-font-family-base: "Inter", ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";--vp-font-family-mono: ui-monospace, "Menlo", "Monaco", "Consolas", "Liberation Mono", "Courier New", monospace;font-optical-sizing:auto}:root:where(:lang(zh)){--vp-font-family-base: "Punctuation SC", "Inter", ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"}:root{--vp-shadow-1: 0 1px 2px rgba(0, 0, 0, .04), 0 1px 2px rgba(0, 0, 0, .06);--vp-shadow-2: 0 3px 12px rgba(0, 0, 0, .07), 0 1px 4px rgba(0, 0, 0, .07);--vp-shadow-3: 0 12px 32px rgba(0, 0, 0, .1), 0 2px 6px rgba(0, 0, 0, .08);--vp-shadow-4: 0 14px 44px rgba(0, 0, 0, .12), 0 3px 9px rgba(0, 0, 0, .12);--vp-shadow-5: 0 18px 56px rgba(0, 0, 0, .16), 0 4px 12px rgba(0, 0, 0, .16)}:root{--vp-z-index-footer: 10;--vp-z-index-local-nav: 20;--vp-z-index-nav: 30;--vp-z-index-layout-top: 40;--vp-z-index-backdrop: 50;--vp-z-index-sidebar: 60}@media (min-width: 960px){:root{--vp-z-index-sidebar: 25}}:root{--vp-layout-max-width: 1440px}:root{--vp-header-anchor-symbol: "#"}:root{--vp-code-line-height: 1.7;--vp-code-font-size: .875em;--vp-code-color: var(--vp-c-brand-1);--vp-code-link-color: var(--vp-c-brand-1);--vp-code-link-hover-color: var(--vp-c-brand-2);--vp-code-bg: var(--vp-c-default-soft);--vp-code-block-color: var(--vp-c-text-2);--vp-code-block-bg: var(--vp-c-bg-alt);--vp-code-block-divider-color: var(--vp-c-gutter);--vp-code-lang-color: var(--vp-c-text-3);--vp-code-line-highlight-color: var(--vp-c-default-soft);--vp-code-line-number-color: var(--vp-c-text-3);--vp-code-line-diff-add-color: var(--vp-c-success-soft);--vp-code-line-diff-add-symbol-color: var(--vp-c-success-1);--vp-code-line-diff-remove-color: var(--vp-c-danger-soft);--vp-code-line-diff-remove-symbol-color: var(--vp-c-danger-1);--vp-code-line-warning-color: var(--vp-c-warning-soft);--vp-code-line-error-color: var(--vp-c-danger-soft);--vp-code-copy-code-border-color: var(--vp-c-divider);--vp-code-copy-code-bg: var(--vp-c-bg-soft);--vp-code-copy-code-hover-border-color: var(--vp-c-divider);--vp-code-copy-code-hover-bg: var(--vp-c-bg);--vp-code-copy-code-active-text: var(--vp-c-text-2);--vp-code-copy-copied-text-content: "Copied";--vp-code-tab-divider: var(--vp-code-block-divider-color);--vp-code-tab-text-color: var(--vp-c-text-2);--vp-code-tab-bg: var(--vp-code-block-bg);--vp-code-tab-hover-text-color: var(--vp-c-text-1);--vp-code-tab-active-text-color: var(--vp-c-text-1);--vp-code-tab-active-bar-color: var(--vp-c-brand-1)}:root{--vp-button-brand-border: transparent;--vp-button-brand-text: var(--vp-c-white);--vp-button-brand-bg: var(--vp-c-brand-3);--vp-button-brand-hover-border: transparent;--vp-button-brand-hover-text: var(--vp-c-white);--vp-button-brand-hover-bg: var(--vp-c-brand-2);--vp-button-brand-active-border: transparent;--vp-button-brand-active-text: var(--vp-c-white);--vp-button-brand-active-bg: var(--vp-c-brand-1);--vp-button-alt-border: transparent;--vp-button-alt-text: var(--vp-c-text-1);--vp-button-alt-bg: var(--vp-c-default-3);--vp-button-alt-hover-border: transparent;--vp-button-alt-hover-text: var(--vp-c-text-1);--vp-button-alt-hover-bg: var(--vp-c-default-2);--vp-button-alt-active-border: transparent;--vp-button-alt-active-text: var(--vp-c-text-1);--vp-button-alt-active-bg: var(--vp-c-default-1);--vp-button-sponsor-border: var(--vp-c-text-2);--vp-button-sponsor-text: var(--vp-c-text-2);--vp-button-sponsor-bg: transparent;--vp-button-sponsor-hover-border: var(--vp-c-sponsor);--vp-button-sponsor-hover-text: var(--vp-c-sponsor);--vp-button-sponsor-hover-bg: transparent;--vp-button-sponsor-active-border: var(--vp-c-sponsor);--vp-button-sponsor-active-text: var(--vp-c-sponsor);--vp-button-sponsor-active-bg: transparent}:root{--vp-custom-block-font-size: 14px;--vp-custom-block-code-font-size: 13px;--vp-custom-block-info-border: transparent;--vp-custom-block-info-text: var(--vp-c-text-1);--vp-custom-block-info-bg: var(--vp-c-default-soft);--vp-custom-block-info-code-bg: var(--vp-c-default-soft);--vp-custom-block-note-border: transparent;--vp-custom-block-note-text: var(--vp-c-text-1);--vp-custom-block-note-bg: var(--vp-c-default-soft);--vp-custom-block-note-code-bg: var(--vp-c-default-soft);--vp-custom-block-tip-border: transparent;--vp-custom-block-tip-text: var(--vp-c-text-1);--vp-custom-block-tip-bg: var(--vp-c-tip-soft);--vp-custom-block-tip-code-bg: var(--vp-c-tip-soft);--vp-custom-block-important-border: transparent;--vp-custom-block-important-text: var(--vp-c-text-1);--vp-custom-block-important-bg: var(--vp-c-important-soft);--vp-custom-block-important-code-bg: var(--vp-c-important-soft);--vp-custom-block-warning-border: transparent;--vp-custom-block-warning-text: var(--vp-c-text-1);--vp-custom-block-warning-bg: var(--vp-c-warning-soft);--vp-custom-block-warning-code-bg: var(--vp-c-warning-soft);--vp-custom-block-danger-border: transparent;--vp-custom-block-danger-text: var(--vp-c-text-1);--vp-custom-block-danger-bg: var(--vp-c-danger-soft);--vp-custom-block-danger-code-bg: var(--vp-c-danger-soft);--vp-custom-block-caution-border: transparent;--vp-custom-block-caution-text: var(--vp-c-text-1);--vp-custom-block-caution-bg: var(--vp-c-caution-soft);--vp-custom-block-caution-code-bg: var(--vp-c-caution-soft);--vp-custom-block-details-border: var(--vp-custom-block-info-border);--vp-custom-block-details-text: var(--vp-custom-block-info-text);--vp-custom-block-details-bg: var(--vp-custom-block-info-bg);--vp-custom-block-details-code-bg: var(--vp-custom-block-info-code-bg)}:root{--vp-input-border-color: var(--vp-c-border);--vp-input-bg-color: var(--vp-c-bg-alt);--vp-input-switch-bg-color: var(--vp-c-default-soft)}:root{--vp-nav-height: 64px;--vp-nav-bg-color: var(--vp-c-bg);--vp-nav-screen-bg-color: var(--vp-c-bg);--vp-nav-logo-height: 24px}.hide-nav{--vp-nav-height: 0px}.hide-nav .VPSidebar{--vp-nav-height: 22px}:root{--vp-local-nav-bg-color: var(--vp-c-bg)}:root{--vp-sidebar-width: 272px;--vp-sidebar-bg-color: var(--vp-c-bg-alt)}:root{--vp-backdrop-bg-color: rgba(0, 0, 0, .6)}:root{--vp-home-hero-name-color: var(--vp-c-brand-1);--vp-home-hero-name-background: transparent;--vp-home-hero-image-background-image: none;--vp-home-hero-image-filter: none}:root{--vp-badge-info-border: transparent;--vp-badge-info-text: var(--vp-c-text-2);--vp-badge-info-bg: var(--vp-c-default-soft);--vp-badge-tip-border: transparent;--vp-badge-tip-text: var(--vp-c-tip-1);--vp-badge-tip-bg: var(--vp-c-tip-soft);--vp-badge-warning-border: transparent;--vp-badge-warning-text: var(--vp-c-warning-1);--vp-badge-warning-bg: var(--vp-c-warning-soft);--vp-badge-danger-border: transparent;--vp-badge-danger-text: var(--vp-c-danger-1);--vp-badge-danger-bg: var(--vp-c-danger-soft)}:root{--vp-carbon-ads-text-color: var(--vp-c-text-1);--vp-carbon-ads-poweredby-color: var(--vp-c-text-2);--vp-carbon-ads-bg-color: var(--vp-c-bg-soft);--vp-carbon-ads-hover-text-color: var(--vp-c-brand-1);--vp-carbon-ads-hover-poweredby-color: var(--vp-c-text-1)}:root{--vp-local-search-bg: var(--vp-c-bg);--vp-local-search-result-bg: var(--vp-c-bg);--vp-local-search-result-border: var(--vp-c-divider);--vp-local-search-result-selected-bg: var(--vp-c-bg);--vp-local-search-result-selected-border: var(--vp-c-brand-1);--vp-local-search-highlight-bg: var(--vp-c-brand-1);--vp-local-search-highlight-text: var(--vp-c-neutral-inverse)}@media (prefers-reduced-motion: reduce){*,:before,:after{animation-delay:-1ms!important;animation-duration:1ms!important;animation-iteration-count:1!important;background-attachment:initial!important;scroll-behavior:auto!important;transition-duration:0s!important;transition-delay:0s!important}}*,:before,:after{box-sizing:border-box}html{line-height:1.4;font-size:16px;-webkit-text-size-adjust:100%}html.dark{color-scheme:dark}body{margin:0;width:100%;min-width:320px;min-height:100vh;line-height:24px;font-family:var(--vp-font-family-base);font-size:16px;font-weight:400;color:var(--vp-c-text-1);background-color:var(--vp-c-bg);font-synthesis:style;text-rendering:optimizeLegibility;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}main{display:block}h1,h2,h3,h4,h5,h6{margin:0;line-height:24px;font-size:16px;font-weight:400}p{margin:0}strong,b{font-weight:600}a,area,button,[role=button],input,label,select,summary,textarea{touch-action:manipulation}a{color:inherit;text-decoration:inherit}ol,ul{list-style:none;margin:0;padding:0}blockquote{margin:0}pre,code,kbd,samp{font-family:var(--vp-font-family-mono)}img,svg,video,canvas,audio,iframe,embed,object{display:block}figure{margin:0}img,video{max-width:100%;height:auto}button,input,optgroup,select,textarea{border:0;padding:0;line-height:inherit;color:inherit}button{padding:0;font-family:inherit;background-color:transparent;background-image:none}button:enabled,[role=button]:enabled{cursor:pointer}button:focus,button:focus-visible{outline:1px dotted;outline:4px auto -webkit-focus-ring-color}button:focus:not(:focus-visible){outline:none!important}input:focus,textarea:focus,select:focus{outline:none}table{border-collapse:collapse}input{background-color:transparent}input:-ms-input-placeholder,textarea:-ms-input-placeholder{color:var(--vp-c-text-3)}input::-ms-input-placeholder,textarea::-ms-input-placeholder{color:var(--vp-c-text-3)}input::placeholder,textarea::placeholder{color:var(--vp-c-text-3)}input::-webkit-outer-spin-button,input::-webkit-inner-spin-button{-webkit-appearance:none;margin:0}input[type=number]{-moz-appearance:textfield}textarea{resize:vertical}select{-webkit-appearance:none}fieldset{margin:0;padding:0}h1,h2,h3,h4,h5,h6,li,p{overflow-wrap:break-word}vite-error-overlay{z-index:9999}mjx-container{overflow-x:auto}mjx-container>svg{display:inline-block;margin:auto}[class^=vpi-],[class*=" vpi-"],.vp-icon{width:1em;height:1em}[class^=vpi-].bg,[class*=" vpi-"].bg,.vp-icon.bg{background-size:100% 100%;background-color:transparent}[class^=vpi-]:not(.bg),[class*=" vpi-"]:not(.bg),.vp-icon:not(.bg){-webkit-mask:var(--icon) no-repeat;mask:var(--icon) no-repeat;-webkit-mask-size:100% 100%;mask-size:100% 100%;background-color:currentColor;color:inherit}.vpi-align-left{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M21 6H3M15 12H3M17 18H3'/%3E%3C/svg%3E")}.vpi-arrow-right,.vpi-arrow-down,.vpi-arrow-left,.vpi-arrow-up{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M5 12h14M12 5l7 7-7 7'/%3E%3C/svg%3E")}.vpi-chevron-right,.vpi-chevron-down,.vpi-chevron-left,.vpi-chevron-up{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='m9 18 6-6-6-6'/%3E%3C/svg%3E")}.vpi-chevron-down,.vpi-arrow-down{transform:rotate(90deg)}.vpi-chevron-left,.vpi-arrow-left{transform:rotate(180deg)}.vpi-chevron-up,.vpi-arrow-up{transform:rotate(-90deg)}.vpi-square-pen{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M12 3H5a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h14a2 2 0 0 0 2-2v-7'/%3E%3Cpath d='M18.375 2.625a2.121 2.121 0 1 1 3 3L12 15l-4 1 1-4Z'/%3E%3C/svg%3E")}.vpi-plus{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M5 12h14M12 5v14'/%3E%3C/svg%3E")}.vpi-sun{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Ccircle cx='12' cy='12' r='4'/%3E%3Cpath d='M12 2v2M12 20v2M4.93 4.93l1.41 1.41M17.66 17.66l1.41 1.41M2 12h2M20 12h2M6.34 17.66l-1.41 1.41M19.07 4.93l-1.41 1.41'/%3E%3C/svg%3E")}.vpi-moon{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M12 3a6 6 0 0 0 9 9 9 9 0 1 1-9-9Z'/%3E%3C/svg%3E")}.vpi-more-horizontal{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Ccircle cx='12' cy='12' r='1'/%3E%3Ccircle cx='19' cy='12' r='1'/%3E%3Ccircle cx='5' cy='12' r='1'/%3E%3C/svg%3E")}.vpi-languages{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='m5 8 6 6M4 14l6-6 2-3M2 5h12M7 2h1M22 22l-5-10-5 10M14 18h6'/%3E%3C/svg%3E")}.vpi-heart{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M19 14c1.49-1.46 3-3.21 3-5.5A5.5 5.5 0 0 0 16.5 3c-1.76 0-3 .5-4.5 2-1.5-1.5-2.74-2-4.5-2A5.5 5.5 0 0 0 2 8.5c0 2.3 1.5 4.05 3 5.5l7 7Z'/%3E%3C/svg%3E")}.vpi-search{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Ccircle cx='11' cy='11' r='8'/%3E%3Cpath d='m21 21-4.3-4.3'/%3E%3C/svg%3E")}.vpi-layout-list{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Crect width='7' height='7' x='3' y='3' rx='1'/%3E%3Crect width='7' height='7' x='3' y='14' rx='1'/%3E%3Cpath d='M14 4h7M14 9h7M14 15h7M14 20h7'/%3E%3C/svg%3E")}.vpi-delete{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M20 5H9l-7 7 7 7h11a2 2 0 0 0 2-2V7a2 2 0 0 0-2-2ZM18 9l-6 6M12 9l6 6'/%3E%3C/svg%3E")}.vpi-corner-down-left{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='m9 10-5 5 5 5'/%3E%3Cpath d='M20 4v7a4 4 0 0 1-4 4H4'/%3E%3C/svg%3E")}:root{--vp-icon-copy: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='rgba(128,128,128,1)' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Crect width='8' height='4' x='8' y='2' rx='1' ry='1'/%3E%3Cpath d='M16 4h2a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2'/%3E%3C/svg%3E");--vp-icon-copied: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='rgba(128,128,128,1)' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Crect width='8' height='4' x='8' y='2' rx='1' ry='1'/%3E%3Cpath d='M16 4h2a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2'/%3E%3Cpath d='m9 14 2 2 4-4'/%3E%3C/svg%3E")}.vpi-social-discord{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M20.317 4.37a19.791 19.791 0 0 0-4.885-1.515.074.074 0 0 0-.079.037c-.21.375-.444.864-.608 1.25a18.27 18.27 0 0 0-5.487 0 12.64 12.64 0 0 0-.617-1.25.077.077 0 0 0-.079-.037A19.736 19.736 0 0 0 3.677 4.37a.07.07 0 0 0-.032.027C.533 9.046-.32 13.58.099 18.057a.082.082 0 0 0 .031.057 19.9 19.9 0 0 0 5.993 3.03.078.078 0 0 0 .084-.028c.462-.63.874-1.295 1.226-1.994a.076.076 0 0 0-.041-.106 13.107 13.107 0 0 1-1.872-.892.077.077 0 0 1-.008-.128 10.2 10.2 0 0 0 .372-.292.074.074 0 0 1 .077-.01c3.928 1.793 8.18 1.793 12.062 0a.074.074 0 0 1 .078.01c.12.098.246.198.373.292a.077.077 0 0 1-.006.127 12.299 12.299 0 0 1-1.873.892.077.077 0 0 0-.041.107c.36.698.772 1.362 1.225 1.993a.076.076 0 0 0 .084.028 19.839 19.839 0 0 0 6.002-3.03.077.077 0 0 0 .032-.054c.5-5.177-.838-9.674-3.549-13.66a.061.061 0 0 0-.031-.03zM8.02 15.33c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.956-2.419 2.157-2.419 1.21 0 2.176 1.096 2.157 2.42 0 1.333-.956 2.418-2.157 2.418zm7.975 0c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.955-2.419 2.157-2.419 1.21 0 2.176 1.096 2.157 2.42 0 1.333-.946 2.418-2.157 2.418Z'/%3E%3C/svg%3E")}.vpi-social-facebook{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M9.101 23.691v-7.98H6.627v-3.667h2.474v-1.58c0-4.085 1.848-5.978 5.858-5.978.401 0 .955.042 1.468.103a8.68 8.68 0 0 1 1.141.195v3.325a8.623 8.623 0 0 0-.653-.036 26.805 26.805 0 0 0-.733-.009c-.707 0-1.259.096-1.675.309a1.686 1.686 0 0 0-.679.622c-.258.42-.374.995-.374 1.752v1.297h3.919l-.386 2.103-.287 1.564h-3.246v8.245C19.396 23.238 24 18.179 24 12.044c0-6.627-5.373-12-12-12s-12 5.373-12 12c0 5.628 3.874 10.35 9.101 11.647Z'/%3E%3C/svg%3E")}.vpi-social-github{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12'/%3E%3C/svg%3E")}.vpi-social-instagram{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M7.03.084c-1.277.06-2.149.264-2.91.563a5.874 5.874 0 0 0-2.124 1.388 5.878 5.878 0 0 0-1.38 2.127C.321 4.926.12 5.8.064 7.076.008 8.354-.005 8.764.001 12.023c.007 3.259.021 3.667.083 4.947.061 1.277.264 2.149.563 2.911.308.789.72 1.457 1.388 2.123a5.872 5.872 0 0 0 2.129 1.38c.763.295 1.636.496 2.913.552 1.278.056 1.689.069 4.947.063 3.257-.007 3.668-.021 4.947-.082 1.28-.06 2.147-.265 2.91-.563a5.881 5.881 0 0 0 2.123-1.388 5.881 5.881 0 0 0 1.38-2.129c.295-.763.496-1.636.551-2.912.056-1.28.07-1.69.063-4.948-.006-3.258-.02-3.667-.081-4.947-.06-1.28-.264-2.148-.564-2.911a5.892 5.892 0 0 0-1.387-2.123 5.857 5.857 0 0 0-2.128-1.38C19.074.322 18.202.12 16.924.066 15.647.009 15.236-.006 11.977 0 8.718.008 8.31.021 7.03.084m.14 21.693c-1.17-.05-1.805-.245-2.228-.408a3.736 3.736 0 0 1-1.382-.895 3.695 3.695 0 0 1-.9-1.378c-.165-.423-.363-1.058-.417-2.228-.06-1.264-.072-1.644-.08-4.848-.006-3.204.006-3.583.061-4.848.05-1.169.246-1.805.408-2.228.216-.561.477-.96.895-1.382a3.705 3.705 0 0 1 1.379-.9c.423-.165 1.057-.361 2.227-.417 1.265-.06 1.644-.072 4.848-.08 3.203-.006 3.583.006 4.85.062 1.168.05 1.804.244 2.227.408.56.216.96.475 1.382.895.421.42.681.817.9 1.378.165.422.362 1.056.417 2.227.06 1.265.074 1.645.08 4.848.005 3.203-.006 3.583-.061 4.848-.051 1.17-.245 1.805-.408 2.23-.216.56-.477.96-.896 1.38a3.705 3.705 0 0 1-1.378.9c-.422.165-1.058.362-2.226.418-1.266.06-1.645.072-4.85.079-3.204.007-3.582-.006-4.848-.06m9.783-16.192a1.44 1.44 0 1 0 1.437-1.442 1.44 1.44 0 0 0-1.437 1.442M5.839 12.012a6.161 6.161 0 1 0 12.323-.024 6.162 6.162 0 0 0-12.323.024M8 12.008A4 4 0 1 1 12.008 16 4 4 0 0 1 8 12.008'/%3E%3C/svg%3E")}.vpi-social-linkedin{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M20.447 20.452h-3.554v-5.569c0-1.328-.027-3.037-1.852-3.037-1.853 0-2.136 1.445-2.136 2.939v5.667H9.351V9h3.414v1.561h.046c.477-.9 1.637-1.85 3.37-1.85 3.601 0 4.267 2.37 4.267 5.455v6.286zM5.337 7.433a2.062 2.062 0 0 1-2.063-2.065 2.064 2.064 0 1 1 2.063 2.065zm1.782 13.019H3.555V9h3.564v11.452zM22.225 0H1.771C.792 0 0 .774 0 1.729v20.542C0 23.227.792 24 1.771 24h20.451C23.2 24 24 23.227 24 22.271V1.729C24 .774 23.2 0 22.222 0h.003z'/%3E%3C/svg%3E")}.vpi-social-mastodon{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M23.268 5.313c-.35-2.578-2.617-4.61-5.304-5.004C17.51.242 15.792 0 11.813 0h-.03c-3.98 0-4.835.242-5.288.309C3.882.692 1.496 2.518.917 5.127.64 6.412.61 7.837.661 9.143c.074 1.874.088 3.745.26 5.611.118 1.24.325 2.47.62 3.68.55 2.237 2.777 4.098 4.96 4.857 2.336.792 4.849.923 7.256.38.265-.061.527-.132.786-.213.585-.184 1.27-.39 1.774-.753a.057.057 0 0 0 .023-.043v-1.809a.052.052 0 0 0-.02-.041.053.053 0 0 0-.046-.01 20.282 20.282 0 0 1-4.709.545c-2.73 0-3.463-1.284-3.674-1.818a5.593 5.593 0 0 1-.319-1.433.053.053 0 0 1 .066-.054c1.517.363 3.072.546 4.632.546.376 0 .75 0 1.125-.01 1.57-.044 3.224-.124 4.768-.422.038-.008.077-.015.11-.024 2.435-.464 4.753-1.92 4.989-5.604.008-.145.03-1.52.03-1.67.002-.512.167-3.63-.024-5.545zm-3.748 9.195h-2.561V8.29c0-1.309-.55-1.976-1.67-1.976-1.23 0-1.846.79-1.846 2.35v3.403h-2.546V8.663c0-1.56-.617-2.35-1.848-2.35-1.112 0-1.668.668-1.67 1.977v6.218H4.822V8.102c0-1.31.337-2.35 1.011-3.12.696-.77 1.608-1.164 2.74-1.164 1.311 0 2.302.5 2.962 1.498l.638 1.06.638-1.06c.66-.999 1.65-1.498 2.96-1.498 1.13 0 2.043.395 2.74 1.164.675.77 1.012 1.81 1.012 3.12z'/%3E%3C/svg%3E")}.vpi-social-npm{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M1.763 0C.786 0 0 .786 0 1.763v20.474C0 23.214.786 24 1.763 24h20.474c.977 0 1.763-.786 1.763-1.763V1.763C24 .786 23.214 0 22.237 0zM5.13 5.323l13.837.019-.009 13.836h-3.464l.01-10.382h-3.456L12.04 19.17H5.113z'/%3E%3C/svg%3E")}.vpi-social-slack{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M5.042 15.165a2.528 2.528 0 0 1-2.52 2.523A2.528 2.528 0 0 1 0 15.165a2.527 2.527 0 0 1 2.522-2.52h2.52v2.52zm1.271 0a2.527 2.527 0 0 1 2.521-2.52 2.527 2.527 0 0 1 2.521 2.52v6.313A2.528 2.528 0 0 1 8.834 24a2.528 2.528 0 0 1-2.521-2.522v-6.313zM8.834 5.042a2.528 2.528 0 0 1-2.521-2.52A2.528 2.528 0 0 1 8.834 0a2.528 2.528 0 0 1 2.521 2.522v2.52H8.834zm0 1.271a2.528 2.528 0 0 1 2.521 2.521 2.528 2.528 0 0 1-2.521 2.521H2.522A2.528 2.528 0 0 1 0 8.834a2.528 2.528 0 0 1 2.522-2.521h6.312zm10.122 2.521a2.528 2.528 0 0 1 2.522-2.521A2.528 2.528 0 0 1 24 8.834a2.528 2.528 0 0 1-2.522 2.521h-2.522V8.834zm-1.268 0a2.528 2.528 0 0 1-2.523 2.521 2.527 2.527 0 0 1-2.52-2.521V2.522A2.527 2.527 0 0 1 15.165 0a2.528 2.528 0 0 1 2.523 2.522v6.312zm-2.523 10.122a2.528 2.528 0 0 1 2.523 2.522A2.528 2.528 0 0 1 15.165 24a2.527 2.527 0 0 1-2.52-2.522v-2.522h2.52zm0-1.268a2.527 2.527 0 0 1-2.52-2.523 2.526 2.526 0 0 1 2.52-2.52h6.313A2.527 2.527 0 0 1 24 15.165a2.528 2.528 0 0 1-2.522 2.523h-6.313z'/%3E%3C/svg%3E")}.vpi-social-twitter,.vpi-social-x{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M18.901 1.153h3.68l-8.04 9.19L24 22.846h-7.406l-5.8-7.584-6.638 7.584H.474l8.6-9.83L0 1.154h7.594l5.243 6.932ZM17.61 20.644h2.039L6.486 3.24H4.298Z'/%3E%3C/svg%3E")}.vpi-social-youtube{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M23.498 6.186a3.016 3.016 0 0 0-2.122-2.136C19.505 3.545 12 3.545 12 3.545s-7.505 0-9.377.505A3.017 3.017 0 0 0 .502 6.186C0 8.07 0 12 0 12s0 3.93.502 5.814a3.016 3.016 0 0 0 2.122 2.136c1.871.505 9.376.505 9.376.505s7.505 0 9.377-.505a3.015 3.015 0 0 0 2.122-2.136C24 15.93 24 12 24 12s0-3.93-.502-5.814zM9.545 15.568V8.432L15.818 12l-6.273 3.568z'/%3E%3C/svg%3E")}.visually-hidden{position:absolute;width:1px;height:1px;white-space:nowrap;clip:rect(0 0 0 0);clip-path:inset(50%);overflow:hidden}.custom-block{border:1px solid transparent;border-radius:8px;padding:16px 16px 8px;line-height:24px;font-size:var(--vp-custom-block-font-size);color:var(--vp-c-text-2)}.custom-block.info{border-color:var(--vp-custom-block-info-border);color:var(--vp-custom-block-info-text);background-color:var(--vp-custom-block-info-bg)}.custom-block.info a,.custom-block.info code{color:var(--vp-c-brand-1)}.custom-block.info a:hover,.custom-block.info a:hover>code{color:var(--vp-c-brand-2)}.custom-block.info code{background-color:var(--vp-custom-block-info-code-bg)}.custom-block.note{border-color:var(--vp-custom-block-note-border);color:var(--vp-custom-block-note-text);background-color:var(--vp-custom-block-note-bg)}.custom-block.note a,.custom-block.note code{color:var(--vp-c-brand-1)}.custom-block.note a:hover,.custom-block.note a:hover>code{color:var(--vp-c-brand-2)}.custom-block.note code{background-color:var(--vp-custom-block-note-code-bg)}.custom-block.tip{border-color:var(--vp-custom-block-tip-border);color:var(--vp-custom-block-tip-text);background-color:var(--vp-custom-block-tip-bg)}.custom-block.tip a,.custom-block.tip code{color:var(--vp-c-tip-1)}.custom-block.tip a:hover,.custom-block.tip a:hover>code{color:var(--vp-c-tip-2)}.custom-block.tip code{background-color:var(--vp-custom-block-tip-code-bg)}.custom-block.important{border-color:var(--vp-custom-block-important-border);color:var(--vp-custom-block-important-text);background-color:var(--vp-custom-block-important-bg)}.custom-block.important a,.custom-block.important code{color:var(--vp-c-important-1)}.custom-block.important a:hover,.custom-block.important a:hover>code{color:var(--vp-c-important-2)}.custom-block.important code{background-color:var(--vp-custom-block-important-code-bg)}.custom-block.warning{border-color:var(--vp-custom-block-warning-border);color:var(--vp-custom-block-warning-text);background-color:var(--vp-custom-block-warning-bg)}.custom-block.warning a,.custom-block.warning code{color:var(--vp-c-warning-1)}.custom-block.warning a:hover,.custom-block.warning a:hover>code{color:var(--vp-c-warning-2)}.custom-block.warning code{background-color:var(--vp-custom-block-warning-code-bg)}.custom-block.danger{border-color:var(--vp-custom-block-danger-border);color:var(--vp-custom-block-danger-text);background-color:var(--vp-custom-block-danger-bg)}.custom-block.danger a,.custom-block.danger code{color:var(--vp-c-danger-1)}.custom-block.danger a:hover,.custom-block.danger a:hover>code{color:var(--vp-c-danger-2)}.custom-block.danger code{background-color:var(--vp-custom-block-danger-code-bg)}.custom-block.caution{border-color:var(--vp-custom-block-caution-border);color:var(--vp-custom-block-caution-text);background-color:var(--vp-custom-block-caution-bg)}.custom-block.caution a,.custom-block.caution code{color:var(--vp-c-caution-1)}.custom-block.caution a:hover,.custom-block.caution a:hover>code{color:var(--vp-c-caution-2)}.custom-block.caution code{background-color:var(--vp-custom-block-caution-code-bg)}.custom-block.details{border-color:var(--vp-custom-block-details-border);color:var(--vp-custom-block-details-text);background-color:var(--vp-custom-block-details-bg)}.custom-block.details a{color:var(--vp-c-brand-1)}.custom-block.details a:hover,.custom-block.details a:hover>code{color:var(--vp-c-brand-2)}.custom-block.details code{background-color:var(--vp-custom-block-details-code-bg)}.custom-block-title{font-weight:600}.custom-block p+p{margin:8px 0}.custom-block.details summary{margin:0 0 8px;font-weight:700;cursor:pointer;-webkit-user-select:none;user-select:none}.custom-block.details summary+p{margin:8px 0}.custom-block a{color:inherit;font-weight:600;text-decoration:underline;text-underline-offset:2px;transition:opacity .25s}.custom-block a:hover{opacity:.75}.custom-block code{font-size:var(--vp-custom-block-code-font-size)}.custom-block.custom-block th,.custom-block.custom-block blockquote>p{font-size:var(--vp-custom-block-font-size);color:inherit}.dark .vp-code span{color:var(--shiki-dark, inherit)}html:not(.dark) .vp-code span{color:var(--shiki-light, inherit)}.vp-code-group{margin-top:16px}.vp-code-group .tabs{position:relative;display:flex;margin-right:-24px;margin-left:-24px;padding:0 12px;background-color:var(--vp-code-tab-bg);overflow-x:auto;overflow-y:hidden;box-shadow:inset 0 -1px var(--vp-code-tab-divider)}@media (min-width: 640px){.vp-code-group .tabs{margin-right:0;margin-left:0;border-radius:8px 8px 0 0}}.vp-code-group .tabs input{position:fixed;opacity:0;pointer-events:none}.vp-code-group .tabs label{position:relative;display:inline-block;border-bottom:1px solid transparent;padding:0 12px;line-height:48px;font-size:14px;font-weight:500;color:var(--vp-code-tab-text-color);white-space:nowrap;cursor:pointer;transition:color .25s}.vp-code-group .tabs label:after{position:absolute;right:8px;bottom:-1px;left:8px;z-index:1;height:2px;border-radius:2px;content:"";background-color:transparent;transition:background-color .25s}.vp-code-group label:hover{color:var(--vp-code-tab-hover-text-color)}.vp-code-group input:checked+label{color:var(--vp-code-tab-active-text-color)}.vp-code-group input:checked+label:after{background-color:var(--vp-code-tab-active-bar-color)}.vp-code-group div[class*=language-],.vp-block{display:none;margin-top:0!important;border-top-left-radius:0!important;border-top-right-radius:0!important}.vp-code-group div[class*=language-].active,.vp-block.active{display:block}.vp-block{padding:20px 24px}.vp-doc h1,.vp-doc h2,.vp-doc h3,.vp-doc h4,.vp-doc h5,.vp-doc h6{position:relative;font-weight:600;outline:none}.vp-doc h1{letter-spacing:-.02em;line-height:40px;font-size:28px}.vp-doc h2{margin:48px 0 16px;border-top:1px solid var(--vp-c-divider);padding-top:24px;letter-spacing:-.02em;line-height:32px;font-size:24px}.vp-doc h3{margin:32px 0 0;letter-spacing:-.01em;line-height:28px;font-size:20px}.vp-doc h4{margin:24px 0 0;letter-spacing:-.01em;line-height:24px;font-size:18px}.vp-doc .header-anchor{position:absolute;top:0;left:0;margin-left:-.87em;font-weight:500;-webkit-user-select:none;user-select:none;opacity:0;text-decoration:none;transition:color .25s,opacity .25s}.vp-doc .header-anchor:before{content:var(--vp-header-anchor-symbol)}.vp-doc h1:hover .header-anchor,.vp-doc h1 .header-anchor:focus,.vp-doc h2:hover .header-anchor,.vp-doc h2 .header-anchor:focus,.vp-doc h3:hover .header-anchor,.vp-doc h3 .header-anchor:focus,.vp-doc h4:hover .header-anchor,.vp-doc h4 .header-anchor:focus,.vp-doc h5:hover .header-anchor,.vp-doc h5 .header-anchor:focus,.vp-doc h6:hover .header-anchor,.vp-doc h6 .header-anchor:focus{opacity:1}@media (min-width: 768px){.vp-doc h1{letter-spacing:-.02em;line-height:40px;font-size:32px}}.vp-doc h2 .header-anchor{top:24px}.vp-doc p,.vp-doc summary{margin:16px 0}.vp-doc p{line-height:28px}.vp-doc blockquote{margin:16px 0;border-left:2px solid var(--vp-c-divider);padding-left:16px;transition:border-color .5s;color:var(--vp-c-text-2)}.vp-doc blockquote>p{margin:0;font-size:16px;transition:color .5s}.vp-doc a{font-weight:500;color:var(--vp-c-brand-1);text-decoration:underline;text-underline-offset:2px;transition:color .25s,opacity .25s}.vp-doc a:hover{color:var(--vp-c-brand-2)}.vp-doc strong{font-weight:600}.vp-doc ul,.vp-doc ol{padding-left:1.25rem;margin:16px 0}.vp-doc ul{list-style:disc}.vp-doc ol{list-style:decimal}.vp-doc li+li{margin-top:8px}.vp-doc li>ol,.vp-doc li>ul{margin:8px 0 0}.vp-doc table{display:block;border-collapse:collapse;margin:20px 0;overflow-x:auto}.vp-doc tr{background-color:var(--vp-c-bg);border-top:1px solid var(--vp-c-divider);transition:background-color .5s}.vp-doc tr:nth-child(2n){background-color:var(--vp-c-bg-soft)}.vp-doc th,.vp-doc td{border:1px solid var(--vp-c-divider);padding:8px 16px}.vp-doc th{text-align:left;font-size:14px;font-weight:600;color:var(--vp-c-text-2);background-color:var(--vp-c-bg-soft)}.vp-doc td{font-size:14px}.vp-doc hr{margin:16px 0;border:none;border-top:1px solid var(--vp-c-divider)}.vp-doc .custom-block{margin:16px 0}.vp-doc .custom-block p{margin:8px 0;line-height:24px}.vp-doc .custom-block p:first-child{margin:0}.vp-doc .custom-block div[class*=language-]{margin:8px 0;border-radius:8px}.vp-doc .custom-block div[class*=language-] code{font-weight:400;background-color:transparent}.vp-doc .custom-block .vp-code-group .tabs{margin:0;border-radius:8px 8px 0 0}.vp-doc :not(pre,h1,h2,h3,h4,h5,h6)>code{font-size:var(--vp-code-font-size);color:var(--vp-code-color)}.vp-doc :not(pre)>code{border-radius:4px;padding:3px 6px;background-color:var(--vp-code-bg);transition:color .25s,background-color .5s}.vp-doc a>code{color:var(--vp-code-link-color)}.vp-doc a:hover>code{color:var(--vp-code-link-hover-color)}.vp-doc h1>code,.vp-doc h2>code,.vp-doc h3>code,.vp-doc h4>code{font-size:.9em}.vp-doc div[class*=language-],.vp-block{position:relative;margin:16px -24px;background-color:var(--vp-code-block-bg);overflow-x:auto;transition:background-color .5s}@media (min-width: 640px){.vp-doc div[class*=language-],.vp-block{border-radius:8px;margin:16px 0}}@media (max-width: 639px){.vp-doc li div[class*=language-]{border-radius:8px 0 0 8px}}.vp-doc div[class*=language-]+div[class*=language-],.vp-doc div[class$=-api]+div[class*=language-],.vp-doc div[class*=language-]+div[class$=-api]>div[class*=language-]{margin-top:-8px}.vp-doc [class*=language-] pre,.vp-doc [class*=language-] code{direction:ltr;text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none}.vp-doc [class*=language-] pre{position:relative;z-index:1;margin:0;padding:20px 0;background:transparent;overflow-x:auto}.vp-doc [class*=language-] code{display:block;padding:0 24px;width:fit-content;min-width:100%;line-height:var(--vp-code-line-height);font-size:var(--vp-code-font-size);color:var(--vp-code-block-color);transition:color .5s}.vp-doc [class*=language-] code .highlighted{background-color:var(--vp-code-line-highlight-color);transition:background-color .5s;margin:0 -24px;padding:0 24px;width:calc(100% + 48px);display:inline-block}.vp-doc [class*=language-] code .highlighted.error{background-color:var(--vp-code-line-error-color)}.vp-doc [class*=language-] code .highlighted.warning{background-color:var(--vp-code-line-warning-color)}.vp-doc [class*=language-] code .diff{transition:background-color .5s;margin:0 -24px;padding:0 24px;width:calc(100% + 48px);display:inline-block}.vp-doc [class*=language-] code .diff:before{position:absolute;left:10px}.vp-doc [class*=language-] .has-focused-lines .line:not(.has-focus){filter:blur(.095rem);opacity:.4;transition:filter .35s,opacity .35s}.vp-doc [class*=language-] .has-focused-lines .line:not(.has-focus){opacity:.7;transition:filter .35s,opacity .35s}.vp-doc [class*=language-]:hover .has-focused-lines .line:not(.has-focus){filter:blur(0);opacity:1}.vp-doc [class*=language-] code .diff.remove{background-color:var(--vp-code-line-diff-remove-color);opacity:.7}.vp-doc [class*=language-] code .diff.remove:before{content:"-";color:var(--vp-code-line-diff-remove-symbol-color)}.vp-doc [class*=language-] code .diff.add{background-color:var(--vp-code-line-diff-add-color)}.vp-doc [class*=language-] code .diff.add:before{content:"+";color:var(--vp-code-line-diff-add-symbol-color)}.vp-doc div[class*=language-].line-numbers-mode{padding-left:32px}.vp-doc .line-numbers-wrapper{position:absolute;top:0;bottom:0;left:0;z-index:3;border-right:1px solid var(--vp-code-block-divider-color);padding-top:20px;width:32px;text-align:center;font-family:var(--vp-font-family-mono);line-height:var(--vp-code-line-height);font-size:var(--vp-code-font-size);color:var(--vp-code-line-number-color);transition:border-color .5s,color .5s}.vp-doc [class*=language-]>button.copy{direction:ltr;position:absolute;top:12px;right:12px;z-index:3;border:1px solid var(--vp-code-copy-code-border-color);border-radius:4px;width:40px;height:40px;background-color:var(--vp-code-copy-code-bg);opacity:0;cursor:pointer;background-image:var(--vp-icon-copy);background-position:50%;background-size:20px;background-repeat:no-repeat;transition:border-color .25s,background-color .25s,opacity .25s}.vp-doc [class*=language-]:hover>button.copy,.vp-doc [class*=language-]>button.copy:focus{opacity:1}.vp-doc [class*=language-]>button.copy:hover,.vp-doc [class*=language-]>button.copy.copied{border-color:var(--vp-code-copy-code-hover-border-color);background-color:var(--vp-code-copy-code-hover-bg)}.vp-doc [class*=language-]>button.copy.copied,.vp-doc [class*=language-]>button.copy:hover.copied{border-radius:0 4px 4px 0;background-color:var(--vp-code-copy-code-hover-bg);background-image:var(--vp-icon-copied)}.vp-doc [class*=language-]>button.copy.copied:before,.vp-doc [class*=language-]>button.copy:hover.copied:before{position:relative;top:-1px;transform:translate(calc(-100% - 1px));display:flex;justify-content:center;align-items:center;border:1px solid var(--vp-code-copy-code-hover-border-color);border-right:0;border-radius:4px 0 0 4px;padding:0 10px;width:fit-content;height:40px;text-align:center;font-size:12px;font-weight:500;color:var(--vp-code-copy-code-active-text);background-color:var(--vp-code-copy-code-hover-bg);white-space:nowrap;content:var(--vp-code-copy-copied-text-content)}.vp-doc [class*=language-]>span.lang{position:absolute;top:2px;right:8px;z-index:2;font-size:12px;font-weight:500;-webkit-user-select:none;user-select:none;color:var(--vp-code-lang-color);transition:color .4s,opacity .4s}.vp-doc [class*=language-]:hover>button.copy+span.lang,.vp-doc [class*=language-]>button.copy:focus+span.lang{opacity:0}.vp-doc .VPTeamMembers{margin-top:24px}.vp-doc .VPTeamMembers.small.count-1 .container{margin:0!important;max-width:calc((100% - 24px)/2)!important}.vp-doc .VPTeamMembers.small.count-2 .container,.vp-doc .VPTeamMembers.small.count-3 .container{max-width:100%!important}.vp-doc .VPTeamMembers.medium.count-1 .container{margin:0!important;max-width:calc((100% - 24px)/2)!important}:is(.vp-external-link-icon,.vp-doc a[href*="://"],.vp-doc a[target=_blank]):not(.no-icon):after{display:inline-block;margin-top:-1px;margin-left:4px;width:11px;height:11px;background:currentColor;color:var(--vp-c-text-3);flex-shrink:0;--icon: url("data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24' %3E%3Cpath d='M0 0h24v24H0V0z' fill='none' /%3E%3Cpath d='M9 5v2h6.59L4 18.59 5.41 20 17 8.41V15h2V5H9z' /%3E%3C/svg%3E");-webkit-mask-image:var(--icon);mask-image:var(--icon)}.vp-external-link-icon:after{content:""}.external-link-icon-enabled :is(.vp-doc a[href*="://"],.vp-doc a[target=_blank]):after{content:"";color:currentColor}.vp-sponsor{border-radius:16px;overflow:hidden}.vp-sponsor.aside{border-radius:12px}.vp-sponsor-section+.vp-sponsor-section{margin-top:4px}.vp-sponsor-tier{margin:0 0 4px!important;text-align:center;letter-spacing:1px!important;line-height:24px;width:100%;font-weight:600;color:var(--vp-c-text-2);background-color:var(--vp-c-bg-soft)}.vp-sponsor.normal .vp-sponsor-tier{padding:13px 0 11px;font-size:14px}.vp-sponsor.aside .vp-sponsor-tier{padding:9px 0 7px;font-size:12px}.vp-sponsor-grid+.vp-sponsor-tier{margin-top:4px}.vp-sponsor-grid{display:flex;flex-wrap:wrap;gap:4px}.vp-sponsor-grid.xmini .vp-sponsor-grid-link{height:64px}.vp-sponsor-grid.xmini .vp-sponsor-grid-image{max-width:64px;max-height:22px}.vp-sponsor-grid.mini .vp-sponsor-grid-link{height:72px}.vp-sponsor-grid.mini .vp-sponsor-grid-image{max-width:96px;max-height:24px}.vp-sponsor-grid.small .vp-sponsor-grid-link{height:96px}.vp-sponsor-grid.small .vp-sponsor-grid-image{max-width:96px;max-height:24px}.vp-sponsor-grid.medium .vp-sponsor-grid-link{height:112px}.vp-sponsor-grid.medium .vp-sponsor-grid-image{max-width:120px;max-height:36px}.vp-sponsor-grid.big .vp-sponsor-grid-link{height:184px}.vp-sponsor-grid.big .vp-sponsor-grid-image{max-width:192px;max-height:56px}.vp-sponsor-grid[data-vp-grid="2"] .vp-sponsor-grid-item{width:calc((100% - 4px)/2)}.vp-sponsor-grid[data-vp-grid="3"] .vp-sponsor-grid-item{width:calc((100% - 4px * 2) / 3)}.vp-sponsor-grid[data-vp-grid="4"] .vp-sponsor-grid-item{width:calc((100% - 12px)/4)}.vp-sponsor-grid[data-vp-grid="5"] .vp-sponsor-grid-item{width:calc((100% - 16px)/5)}.vp-sponsor-grid[data-vp-grid="6"] .vp-sponsor-grid-item{width:calc((100% - 4px * 5) / 6)}.vp-sponsor-grid-item{flex-shrink:0;width:100%;background-color:var(--vp-c-bg-soft);transition:background-color .25s}.vp-sponsor-grid-item:hover{background-color:var(--vp-c-default-soft)}.vp-sponsor-grid-item:hover .vp-sponsor-grid-image{filter:grayscale(0) invert(0)}.vp-sponsor-grid-item.empty:hover{background-color:var(--vp-c-bg-soft)}.dark .vp-sponsor-grid-item:hover{background-color:var(--vp-c-white)}.dark .vp-sponsor-grid-item.empty:hover{background-color:var(--vp-c-bg-soft)}.vp-sponsor-grid-link{display:flex}.vp-sponsor-grid-box{display:flex;justify-content:center;align-items:center;width:100%}.vp-sponsor-grid-image{max-width:100%;filter:grayscale(1);transition:filter .25s}.dark .vp-sponsor-grid-image{filter:grayscale(1) invert(1)}.VPBadge{display:inline-block;margin-left:2px;border:1px solid transparent;border-radius:12px;padding:0 10px;line-height:22px;font-size:12px;font-weight:500;transform:translateY(-2px)}.VPBadge.small{padding:0 6px;line-height:18px;font-size:10px;transform:translateY(-8px)}.VPDocFooter .VPBadge{display:none}.vp-doc h1>.VPBadge{margin-top:4px;vertical-align:top}.vp-doc h2>.VPBadge{margin-top:3px;padding:0 8px;vertical-align:top}.vp-doc h3>.VPBadge{vertical-align:middle}.vp-doc h4>.VPBadge,.vp-doc h5>.VPBadge,.vp-doc h6>.VPBadge{vertical-align:middle;line-height:18px}.VPBadge.info{border-color:var(--vp-badge-info-border);color:var(--vp-badge-info-text);background-color:var(--vp-badge-info-bg)}.VPBadge.tip{border-color:var(--vp-badge-tip-border);color:var(--vp-badge-tip-text);background-color:var(--vp-badge-tip-bg)}.VPBadge.warning{border-color:var(--vp-badge-warning-border);color:var(--vp-badge-warning-text);background-color:var(--vp-badge-warning-bg)}.VPBadge.danger{border-color:var(--vp-badge-danger-border);color:var(--vp-badge-danger-text);background-color:var(--vp-badge-danger-bg)}.VPBackdrop[data-v-b06cdb19]{position:fixed;top:0;right:0;bottom:0;left:0;z-index:var(--vp-z-index-backdrop);background:var(--vp-backdrop-bg-color);transition:opacity .5s}.VPBackdrop.fade-enter-from[data-v-b06cdb19],.VPBackdrop.fade-leave-to[data-v-b06cdb19]{opacity:0}.VPBackdrop.fade-leave-active[data-v-b06cdb19]{transition-duration:.25s}@media (min-width: 1280px){.VPBackdrop[data-v-b06cdb19]{display:none}}.NotFound[data-v-951cab6c]{padding:64px 24px 96px;text-align:center}@media (min-width: 768px){.NotFound[data-v-951cab6c]{padding:96px 32px 168px}}.code[data-v-951cab6c]{line-height:64px;font-size:64px;font-weight:600}.title[data-v-951cab6c]{padding-top:12px;letter-spacing:2px;line-height:20px;font-size:20px;font-weight:700}.divider[data-v-951cab6c]{margin:24px auto 18px;width:64px;height:1px;background-color:var(--vp-c-divider)}.quote[data-v-951cab6c]{margin:0 auto;max-width:256px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}.action[data-v-951cab6c]{padding-top:20px}.link[data-v-951cab6c]{display:inline-block;border:1px solid var(--vp-c-brand-1);border-radius:16px;padding:3px 16px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1);transition:border-color .25s,color .25s}.link[data-v-951cab6c]:hover{border-color:var(--vp-c-brand-2);color:var(--vp-c-brand-2)}.root[data-v-3f927ebe]{position:relative;z-index:1}.nested[data-v-3f927ebe]{padding-right:16px;padding-left:16px}.outline-link[data-v-3f927ebe]{display:block;line-height:32px;font-size:14px;font-weight:400;color:var(--vp-c-text-2);white-space:nowrap;overflow:hidden;text-overflow:ellipsis;transition:color .5s}.outline-link[data-v-3f927ebe]:hover,.outline-link.active[data-v-3f927ebe]{color:var(--vp-c-text-1);transition:color .25s}.outline-link.nested[data-v-3f927ebe]{padding-left:13px}.VPDocAsideOutline[data-v-b38bf2ff]{display:none}.VPDocAsideOutline.has-outline[data-v-b38bf2ff]{display:block}.content[data-v-b38bf2ff]{position:relative;border-left:1px solid var(--vp-c-divider);padding-left:16px;font-size:13px;font-weight:500}.outline-marker[data-v-b38bf2ff]{position:absolute;top:32px;left:-1px;z-index:0;opacity:0;width:2px;border-radius:2px;height:18px;background-color:var(--vp-c-brand-1);transition:top .25s cubic-bezier(0,1,.5,1),background-color .5s,opacity .25s}.outline-title[data-v-b38bf2ff]{line-height:32px;font-size:14px;font-weight:600}.VPDocAside[data-v-6d7b3c46]{display:flex;flex-direction:column;flex-grow:1}.spacer[data-v-6d7b3c46]{flex-grow:1}.VPDocAside[data-v-6d7b3c46] .spacer+.VPDocAsideSponsors,.VPDocAside[data-v-6d7b3c46] .spacer+.VPDocAsideCarbonAds{margin-top:24px}.VPDocAside[data-v-6d7b3c46] .VPDocAsideSponsors+.VPDocAsideCarbonAds{margin-top:16px}.VPLastUpdated[data-v-475f71b8]{line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}@media (min-width: 640px){.VPLastUpdated[data-v-475f71b8]{line-height:32px;font-size:14px;font-weight:500}}.VPDocFooter[data-v-4f9813fa]{margin-top:64px}.edit-info[data-v-4f9813fa]{padding-bottom:18px}@media (min-width: 640px){.edit-info[data-v-4f9813fa]{display:flex;justify-content:space-between;align-items:center;padding-bottom:14px}}.edit-link-button[data-v-4f9813fa]{display:flex;align-items:center;border:0;line-height:32px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1);transition:color .25s}.edit-link-button[data-v-4f9813fa]:hover{color:var(--vp-c-brand-2)}.edit-link-icon[data-v-4f9813fa]{margin-right:8px}.prev-next[data-v-4f9813fa]{border-top:1px solid var(--vp-c-divider);padding-top:24px;display:grid;grid-row-gap:8px}@media (min-width: 640px){.prev-next[data-v-4f9813fa]{grid-template-columns:repeat(2,1fr);grid-column-gap:16px}}.pager-link[data-v-4f9813fa]{display:block;border:1px solid var(--vp-c-divider);border-radius:8px;padding:11px 16px 13px;width:100%;height:100%;transition:border-color .25s}.pager-link[data-v-4f9813fa]:hover{border-color:var(--vp-c-brand-1)}.pager-link.next[data-v-4f9813fa]{margin-left:auto;text-align:right}.desc[data-v-4f9813fa]{display:block;line-height:20px;font-size:12px;font-weight:500;color:var(--vp-c-text-2)}.title[data-v-4f9813fa]{display:block;line-height:20px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1);transition:color .25s}.VPDoc[data-v-83890dd9]{padding:32px 24px 96px;width:100%}@media (min-width: 768px){.VPDoc[data-v-83890dd9]{padding:48px 32px 128px}}@media (min-width: 960px){.VPDoc[data-v-83890dd9]{padding:48px 32px 0}.VPDoc:not(.has-sidebar) .container[data-v-83890dd9]{display:flex;justify-content:center;max-width:992px}.VPDoc:not(.has-sidebar) .content[data-v-83890dd9]{max-width:752px}}@media (min-width: 1280px){.VPDoc .container[data-v-83890dd9]{display:flex;justify-content:center}.VPDoc .aside[data-v-83890dd9]{display:block}}@media (min-width: 1440px){.VPDoc:not(.has-sidebar) .content[data-v-83890dd9]{max-width:784px}.VPDoc:not(.has-sidebar) .container[data-v-83890dd9]{max-width:1104px}}.container[data-v-83890dd9]{margin:0 auto;width:100%}.aside[data-v-83890dd9]{position:relative;display:none;order:2;flex-grow:1;padding-left:32px;width:100%;max-width:256px}.left-aside[data-v-83890dd9]{order:1;padding-left:unset;padding-right:32px}.aside-container[data-v-83890dd9]{position:fixed;top:0;padding-top:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + var(--vp-doc-top-height, 0px) + 48px);width:224px;height:100vh;overflow-x:hidden;overflow-y:auto;scrollbar-width:none}.aside-container[data-v-83890dd9]::-webkit-scrollbar{display:none}.aside-curtain[data-v-83890dd9]{position:fixed;bottom:0;z-index:10;width:224px;height:32px;background:linear-gradient(transparent,var(--vp-c-bg) 70%)}.aside-content[data-v-83890dd9]{display:flex;flex-direction:column;min-height:calc(100vh - (var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 48px));padding-bottom:32px}.content[data-v-83890dd9]{position:relative;margin:0 auto;width:100%}@media (min-width: 960px){.content[data-v-83890dd9]{padding:0 32px 128px}}@media (min-width: 1280px){.content[data-v-83890dd9]{order:1;margin:0;min-width:640px}}.content-container[data-v-83890dd9]{margin:0 auto}.VPDoc.has-aside .content-container[data-v-83890dd9]{max-width:688px}.VPButton[data-v-906d7fb4]{display:inline-block;border:1px solid transparent;text-align:center;font-weight:600;white-space:nowrap;transition:color .25s,border-color .25s,background-color .25s}.VPButton[data-v-906d7fb4]:active{transition:color .1s,border-color .1s,background-color .1s}.VPButton.medium[data-v-906d7fb4]{border-radius:20px;padding:0 20px;line-height:38px;font-size:14px}.VPButton.big[data-v-906d7fb4]{border-radius:24px;padding:0 24px;line-height:46px;font-size:16px}.VPButton.brand[data-v-906d7fb4]{border-color:var(--vp-button-brand-border);color:var(--vp-button-brand-text);background-color:var(--vp-button-brand-bg)}.VPButton.brand[data-v-906d7fb4]:hover{border-color:var(--vp-button-brand-hover-border);color:var(--vp-button-brand-hover-text);background-color:var(--vp-button-brand-hover-bg)}.VPButton.brand[data-v-906d7fb4]:active{border-color:var(--vp-button-brand-active-border);color:var(--vp-button-brand-active-text);background-color:var(--vp-button-brand-active-bg)}.VPButton.alt[data-v-906d7fb4]{border-color:var(--vp-button-alt-border);color:var(--vp-button-alt-text);background-color:var(--vp-button-alt-bg)}.VPButton.alt[data-v-906d7fb4]:hover{border-color:var(--vp-button-alt-hover-border);color:var(--vp-button-alt-hover-text);background-color:var(--vp-button-alt-hover-bg)}.VPButton.alt[data-v-906d7fb4]:active{border-color:var(--vp-button-alt-active-border);color:var(--vp-button-alt-active-text);background-color:var(--vp-button-alt-active-bg)}.VPButton.sponsor[data-v-906d7fb4]{border-color:var(--vp-button-sponsor-border);color:var(--vp-button-sponsor-text);background-color:var(--vp-button-sponsor-bg)}.VPButton.sponsor[data-v-906d7fb4]:hover{border-color:var(--vp-button-sponsor-hover-border);color:var(--vp-button-sponsor-hover-text);background-color:var(--vp-button-sponsor-hover-bg)}.VPButton.sponsor[data-v-906d7fb4]:active{border-color:var(--vp-button-sponsor-active-border);color:var(--vp-button-sponsor-active-text);background-color:var(--vp-button-sponsor-active-bg)}html:not(.dark) .VPImage.dark[data-v-35a7d0b8]{display:none}.dark .VPImage.light[data-v-35a7d0b8]{display:none}.VPHero[data-v-955009fc]{margin-top:calc((var(--vp-nav-height) + var(--vp-layout-top-height, 0px)) * -1);padding:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 48px) 24px 48px}@media (min-width: 640px){.VPHero[data-v-955009fc]{padding:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 80px) 48px 64px}}@media (min-width: 960px){.VPHero[data-v-955009fc]{padding:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 80px) 64px 64px}}.container[data-v-955009fc]{display:flex;flex-direction:column;margin:0 auto;max-width:1152px}@media (min-width: 960px){.container[data-v-955009fc]{flex-direction:row}}.main[data-v-955009fc]{position:relative;z-index:10;order:2;flex-grow:1;flex-shrink:0}.VPHero.has-image .container[data-v-955009fc]{text-align:center}@media (min-width: 960px){.VPHero.has-image .container[data-v-955009fc]{text-align:left}}@media (min-width: 960px){.main[data-v-955009fc]{order:1;width:calc((100% / 3) * 2)}.VPHero.has-image .main[data-v-955009fc]{max-width:592px}}.name[data-v-955009fc],.text[data-v-955009fc]{max-width:392px;letter-spacing:-.4px;line-height:40px;font-size:32px;font-weight:700;white-space:pre-wrap}.VPHero.has-image .name[data-v-955009fc],.VPHero.has-image .text[data-v-955009fc]{margin:0 auto}.name[data-v-955009fc]{color:var(--vp-home-hero-name-color)}.clip[data-v-955009fc]{background:var(--vp-home-hero-name-background);-webkit-background-clip:text;background-clip:text;-webkit-text-fill-color:var(--vp-home-hero-name-color)}@media (min-width: 640px){.name[data-v-955009fc],.text[data-v-955009fc]{max-width:576px;line-height:56px;font-size:48px}}@media (min-width: 960px){.name[data-v-955009fc],.text[data-v-955009fc]{line-height:64px;font-size:56px}.VPHero.has-image .name[data-v-955009fc],.VPHero.has-image .text[data-v-955009fc]{margin:0}}.tagline[data-v-955009fc]{padding-top:8px;max-width:392px;line-height:28px;font-size:18px;font-weight:500;white-space:pre-wrap;color:var(--vp-c-text-2)}.VPHero.has-image .tagline[data-v-955009fc]{margin:0 auto}@media (min-width: 640px){.tagline[data-v-955009fc]{padding-top:12px;max-width:576px;line-height:32px;font-size:20px}}@media (min-width: 960px){.tagline[data-v-955009fc]{line-height:36px;font-size:24px}.VPHero.has-image .tagline[data-v-955009fc]{margin:0}}.actions[data-v-955009fc]{display:flex;flex-wrap:wrap;margin:-6px;padding-top:24px}.VPHero.has-image .actions[data-v-955009fc]{justify-content:center}@media (min-width: 640px){.actions[data-v-955009fc]{padding-top:32px}}@media (min-width: 960px){.VPHero.has-image .actions[data-v-955009fc]{justify-content:flex-start}}.action[data-v-955009fc]{flex-shrink:0;padding:6px}.image[data-v-955009fc]{order:1;margin:-76px -24px -48px}@media (min-width: 640px){.image[data-v-955009fc]{margin:-108px -24px -48px}}@media (min-width: 960px){.image[data-v-955009fc]{flex-grow:1;order:2;margin:0;min-height:100%}}.image-container[data-v-955009fc]{position:relative;margin:0 auto;width:320px;height:320px}@media (min-width: 640px){.image-container[data-v-955009fc]{width:392px;height:392px}}@media (min-width: 960px){.image-container[data-v-955009fc]{display:flex;justify-content:center;align-items:center;width:100%;height:100%;transform:translate(-32px,-32px)}}.image-bg[data-v-955009fc]{position:absolute;top:50%;left:50%;border-radius:50%;width:192px;height:192px;background-image:var(--vp-home-hero-image-background-image);filter:var(--vp-home-hero-image-filter);transform:translate(-50%,-50%)}@media (min-width: 640px){.image-bg[data-v-955009fc]{width:256px;height:256px}}@media (min-width: 960px){.image-bg[data-v-955009fc]{width:320px;height:320px}}[data-v-955009fc] .image-src{position:absolute;top:50%;left:50%;max-width:192px;max-height:192px;transform:translate(-50%,-50%)}@media (min-width: 640px){[data-v-955009fc] .image-src{max-width:256px;max-height:256px}}@media (min-width: 960px){[data-v-955009fc] .image-src{max-width:320px;max-height:320px}}.VPFeature[data-v-f5e9645b]{display:block;border:1px solid var(--vp-c-bg-soft);border-radius:12px;height:100%;background-color:var(--vp-c-bg-soft);transition:border-color .25s,background-color .25s}.VPFeature.link[data-v-f5e9645b]:hover{border-color:var(--vp-c-brand-1)}.box[data-v-f5e9645b]{display:flex;flex-direction:column;padding:24px;height:100%}.box[data-v-f5e9645b]>.VPImage{margin-bottom:20px}.icon[data-v-f5e9645b]{display:flex;justify-content:center;align-items:center;margin-bottom:20px;border-radius:6px;background-color:var(--vp-c-default-soft);width:48px;height:48px;font-size:24px;transition:background-color .25s}.title[data-v-f5e9645b]{line-height:24px;font-size:16px;font-weight:600}.details[data-v-f5e9645b]{flex-grow:1;padding-top:8px;line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}.link-text[data-v-f5e9645b]{padding-top:8px}.link-text-value[data-v-f5e9645b]{display:flex;align-items:center;font-size:14px;font-weight:500;color:var(--vp-c-brand-1)}.link-text-icon[data-v-f5e9645b]{margin-left:6px}.VPFeatures[data-v-d0a190d7]{position:relative;padding:0 24px}@media (min-width: 640px){.VPFeatures[data-v-d0a190d7]{padding:0 48px}}@media (min-width: 960px){.VPFeatures[data-v-d0a190d7]{padding:0 64px}}.container[data-v-d0a190d7]{margin:0 auto;max-width:1152px}.items[data-v-d0a190d7]{display:flex;flex-wrap:wrap;margin:-8px}.item[data-v-d0a190d7]{padding:8px;width:100%}@media (min-width: 640px){.item.grid-2[data-v-d0a190d7],.item.grid-4[data-v-d0a190d7],.item.grid-6[data-v-d0a190d7]{width:50%}}@media (min-width: 768px){.item.grid-2[data-v-d0a190d7],.item.grid-4[data-v-d0a190d7]{width:50%}.item.grid-3[data-v-d0a190d7],.item.grid-6[data-v-d0a190d7]{width:calc(100% / 3)}}@media (min-width: 960px){.item.grid-4[data-v-d0a190d7]{width:25%}}.container[data-v-7a48a447]{margin:auto;width:100%;max-width:1280px;padding:0 24px}@media (min-width: 640px){.container[data-v-7a48a447]{padding:0 48px}}@media (min-width: 960px){.container[data-v-7a48a447]{width:100%;padding:0 64px}}.vp-doc[data-v-7a48a447] .VPHomeSponsors,.vp-doc[data-v-7a48a447] .VPTeamPage{margin-left:var(--vp-offset, calc(50% - 50vw) );margin-right:var(--vp-offset, calc(50% - 50vw) )}.vp-doc[data-v-7a48a447] .VPHomeSponsors h2{border-top:none;letter-spacing:normal}.vp-doc[data-v-7a48a447] .VPHomeSponsors a,.vp-doc[data-v-7a48a447] .VPTeamPage a{text-decoration:none}.VPHome[data-v-cbb6ec48]{margin-bottom:96px}@media (min-width: 768px){.VPHome[data-v-cbb6ec48]{margin-bottom:128px}}.VPContent[data-v-91765379]{flex-grow:1;flex-shrink:0;margin:var(--vp-layout-top-height, 0px) auto 0;width:100%}.VPContent.is-home[data-v-91765379]{width:100%;max-width:100%}.VPContent.has-sidebar[data-v-91765379]{margin:0}@media (min-width: 960px){.VPContent[data-v-91765379]{padding-top:var(--vp-nav-height)}.VPContent.has-sidebar[data-v-91765379]{margin:var(--vp-layout-top-height, 0px) 0 0;padding-left:var(--vp-sidebar-width)}}@media (min-width: 1440px){.VPContent.has-sidebar[data-v-91765379]{padding-right:calc((100vw - var(--vp-layout-max-width)) / 2);padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.VPFooter[data-v-c970a860]{position:relative;z-index:var(--vp-z-index-footer);border-top:1px solid var(--vp-c-gutter);padding:32px 24px;background-color:var(--vp-c-bg)}.VPFooter.has-sidebar[data-v-c970a860]{display:none}.VPFooter[data-v-c970a860] a{text-decoration-line:underline;text-underline-offset:2px;transition:color .25s}.VPFooter[data-v-c970a860] a:hover{color:var(--vp-c-text-1)}@media (min-width: 768px){.VPFooter[data-v-c970a860]{padding:32px}}.container[data-v-c970a860]{margin:0 auto;max-width:var(--vp-layout-max-width);text-align:center}.message[data-v-c970a860],.copyright[data-v-c970a860]{line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}.VPLocalNavOutlineDropdown[data-v-bc9dc845]{padding:12px 20px 11px}@media (min-width: 960px){.VPLocalNavOutlineDropdown[data-v-bc9dc845]{padding:12px 36px 11px}}.VPLocalNavOutlineDropdown button[data-v-bc9dc845]{display:block;font-size:12px;font-weight:500;line-height:24px;color:var(--vp-c-text-2);transition:color .5s;position:relative}.VPLocalNavOutlineDropdown button[data-v-bc9dc845]:hover{color:var(--vp-c-text-1);transition:color .25s}.VPLocalNavOutlineDropdown button.open[data-v-bc9dc845]{color:var(--vp-c-text-1)}.icon[data-v-bc9dc845]{display:inline-block;vertical-align:middle;margin-left:2px;font-size:14px;transform:rotate(0);transition:transform .25s}@media (min-width: 960px){.VPLocalNavOutlineDropdown button[data-v-bc9dc845]{font-size:14px}.icon[data-v-bc9dc845]{font-size:16px}}.open>.icon[data-v-bc9dc845]{transform:rotate(90deg)}.items[data-v-bc9dc845]{position:absolute;top:40px;right:16px;left:16px;display:grid;gap:1px;border:1px solid var(--vp-c-border);border-radius:8px;background-color:var(--vp-c-gutter);max-height:calc(var(--vp-vh, 100vh) - 86px);overflow:hidden auto;box-shadow:var(--vp-shadow-3)}@media (min-width: 960px){.items[data-v-bc9dc845]{right:auto;left:calc(var(--vp-sidebar-width) + 32px);width:320px}}.header[data-v-bc9dc845]{background-color:var(--vp-c-bg-soft)}.top-link[data-v-bc9dc845]{display:block;padding:0 16px;line-height:48px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1)}.outline[data-v-bc9dc845]{padding:8px 0;background-color:var(--vp-c-bg-soft)}.flyout-enter-active[data-v-bc9dc845]{transition:all .2s ease-out}.flyout-leave-active[data-v-bc9dc845]{transition:all .15s ease-in}.flyout-enter-from[data-v-bc9dc845],.flyout-leave-to[data-v-bc9dc845]{opacity:0;transform:translateY(-16px)}.VPLocalNav[data-v-070ab83d]{position:sticky;top:0;left:0;z-index:var(--vp-z-index-local-nav);border-bottom:1px solid var(--vp-c-gutter);padding-top:var(--vp-layout-top-height, 0px);width:100%;background-color:var(--vp-local-nav-bg-color)}.VPLocalNav.fixed[data-v-070ab83d]{position:fixed}@media (min-width: 960px){.VPLocalNav[data-v-070ab83d]{top:var(--vp-nav-height)}.VPLocalNav.has-sidebar[data-v-070ab83d]{padding-left:var(--vp-sidebar-width)}.VPLocalNav.empty[data-v-070ab83d]{display:none}}@media (min-width: 1280px){.VPLocalNav[data-v-070ab83d]{display:none}}@media (min-width: 1440px){.VPLocalNav.has-sidebar[data-v-070ab83d]{padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.container[data-v-070ab83d]{display:flex;justify-content:space-between;align-items:center}.menu[data-v-070ab83d]{display:flex;align-items:center;padding:12px 24px 11px;line-height:24px;font-size:12px;font-weight:500;color:var(--vp-c-text-2);transition:color .5s}.menu[data-v-070ab83d]:hover{color:var(--vp-c-text-1);transition:color .25s}@media (min-width: 768px){.menu[data-v-070ab83d]{padding:0 32px}}@media (min-width: 960px){.menu[data-v-070ab83d]{display:none}}.menu-icon[data-v-070ab83d]{margin-right:8px;font-size:14px}.VPOutlineDropdown[data-v-070ab83d]{padding:12px 24px 11px}@media (min-width: 768px){.VPOutlineDropdown[data-v-070ab83d]{padding:12px 32px 11px}}.VPSwitch[data-v-4a1c76db]{position:relative;border-radius:11px;display:block;width:40px;height:22px;flex-shrink:0;border:1px solid var(--vp-input-border-color);background-color:var(--vp-input-switch-bg-color);transition:border-color .25s!important}.VPSwitch[data-v-4a1c76db]:hover{border-color:var(--vp-c-brand-1)}.check[data-v-4a1c76db]{position:absolute;top:1px;left:1px;width:18px;height:18px;border-radius:50%;background-color:var(--vp-c-neutral-inverse);box-shadow:var(--vp-shadow-1);transition:transform .25s!important}.icon[data-v-4a1c76db]{position:relative;display:block;width:18px;height:18px;border-radius:50%;overflow:hidden}.icon[data-v-4a1c76db] [class^=vpi-]{position:absolute;top:3px;left:3px;width:12px;height:12px;color:var(--vp-c-text-2)}.dark .icon[data-v-4a1c76db] [class^=vpi-]{color:var(--vp-c-text-1);transition:opacity .25s!important}.sun[data-v-e40a8bb6]{opacity:1}.moon[data-v-e40a8bb6],.dark .sun[data-v-e40a8bb6]{opacity:0}.dark .moon[data-v-e40a8bb6]{opacity:1}.dark .VPSwitchAppearance[data-v-e40a8bb6] .check{transform:translate(18px)}.VPNavBarAppearance[data-v-af096f4a]{display:none}@media (min-width: 1280px){.VPNavBarAppearance[data-v-af096f4a]{display:flex;align-items:center}}.VPMenuGroup+.VPMenuLink[data-v-acbfed09]{margin:12px -12px 0;border-top:1px solid var(--vp-c-divider);padding:12px 12px 0}.link[data-v-acbfed09]{display:block;border-radius:6px;padding:0 12px;line-height:32px;font-size:14px;font-weight:500;color:var(--vp-c-text-1);white-space:nowrap;transition:background-color .25s,color .25s}.link[data-v-acbfed09]:hover{color:var(--vp-c-brand-1);background-color:var(--vp-c-default-soft)}.link.active[data-v-acbfed09]{color:var(--vp-c-brand-1)}.VPMenuGroup[data-v-48c802d0]{margin:12px -12px 0;border-top:1px solid var(--vp-c-divider);padding:12px 12px 0}.VPMenuGroup[data-v-48c802d0]:first-child{margin-top:0;border-top:0;padding-top:0}.VPMenuGroup+.VPMenuGroup[data-v-48c802d0]{margin-top:12px;border-top:1px solid var(--vp-c-divider)}.title[data-v-48c802d0]{padding:0 12px;line-height:32px;font-size:14px;font-weight:600;color:var(--vp-c-text-2);white-space:nowrap;transition:color .25s}.VPMenu[data-v-7dd3104a]{border-radius:12px;padding:12px;min-width:128px;border:1px solid var(--vp-c-divider);background-color:var(--vp-c-bg-elv);box-shadow:var(--vp-shadow-3);transition:background-color .5s;max-height:calc(100vh - var(--vp-nav-height));overflow-y:auto}.VPMenu[data-v-7dd3104a] .group{margin:0 -12px;padding:0 12px 12px}.VPMenu[data-v-7dd3104a] .group+.group{border-top:1px solid var(--vp-c-divider);padding:11px 12px 12px}.VPMenu[data-v-7dd3104a] .group:last-child{padding-bottom:0}.VPMenu[data-v-7dd3104a] .group+.item{border-top:1px solid var(--vp-c-divider);padding:11px 16px 0}.VPMenu[data-v-7dd3104a] .item{padding:0 16px;white-space:nowrap}.VPMenu[data-v-7dd3104a] .label{flex-grow:1;line-height:28px;font-size:12px;font-weight:500;color:var(--vp-c-text-2);transition:color .5s}.VPMenu[data-v-7dd3104a] .action{padding-left:24px}.VPFlyout[data-v-04f5c5e9]{position:relative}.VPFlyout[data-v-04f5c5e9]:hover{color:var(--vp-c-brand-1);transition:color .25s}.VPFlyout:hover .text[data-v-04f5c5e9]{color:var(--vp-c-text-2)}.VPFlyout:hover .icon[data-v-04f5c5e9]{fill:var(--vp-c-text-2)}.VPFlyout.active .text[data-v-04f5c5e9]{color:var(--vp-c-brand-1)}.VPFlyout.active:hover .text[data-v-04f5c5e9]{color:var(--vp-c-brand-2)}.button[aria-expanded=false]+.menu[data-v-04f5c5e9]{opacity:0;visibility:hidden;transform:translateY(0)}.VPFlyout:hover .menu[data-v-04f5c5e9],.button[aria-expanded=true]+.menu[data-v-04f5c5e9]{opacity:1;visibility:visible;transform:translateY(0)}.button[data-v-04f5c5e9]{display:flex;align-items:center;padding:0 12px;height:var(--vp-nav-height);color:var(--vp-c-text-1);transition:color .5s}.text[data-v-04f5c5e9]{display:flex;align-items:center;line-height:var(--vp-nav-height);font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:color .25s}.option-icon[data-v-04f5c5e9]{margin-right:0;font-size:16px}.text-icon[data-v-04f5c5e9]{margin-left:4px;font-size:14px}.icon[data-v-04f5c5e9]{font-size:20px;transition:fill .25s}.menu[data-v-04f5c5e9]{position:absolute;top:calc(var(--vp-nav-height) / 2 + 20px);right:0;opacity:0;visibility:hidden;transition:opacity .25s,visibility .25s,transform .25s}.VPSocialLink[data-v-717b8b75]{display:flex;justify-content:center;align-items:center;width:36px;height:36px;color:var(--vp-c-text-2);transition:color .5s}.VPSocialLink[data-v-717b8b75]:hover{color:var(--vp-c-text-1);transition:color .25s}.VPSocialLink[data-v-717b8b75]>svg,.VPSocialLink[data-v-717b8b75]>[class^=vpi-social-]{width:20px;height:20px;fill:currentColor}.VPSocialLinks[data-v-ee7a9424]{display:flex;justify-content:center}.VPNavBarExtra[data-v-925effce]{display:none;margin-right:-12px}@media (min-width: 768px){.VPNavBarExtra[data-v-925effce]{display:block}}@media (min-width: 1280px){.VPNavBarExtra[data-v-925effce]{display:none}}.trans-title[data-v-925effce]{padding:0 24px 0 12px;line-height:32px;font-size:14px;font-weight:700;color:var(--vp-c-text-1)}.item.appearance[data-v-925effce],.item.social-links[data-v-925effce]{display:flex;align-items:center;padding:0 12px}.item.appearance[data-v-925effce]{min-width:176px}.appearance-action[data-v-925effce]{margin-right:-2px}.social-links-list[data-v-925effce]{margin:-4px -8px}.VPNavBarHamburger[data-v-5dea55bf]{display:flex;justify-content:center;align-items:center;width:48px;height:var(--vp-nav-height)}@media (min-width: 768px){.VPNavBarHamburger[data-v-5dea55bf]{display:none}}.container[data-v-5dea55bf]{position:relative;width:16px;height:14px;overflow:hidden}.VPNavBarHamburger:hover .top[data-v-5dea55bf]{top:0;left:0;transform:translate(4px)}.VPNavBarHamburger:hover .middle[data-v-5dea55bf]{top:6px;left:0;transform:translate(0)}.VPNavBarHamburger:hover .bottom[data-v-5dea55bf]{top:12px;left:0;transform:translate(8px)}.VPNavBarHamburger.active .top[data-v-5dea55bf]{top:6px;transform:translate(0) rotate(225deg)}.VPNavBarHamburger.active .middle[data-v-5dea55bf]{top:6px;transform:translate(16px)}.VPNavBarHamburger.active .bottom[data-v-5dea55bf]{top:6px;transform:translate(0) rotate(135deg)}.VPNavBarHamburger.active:hover .top[data-v-5dea55bf],.VPNavBarHamburger.active:hover .middle[data-v-5dea55bf],.VPNavBarHamburger.active:hover .bottom[data-v-5dea55bf]{background-color:var(--vp-c-text-2);transition:top .25s,background-color .25s,transform .25s}.top[data-v-5dea55bf],.middle[data-v-5dea55bf],.bottom[data-v-5dea55bf]{position:absolute;width:16px;height:2px;background-color:var(--vp-c-text-1);transition:top .25s,background-color .5s,transform .25s}.top[data-v-5dea55bf]{top:0;left:0;transform:translate(0)}.middle[data-v-5dea55bf]{top:6px;left:0;transform:translate(8px)}.bottom[data-v-5dea55bf]{top:12px;left:0;transform:translate(4px)}.VPNavBarMenuLink[data-v-956ec74c]{display:flex;align-items:center;padding:0 12px;line-height:var(--vp-nav-height);font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:color .25s}.VPNavBarMenuLink.active[data-v-956ec74c],.VPNavBarMenuLink[data-v-956ec74c]:hover{color:var(--vp-c-brand-1)}.VPNavBarMenu[data-v-e6d46098]{display:none}@media (min-width: 768px){.VPNavBarMenu[data-v-e6d46098]{display:flex}}/*! @docsearch/css 3.6.2 | MIT License | © Algolia, Inc. and contributors | https://docsearch.algolia.com */:root{--docsearch-primary-color:#5468ff;--docsearch-text-color:#1c1e21;--docsearch-spacing:12px;--docsearch-icon-stroke-width:1.4;--docsearch-highlight-color:var(--docsearch-primary-color);--docsearch-muted-color:#969faf;--docsearch-container-background:rgba(101,108,133,.8);--docsearch-logo-color:#5468ff;--docsearch-modal-width:560px;--docsearch-modal-height:600px;--docsearch-modal-background:#f5f6f7;--docsearch-modal-shadow:inset 1px 1px 0 0 hsla(0,0%,100%,.5),0 3px 8px 0 #555a64;--docsearch-searchbox-height:56px;--docsearch-searchbox-background:#ebedf0;--docsearch-searchbox-focus-background:#fff;--docsearch-searchbox-shadow:inset 0 0 0 2px var(--docsearch-primary-color);--docsearch-hit-height:56px;--docsearch-hit-color:#444950;--docsearch-hit-active-color:#fff;--docsearch-hit-background:#fff;--docsearch-hit-shadow:0 1px 3px 0 #d4d9e1;--docsearch-key-gradient:linear-gradient(-225deg,#d5dbe4,#f8f8f8);--docsearch-key-shadow:inset 0 -2px 0 0 #cdcde6,inset 0 0 1px 1px #fff,0 1px 2px 1px rgba(30,35,90,.4);--docsearch-key-pressed-shadow:inset 0 -2px 0 0 #cdcde6,inset 0 0 1px 1px #fff,0 1px 1px 0 rgba(30,35,90,.4);--docsearch-footer-height:44px;--docsearch-footer-background:#fff;--docsearch-footer-shadow:0 -1px 0 0 #e0e3e8,0 -3px 6px 0 rgba(69,98,155,.12)}html[data-theme=dark]{--docsearch-text-color:#f5f6f7;--docsearch-container-background:rgba(9,10,17,.8);--docsearch-modal-background:#15172a;--docsearch-modal-shadow:inset 1px 1px 0 0 #2c2e40,0 3px 8px 0 #000309;--docsearch-searchbox-background:#090a11;--docsearch-searchbox-focus-background:#000;--docsearch-hit-color:#bec3c9;--docsearch-hit-shadow:none;--docsearch-hit-background:#090a11;--docsearch-key-gradient:linear-gradient(-26.5deg,#565872,#31355b);--docsearch-key-shadow:inset 0 -2px 0 0 #282d55,inset 0 0 1px 1px #51577d,0 2px 2px 0 rgba(3,4,9,.3);--docsearch-key-pressed-shadow:inset 0 -2px 0 0 #282d55,inset 0 0 1px 1px #51577d,0 1px 1px 0 rgba(3,4,9,.30196078431372547);--docsearch-footer-background:#1e2136;--docsearch-footer-shadow:inset 0 1px 0 0 rgba(73,76,106,.5),0 -4px 8px 0 rgba(0,0,0,.2);--docsearch-logo-color:#fff;--docsearch-muted-color:#7f8497}.DocSearch-Button{align-items:center;background:var(--docsearch-searchbox-background);border:0;border-radius:40px;color:var(--docsearch-muted-color);cursor:pointer;display:flex;font-weight:500;height:36px;justify-content:space-between;margin:0 0 0 16px;padding:0 8px;-webkit-user-select:none;user-select:none}.DocSearch-Button:active,.DocSearch-Button:focus,.DocSearch-Button:hover{background:var(--docsearch-searchbox-focus-background);box-shadow:var(--docsearch-searchbox-shadow);color:var(--docsearch-text-color);outline:none}.DocSearch-Button-Container{align-items:center;display:flex}.DocSearch-Search-Icon{stroke-width:1.6}.DocSearch-Button .DocSearch-Search-Icon{color:var(--docsearch-text-color)}.DocSearch-Button-Placeholder{font-size:1rem;padding:0 12px 0 6px}.DocSearch-Button-Keys{display:flex;min-width:calc(40px + .8em)}.DocSearch-Button-Key{align-items:center;background:var(--docsearch-key-gradient);border-radius:3px;box-shadow:var(--docsearch-key-shadow);color:var(--docsearch-muted-color);display:flex;height:18px;justify-content:center;margin-right:.4em;position:relative;padding:0 0 2px;border:0;top:-1px;width:20px}.DocSearch-Button-Key--pressed{transform:translate3d(0,1px,0);box-shadow:var(--docsearch-key-pressed-shadow)}@media (max-width:768px){.DocSearch-Button-Keys,.DocSearch-Button-Placeholder{display:none}}.DocSearch--active{overflow:hidden!important}.DocSearch-Container,.DocSearch-Container *{box-sizing:border-box}.DocSearch-Container{background-color:var(--docsearch-container-background);height:100vh;left:0;position:fixed;top:0;width:100vw;z-index:200}.DocSearch-Container a{text-decoration:none}.DocSearch-Link{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;color:var(--docsearch-highlight-color);cursor:pointer;font:inherit;margin:0;padding:0}.DocSearch-Modal{background:var(--docsearch-modal-background);border-radius:6px;box-shadow:var(--docsearch-modal-shadow);flex-direction:column;margin:60px auto auto;max-width:var(--docsearch-modal-width);position:relative}.DocSearch-SearchBar{display:flex;padding:var(--docsearch-spacing) var(--docsearch-spacing) 0}.DocSearch-Form{align-items:center;background:var(--docsearch-searchbox-focus-background);border-radius:4px;box-shadow:var(--docsearch-searchbox-shadow);display:flex;height:var(--docsearch-searchbox-height);margin:0;padding:0 var(--docsearch-spacing);position:relative;width:100%}.DocSearch-Input{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:transparent;border:0;color:var(--docsearch-text-color);flex:1;font:inherit;font-size:1.2em;height:100%;outline:none;padding:0 0 0 8px;width:80%}.DocSearch-Input::placeholder{color:var(--docsearch-muted-color);opacity:1}.DocSearch-Input::-webkit-search-cancel-button,.DocSearch-Input::-webkit-search-decoration,.DocSearch-Input::-webkit-search-results-button,.DocSearch-Input::-webkit-search-results-decoration{display:none}.DocSearch-LoadingIndicator,.DocSearch-MagnifierLabel,.DocSearch-Reset{margin:0;padding:0}.DocSearch-MagnifierLabel,.DocSearch-Reset{align-items:center;color:var(--docsearch-highlight-color);display:flex;justify-content:center}.DocSearch-Container--Stalled .DocSearch-MagnifierLabel,.DocSearch-LoadingIndicator{display:none}.DocSearch-Container--Stalled .DocSearch-LoadingIndicator{align-items:center;color:var(--docsearch-highlight-color);display:flex;justify-content:center}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Reset{animation:none;-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:50%;color:var(--docsearch-icon-color);cursor:pointer;right:0;stroke-width:var(--docsearch-icon-stroke-width)}}.DocSearch-Reset{animation:fade-in .1s ease-in forwards;-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:50%;color:var(--docsearch-icon-color);cursor:pointer;padding:2px;right:0;stroke-width:var(--docsearch-icon-stroke-width)}.DocSearch-Reset[hidden]{display:none}.DocSearch-Reset:hover{color:var(--docsearch-highlight-color)}.DocSearch-LoadingIndicator svg,.DocSearch-MagnifierLabel svg{height:24px;width:24px}.DocSearch-Cancel{display:none}.DocSearch-Dropdown{max-height:calc(var(--docsearch-modal-height) - var(--docsearch-searchbox-height) - var(--docsearch-spacing) - var(--docsearch-footer-height));min-height:var(--docsearch-spacing);overflow-y:auto;overflow-y:overlay;padding:0 var(--docsearch-spacing);scrollbar-color:var(--docsearch-muted-color) var(--docsearch-modal-background);scrollbar-width:thin}.DocSearch-Dropdown::-webkit-scrollbar{width:12px}.DocSearch-Dropdown::-webkit-scrollbar-track{background:transparent}.DocSearch-Dropdown::-webkit-scrollbar-thumb{background-color:var(--docsearch-muted-color);border:3px solid var(--docsearch-modal-background);border-radius:20px}.DocSearch-Dropdown ul{list-style:none;margin:0;padding:0}.DocSearch-Label{font-size:.75em;line-height:1.6em}.DocSearch-Help,.DocSearch-Label{color:var(--docsearch-muted-color)}.DocSearch-Help{font-size:.9em;margin:0;-webkit-user-select:none;user-select:none}.DocSearch-Title{font-size:1.2em}.DocSearch-Logo a{display:flex}.DocSearch-Logo svg{color:var(--docsearch-logo-color);margin-left:8px}.DocSearch-Hits:last-of-type{margin-bottom:24px}.DocSearch-Hits mark{background:none;color:var(--docsearch-highlight-color)}.DocSearch-HitsFooter{color:var(--docsearch-muted-color);display:flex;font-size:.85em;justify-content:center;margin-bottom:var(--docsearch-spacing);padding:var(--docsearch-spacing)}.DocSearch-HitsFooter a{border-bottom:1px solid;color:inherit}.DocSearch-Hit{border-radius:4px;display:flex;padding-bottom:4px;position:relative}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit--deleting{transition:none}}.DocSearch-Hit--deleting{opacity:0;transition:all .25s linear}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit--favoriting{transition:none}}.DocSearch-Hit--favoriting{transform:scale(0);transform-origin:top center;transition:all .25s linear;transition-delay:.25s}.DocSearch-Hit a{background:var(--docsearch-hit-background);border-radius:4px;box-shadow:var(--docsearch-hit-shadow);display:block;padding-left:var(--docsearch-spacing);width:100%}.DocSearch-Hit-source{background:var(--docsearch-modal-background);color:var(--docsearch-highlight-color);font-size:.85em;font-weight:600;line-height:32px;margin:0 -4px;padding:8px 4px 0;position:sticky;top:0;z-index:10}.DocSearch-Hit-Tree{color:var(--docsearch-muted-color);height:var(--docsearch-hit-height);opacity:.5;stroke-width:var(--docsearch-icon-stroke-width);width:24px}.DocSearch-Hit[aria-selected=true] a{background-color:var(--docsearch-highlight-color)}.DocSearch-Hit[aria-selected=true] mark{text-decoration:underline}.DocSearch-Hit-Container{align-items:center;color:var(--docsearch-hit-color);display:flex;flex-direction:row;height:var(--docsearch-hit-height);padding:0 var(--docsearch-spacing) 0 0}.DocSearch-Hit-icon{height:20px;width:20px}.DocSearch-Hit-action,.DocSearch-Hit-icon{color:var(--docsearch-muted-color);stroke-width:var(--docsearch-icon-stroke-width)}.DocSearch-Hit-action{align-items:center;display:flex;height:22px;width:22px}.DocSearch-Hit-action svg{display:block;height:18px;width:18px}.DocSearch-Hit-action+.DocSearch-Hit-action{margin-left:6px}.DocSearch-Hit-action-button{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:50%;color:inherit;cursor:pointer;padding:2px}svg.DocSearch-Hit-Select-Icon{display:none}.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-Select-Icon{display:block}.DocSearch-Hit-action-button:focus,.DocSearch-Hit-action-button:hover{background:#0003;transition:background-color .1s ease-in}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit-action-button:focus,.DocSearch-Hit-action-button:hover{transition:none}}.DocSearch-Hit-action-button:focus path,.DocSearch-Hit-action-button:hover path{fill:#fff}.DocSearch-Hit-content-wrapper{display:flex;flex:1 1 auto;flex-direction:column;font-weight:500;justify-content:center;line-height:1.2em;margin:0 8px;overflow-x:hidden;position:relative;text-overflow:ellipsis;white-space:nowrap;width:80%}.DocSearch-Hit-title{font-size:.9em}.DocSearch-Hit-path{color:var(--docsearch-muted-color);font-size:.75em}.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-action,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-icon,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-path,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-text,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-title,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-Tree,.DocSearch-Hit[aria-selected=true] mark{color:var(--docsearch-hit-active-color)!important}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit-action-button:focus,.DocSearch-Hit-action-button:hover{background:#0003;transition:none}}.DocSearch-ErrorScreen,.DocSearch-NoResults,.DocSearch-StartScreen{font-size:.9em;margin:0 auto;padding:36px 0;text-align:center;width:80%}.DocSearch-Screen-Icon{color:var(--docsearch-muted-color);padding-bottom:12px}.DocSearch-NoResults-Prefill-List{display:inline-block;padding-bottom:24px;text-align:left}.DocSearch-NoResults-Prefill-List ul{display:inline-block;padding:8px 0 0}.DocSearch-NoResults-Prefill-List li{list-style-position:inside;list-style-type:"» "}.DocSearch-Prefill{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:1em;color:var(--docsearch-highlight-color);cursor:pointer;display:inline-block;font-size:1em;font-weight:700;padding:0}.DocSearch-Prefill:focus,.DocSearch-Prefill:hover{outline:none;text-decoration:underline}.DocSearch-Footer{align-items:center;background:var(--docsearch-footer-background);border-radius:0 0 8px 8px;box-shadow:var(--docsearch-footer-shadow);display:flex;flex-direction:row-reverse;flex-shrink:0;height:var(--docsearch-footer-height);justify-content:space-between;padding:0 var(--docsearch-spacing);position:relative;-webkit-user-select:none;user-select:none;width:100%;z-index:300}.DocSearch-Commands{color:var(--docsearch-muted-color);display:flex;list-style:none;margin:0;padding:0}.DocSearch-Commands li{align-items:center;display:flex}.DocSearch-Commands li:not(:last-of-type){margin-right:.8em}.DocSearch-Commands-Key{align-items:center;background:var(--docsearch-key-gradient);border-radius:2px;box-shadow:var(--docsearch-key-shadow);display:flex;height:18px;justify-content:center;margin-right:.4em;padding:0 0 1px;color:var(--docsearch-muted-color);border:0;width:20px}.DocSearch-VisuallyHiddenForAccessibility{clip:rect(0 0 0 0);clip-path:inset(50%);height:1px;overflow:hidden;position:absolute;white-space:nowrap;width:1px}@media (max-width:768px){:root{--docsearch-spacing:10px;--docsearch-footer-height:40px}.DocSearch-Dropdown{height:100%}.DocSearch-Container{height:100vh;height:-webkit-fill-available;height:calc(var(--docsearch-vh, 1vh)*100);position:absolute}.DocSearch-Footer{border-radius:0;bottom:0;position:absolute}.DocSearch-Hit-content-wrapper{display:flex;position:relative;width:80%}.DocSearch-Modal{border-radius:0;box-shadow:none;height:100vh;height:-webkit-fill-available;height:calc(var(--docsearch-vh, 1vh)*100);margin:0;max-width:100%;width:100%}.DocSearch-Dropdown{max-height:calc(var(--docsearch-vh, 1vh)*100 - var(--docsearch-searchbox-height) - var(--docsearch-spacing) - var(--docsearch-footer-height))}.DocSearch-Cancel{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;color:var(--docsearch-highlight-color);cursor:pointer;display:inline-block;flex:none;font:inherit;font-size:1em;font-weight:500;margin-left:var(--docsearch-spacing);outline:none;overflow:hidden;padding:0;-webkit-user-select:none;user-select:none;white-space:nowrap}.DocSearch-Commands,.DocSearch-Hit-Tree{display:none}}@keyframes fade-in{0%{opacity:0}to{opacity:1}}[class*=DocSearch]{--docsearch-primary-color: var(--vp-c-brand-1);--docsearch-highlight-color: var(--docsearch-primary-color);--docsearch-text-color: var(--vp-c-text-1);--docsearch-muted-color: var(--vp-c-text-2);--docsearch-searchbox-shadow: none;--docsearch-searchbox-background: transparent;--docsearch-searchbox-focus-background: transparent;--docsearch-key-gradient: transparent;--docsearch-key-shadow: none;--docsearch-modal-background: var(--vp-c-bg-soft);--docsearch-footer-background: var(--vp-c-bg)}.dark [class*=DocSearch]{--docsearch-modal-shadow: none;--docsearch-footer-shadow: none;--docsearch-logo-color: var(--vp-c-text-2);--docsearch-hit-background: var(--vp-c-default-soft);--docsearch-hit-color: var(--vp-c-text-2);--docsearch-hit-shadow: none}.DocSearch-Button{display:flex;justify-content:center;align-items:center;margin:0;padding:0;width:48px;height:55px;background:transparent;transition:border-color .25s}.DocSearch-Button:hover{background:transparent}.DocSearch-Button:focus{outline:1px dotted;outline:5px auto -webkit-focus-ring-color}.DocSearch-Button-Key--pressed{transform:none;box-shadow:none}.DocSearch-Button:focus:not(:focus-visible){outline:none!important}@media (min-width: 768px){.DocSearch-Button{justify-content:flex-start;border:1px solid transparent;border-radius:8px;padding:0 10px 0 12px;width:100%;height:40px;background-color:var(--vp-c-bg-alt)}.DocSearch-Button:hover{border-color:var(--vp-c-brand-1);background:var(--vp-c-bg-alt)}}.DocSearch-Button .DocSearch-Button-Container{display:flex;align-items:center}.DocSearch-Button .DocSearch-Search-Icon{position:relative;width:16px;height:16px;color:var(--vp-c-text-1);fill:currentColor;transition:color .5s}.DocSearch-Button:hover .DocSearch-Search-Icon{color:var(--vp-c-text-1)}@media (min-width: 768px){.DocSearch-Button .DocSearch-Search-Icon{top:1px;margin-right:8px;width:14px;height:14px;color:var(--vp-c-text-2)}}.DocSearch-Button .DocSearch-Button-Placeholder{display:none;margin-top:2px;padding:0 16px 0 0;font-size:13px;font-weight:500;color:var(--vp-c-text-2);transition:color .5s}.DocSearch-Button:hover .DocSearch-Button-Placeholder{color:var(--vp-c-text-1)}@media (min-width: 768px){.DocSearch-Button .DocSearch-Button-Placeholder{display:inline-block}}.DocSearch-Button .DocSearch-Button-Keys{direction:ltr;display:none;min-width:auto}@media (min-width: 768px){.DocSearch-Button .DocSearch-Button-Keys{display:flex;align-items:center}}.DocSearch-Button .DocSearch-Button-Key{display:block;margin:2px 0 0;border:1px solid var(--vp-c-divider);border-right:none;border-radius:4px 0 0 4px;padding-left:6px;min-width:0;width:auto;height:22px;line-height:22px;font-family:var(--vp-font-family-base);font-size:12px;font-weight:500;transition:color .5s,border-color .5s}.DocSearch-Button .DocSearch-Button-Key+.DocSearch-Button-Key{border-right:1px solid var(--vp-c-divider);border-left:none;border-radius:0 4px 4px 0;padding-left:2px;padding-right:6px}.DocSearch-Button .DocSearch-Button-Key:first-child{font-size:0!important}.DocSearch-Button .DocSearch-Button-Key:first-child:after{content:"Ctrl";font-size:12px;letter-spacing:normal;color:var(--docsearch-muted-color)}.mac .DocSearch-Button .DocSearch-Button-Key:first-child:after{content:"⌘"}.DocSearch-Button .DocSearch-Button-Key:first-child>*{display:none}.DocSearch-Search-Icon{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' stroke-width='1.6' viewBox='0 0 20 20'%3E%3Cpath fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' d='m14.386 14.386 4.088 4.088-4.088-4.088A7.533 7.533 0 1 1 3.733 3.733a7.533 7.533 0 0 1 10.653 10.653z'/%3E%3C/svg%3E")}.VPNavBarSearch{display:flex;align-items:center}@media (min-width: 768px){.VPNavBarSearch{flex-grow:1;padding-left:24px}}@media (min-width: 960px){.VPNavBarSearch{padding-left:32px}}.dark .DocSearch-Footer{border-top:1px solid var(--vp-c-divider)}.DocSearch-Form{border:1px solid var(--vp-c-brand-1);background-color:var(--vp-c-white)}.dark .DocSearch-Form{background-color:var(--vp-c-default-soft)}.DocSearch-Screen-Icon>svg{margin:auto}.VPNavBarSocialLinks[data-v-164c457f]{display:none}@media (min-width: 1280px){.VPNavBarSocialLinks[data-v-164c457f]{display:flex;align-items:center}}.title[data-v-0f4f798b]{display:flex;align-items:center;border-bottom:1px solid transparent;width:100%;height:var(--vp-nav-height);font-size:16px;font-weight:600;color:var(--vp-c-text-1);transition:opacity .25s}@media (min-width: 960px){.title[data-v-0f4f798b]{flex-shrink:0}.VPNavBarTitle.has-sidebar .title[data-v-0f4f798b]{border-bottom-color:var(--vp-c-divider)}}[data-v-0f4f798b] .logo{margin-right:8px;height:var(--vp-nav-logo-height)}.VPNavBarTranslations[data-v-c80d9ad0]{display:none}@media (min-width: 1280px){.VPNavBarTranslations[data-v-c80d9ad0]{display:flex;align-items:center}}.title[data-v-c80d9ad0]{padding:0 24px 0 12px;line-height:32px;font-size:14px;font-weight:700;color:var(--vp-c-text-1)}.VPNavBar[data-v-822684d1]{position:relative;height:var(--vp-nav-height);pointer-events:none;white-space:nowrap;transition:background-color .25s}.VPNavBar.screen-open[data-v-822684d1]{transition:none;background-color:var(--vp-nav-bg-color);border-bottom:1px solid var(--vp-c-divider)}.VPNavBar[data-v-822684d1]:not(.home){background-color:var(--vp-nav-bg-color)}@media (min-width: 960px){.VPNavBar[data-v-822684d1]:not(.home){background-color:transparent}.VPNavBar[data-v-822684d1]:not(.has-sidebar):not(.home.top){background-color:var(--vp-nav-bg-color)}}.wrapper[data-v-822684d1]{padding:0 8px 0 24px}@media (min-width: 768px){.wrapper[data-v-822684d1]{padding:0 32px}}@media (min-width: 960px){.VPNavBar.has-sidebar .wrapper[data-v-822684d1]{padding:0}}.container[data-v-822684d1]{display:flex;justify-content:space-between;margin:0 auto;max-width:calc(var(--vp-layout-max-width) - 64px);height:var(--vp-nav-height);pointer-events:none}.container>.title[data-v-822684d1],.container>.content[data-v-822684d1]{pointer-events:none}.container[data-v-822684d1] *{pointer-events:auto}@media (min-width: 960px){.VPNavBar.has-sidebar .container[data-v-822684d1]{max-width:100%}}.title[data-v-822684d1]{flex-shrink:0;height:calc(var(--vp-nav-height) - 1px);transition:background-color .5s}@media (min-width: 960px){.VPNavBar.has-sidebar .title[data-v-822684d1]{position:absolute;top:0;left:0;z-index:2;padding:0 32px;width:var(--vp-sidebar-width);height:var(--vp-nav-height);background-color:transparent}}@media (min-width: 1440px){.VPNavBar.has-sidebar .title[data-v-822684d1]{padding-left:max(32px,calc((100% - (var(--vp-layout-max-width) - 64px)) / 2));width:calc((100% - (var(--vp-layout-max-width) - 64px)) / 2 + var(--vp-sidebar-width) - 32px)}}.content[data-v-822684d1]{flex-grow:1}@media (min-width: 960px){.VPNavBar.has-sidebar .content[data-v-822684d1]{position:relative;z-index:1;padding-right:32px;padding-left:var(--vp-sidebar-width)}}@media (min-width: 1440px){.VPNavBar.has-sidebar .content[data-v-822684d1]{padding-right:calc((100vw - var(--vp-layout-max-width)) / 2 + 32px);padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.content-body[data-v-822684d1]{display:flex;justify-content:flex-end;align-items:center;height:var(--vp-nav-height);transition:background-color .5s}@media (min-width: 960px){.VPNavBar:not(.home.top) .content-body[data-v-822684d1]{position:relative;background-color:var(--vp-nav-bg-color)}.VPNavBar:not(.has-sidebar):not(.home.top) .content-body[data-v-822684d1]{background-color:transparent}}@media (max-width: 767px){.content-body[data-v-822684d1]{column-gap:.5rem}}.menu+.translations[data-v-822684d1]:before,.menu+.appearance[data-v-822684d1]:before,.menu+.social-links[data-v-822684d1]:before,.translations+.appearance[data-v-822684d1]:before,.appearance+.social-links[data-v-822684d1]:before{margin-right:8px;margin-left:8px;width:1px;height:24px;background-color:var(--vp-c-divider);content:""}.menu+.appearance[data-v-822684d1]:before,.translations+.appearance[data-v-822684d1]:before{margin-right:16px}.appearance+.social-links[data-v-822684d1]:before{margin-left:16px}.social-links[data-v-822684d1]{margin-right:-8px}.divider[data-v-822684d1]{width:100%;height:1px}@media (min-width: 960px){.VPNavBar.has-sidebar .divider[data-v-822684d1]{padding-left:var(--vp-sidebar-width)}}@media (min-width: 1440px){.VPNavBar.has-sidebar .divider[data-v-822684d1]{padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.divider-line[data-v-822684d1]{width:100%;height:1px;transition:background-color .5s}.VPNavBar:not(.home) .divider-line[data-v-822684d1]{background-color:var(--vp-c-gutter)}@media (min-width: 960px){.VPNavBar:not(.home.top) .divider-line[data-v-822684d1]{background-color:var(--vp-c-gutter)}.VPNavBar:not(.has-sidebar):not(.home.top) .divider[data-v-822684d1]{background-color:var(--vp-c-gutter)}}.VPNavScreenAppearance[data-v-ffb44008]{display:flex;justify-content:space-between;align-items:center;border-radius:8px;padding:12px 14px 12px 16px;background-color:var(--vp-c-bg-soft)}.text[data-v-ffb44008]{line-height:24px;font-size:12px;font-weight:500;color:var(--vp-c-text-2)}.VPNavScreenMenuLink[data-v-735512b8]{display:block;border-bottom:1px solid var(--vp-c-divider);padding:12px 0 11px;line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:border-color .25s,color .25s}.VPNavScreenMenuLink[data-v-735512b8]:hover{color:var(--vp-c-brand-1)}.VPNavScreenMenuGroupLink[data-v-372ae7c0]{display:block;margin-left:12px;line-height:32px;font-size:14px;font-weight:400;color:var(--vp-c-text-1);transition:color .25s}.VPNavScreenMenuGroupLink[data-v-372ae7c0]:hover{color:var(--vp-c-brand-1)}.VPNavScreenMenuGroupSection[data-v-4b8941ac]{display:block}.title[data-v-4b8941ac]{line-height:32px;font-size:13px;font-weight:700;color:var(--vp-c-text-2);transition:color .25s}.VPNavScreenMenuGroup[data-v-875057a5]{border-bottom:1px solid var(--vp-c-divider);height:48px;overflow:hidden;transition:border-color .5s}.VPNavScreenMenuGroup .items[data-v-875057a5]{visibility:hidden}.VPNavScreenMenuGroup.open .items[data-v-875057a5]{visibility:visible}.VPNavScreenMenuGroup.open[data-v-875057a5]{padding-bottom:10px;height:auto}.VPNavScreenMenuGroup.open .button[data-v-875057a5]{padding-bottom:6px;color:var(--vp-c-brand-1)}.VPNavScreenMenuGroup.open .button-icon[data-v-875057a5]{transform:rotate(45deg)}.button[data-v-875057a5]{display:flex;justify-content:space-between;align-items:center;padding:12px 4px 11px 0;width:100%;line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:color .25s}.button[data-v-875057a5]:hover{color:var(--vp-c-brand-1)}.button-icon[data-v-875057a5]{transition:transform .25s}.group[data-v-875057a5]:first-child{padding-top:0}.group+.group[data-v-875057a5],.group+.item[data-v-875057a5]{padding-top:4px}.VPNavScreenTranslations[data-v-362991c2]{height:24px;overflow:hidden}.VPNavScreenTranslations.open[data-v-362991c2]{height:auto}.title[data-v-362991c2]{display:flex;align-items:center;font-size:14px;font-weight:500;color:var(--vp-c-text-1)}.icon[data-v-362991c2]{font-size:16px}.icon.lang[data-v-362991c2]{margin-right:8px}.icon.chevron[data-v-362991c2]{margin-left:4px}.list[data-v-362991c2]{padding:4px 0 0 24px}.link[data-v-362991c2]{line-height:32px;font-size:13px;color:var(--vp-c-text-1)}.VPNavScreen[data-v-833aabba]{position:fixed;top:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px));right:0;bottom:0;left:0;padding:0 32px;width:100%;background-color:var(--vp-nav-screen-bg-color);overflow-y:auto;transition:background-color .25s;pointer-events:auto}.VPNavScreen.fade-enter-active[data-v-833aabba],.VPNavScreen.fade-leave-active[data-v-833aabba]{transition:opacity .25s}.VPNavScreen.fade-enter-active .container[data-v-833aabba],.VPNavScreen.fade-leave-active .container[data-v-833aabba]{transition:transform .25s ease}.VPNavScreen.fade-enter-from[data-v-833aabba],.VPNavScreen.fade-leave-to[data-v-833aabba]{opacity:0}.VPNavScreen.fade-enter-from .container[data-v-833aabba],.VPNavScreen.fade-leave-to .container[data-v-833aabba]{transform:translateY(-8px)}@media (min-width: 768px){.VPNavScreen[data-v-833aabba]{display:none}}.container[data-v-833aabba]{margin:0 auto;padding:24px 0 96px;max-width:288px}.menu+.translations[data-v-833aabba],.menu+.appearance[data-v-833aabba],.translations+.appearance[data-v-833aabba]{margin-top:24px}.menu+.social-links[data-v-833aabba]{margin-top:16px}.appearance+.social-links[data-v-833aabba]{margin-top:16px}.VPNav[data-v-f1e365da]{position:relative;top:var(--vp-layout-top-height, 0px);left:0;z-index:var(--vp-z-index-nav);width:100%;pointer-events:none;transition:background-color .5s}@media (min-width: 960px){.VPNav[data-v-f1e365da]{position:fixed}}.VPSidebarItem.level-0[data-v-196b2e5f]{padding-bottom:24px}.VPSidebarItem.collapsed.level-0[data-v-196b2e5f]{padding-bottom:10px}.item[data-v-196b2e5f]{position:relative;display:flex;width:100%}.VPSidebarItem.collapsible>.item[data-v-196b2e5f]{cursor:pointer}.indicator[data-v-196b2e5f]{position:absolute;top:6px;bottom:6px;left:-17px;width:2px;border-radius:2px;transition:background-color .25s}.VPSidebarItem.level-2.is-active>.item>.indicator[data-v-196b2e5f],.VPSidebarItem.level-3.is-active>.item>.indicator[data-v-196b2e5f],.VPSidebarItem.level-4.is-active>.item>.indicator[data-v-196b2e5f],.VPSidebarItem.level-5.is-active>.item>.indicator[data-v-196b2e5f]{background-color:var(--vp-c-brand-1)}.link[data-v-196b2e5f]{display:flex;align-items:center;flex-grow:1}.text[data-v-196b2e5f]{flex-grow:1;padding:4px 0;line-height:24px;font-size:14px;transition:color .25s}.VPSidebarItem.level-0 .text[data-v-196b2e5f]{font-weight:700;color:var(--vp-c-text-1)}.VPSidebarItem.level-1 .text[data-v-196b2e5f],.VPSidebarItem.level-2 .text[data-v-196b2e5f],.VPSidebarItem.level-3 .text[data-v-196b2e5f],.VPSidebarItem.level-4 .text[data-v-196b2e5f],.VPSidebarItem.level-5 .text[data-v-196b2e5f]{font-weight:500;color:var(--vp-c-text-2)}.VPSidebarItem.level-0.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-1.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-2.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-3.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-4.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-5.is-link>.item>.link:hover .text[data-v-196b2e5f]{color:var(--vp-c-brand-1)}.VPSidebarItem.level-0.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-1.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-2.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-3.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-4.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-5.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-0.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-1.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-2.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-3.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-4.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-5.has-active>.item>.link>.text[data-v-196b2e5f]{color:var(--vp-c-text-1)}.VPSidebarItem.level-0.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-1.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-2.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-3.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-4.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-5.is-active>.item .link>.text[data-v-196b2e5f]{color:var(--vp-c-brand-1)}.caret[data-v-196b2e5f]{display:flex;justify-content:center;align-items:center;margin-right:-7px;width:32px;height:32px;color:var(--vp-c-text-3);cursor:pointer;transition:color .25s;flex-shrink:0}.item:hover .caret[data-v-196b2e5f]{color:var(--vp-c-text-2)}.item:hover .caret[data-v-196b2e5f]:hover{color:var(--vp-c-text-1)}.caret-icon[data-v-196b2e5f]{font-size:18px;transform:rotate(90deg);transition:transform .25s}.VPSidebarItem.collapsed .caret-icon[data-v-196b2e5f]{transform:rotate(0)}.VPSidebarItem.level-1 .items[data-v-196b2e5f],.VPSidebarItem.level-2 .items[data-v-196b2e5f],.VPSidebarItem.level-3 .items[data-v-196b2e5f],.VPSidebarItem.level-4 .items[data-v-196b2e5f],.VPSidebarItem.level-5 .items[data-v-196b2e5f]{border-left:1px solid var(--vp-c-divider);padding-left:16px}.VPSidebarItem.collapsed .items[data-v-196b2e5f]{display:none}.no-transition[data-v-9e426adc] .caret-icon{transition:none}.group+.group[data-v-9e426adc]{border-top:1px solid var(--vp-c-divider);padding-top:10px}@media (min-width: 960px){.group[data-v-9e426adc]{padding-top:10px;width:calc(var(--vp-sidebar-width) - 64px)}}.VPSidebar[data-v-18756405]{position:fixed;top:var(--vp-layout-top-height, 0px);bottom:0;left:0;z-index:var(--vp-z-index-sidebar);padding:32px 32px 96px;width:calc(100vw - 64px);max-width:320px;background-color:var(--vp-sidebar-bg-color);opacity:0;box-shadow:var(--vp-c-shadow-3);overflow-x:hidden;overflow-y:auto;transform:translate(-100%);transition:opacity .5s,transform .25s ease;overscroll-behavior:contain}.VPSidebar.open[data-v-18756405]{opacity:1;visibility:visible;transform:translate(0);transition:opacity .25s,transform .5s cubic-bezier(.19,1,.22,1)}.dark .VPSidebar[data-v-18756405]{box-shadow:var(--vp-shadow-1)}@media (min-width: 960px){.VPSidebar[data-v-18756405]{padding-top:var(--vp-nav-height);width:var(--vp-sidebar-width);max-width:100%;background-color:var(--vp-sidebar-bg-color);opacity:1;visibility:visible;box-shadow:none;transform:translate(0)}}@media (min-width: 1440px){.VPSidebar[data-v-18756405]{padding-left:max(32px,calc((100% - (var(--vp-layout-max-width) - 64px)) / 2));width:calc((100% - (var(--vp-layout-max-width) - 64px)) / 2 + var(--vp-sidebar-width) - 32px)}}@media (min-width: 960px){.curtain[data-v-18756405]{position:sticky;top:-64px;left:0;z-index:1;margin-top:calc(var(--vp-nav-height) * -1);margin-right:-32px;margin-left:-32px;height:var(--vp-nav-height);background-color:var(--vp-sidebar-bg-color)}}.nav[data-v-18756405]{outline:0}.VPSkipLink[data-v-c3508ec8]{top:8px;left:8px;padding:8px 16px;z-index:999;border-radius:8px;font-size:12px;font-weight:700;text-decoration:none;color:var(--vp-c-brand-1);box-shadow:var(--vp-shadow-3);background-color:var(--vp-c-bg)}.VPSkipLink[data-v-c3508ec8]:focus{height:auto;width:auto;clip:auto;clip-path:none}@media (min-width: 1280px){.VPSkipLink[data-v-c3508ec8]{top:14px;left:16px}}.Layout[data-v-a9a9e638]{display:flex;flex-direction:column;min-height:100vh}.VPHomeSponsors[data-v-db81191c]{border-top:1px solid var(--vp-c-gutter);padding-top:88px!important}.VPHomeSponsors[data-v-db81191c]{margin:96px 0}@media (min-width: 768px){.VPHomeSponsors[data-v-db81191c]{margin:128px 0}}.VPHomeSponsors[data-v-db81191c]{padding:0 24px}@media (min-width: 768px){.VPHomeSponsors[data-v-db81191c]{padding:0 48px}}@media (min-width: 960px){.VPHomeSponsors[data-v-db81191c]{padding:0 64px}}.container[data-v-db81191c]{margin:0 auto;max-width:1152px}.love[data-v-db81191c]{margin:0 auto;width:fit-content;font-size:28px;color:var(--vp-c-text-3)}.icon[data-v-db81191c]{display:inline-block}.message[data-v-db81191c]{margin:0 auto;padding-top:10px;max-width:320px;text-align:center;line-height:24px;font-size:16px;font-weight:500;color:var(--vp-c-text-2)}.sponsors[data-v-db81191c]{padding-top:32px}.action[data-v-db81191c]{padding-top:40px;text-align:center}.VPTeamPage[data-v-c2f8e101]{margin:96px 0}@media (min-width: 768px){.VPTeamPage[data-v-c2f8e101]{margin:128px 0}}.VPHome .VPTeamPageTitle[data-v-c2f8e101-s]{border-top:1px solid var(--vp-c-gutter);padding-top:88px!important}.VPTeamPageSection+.VPTeamPageSection[data-v-c2f8e101-s],.VPTeamMembers+.VPTeamPageSection[data-v-c2f8e101-s]{margin-top:64px}.VPTeamMembers+.VPTeamMembers[data-v-c2f8e101-s]{margin-top:24px}@media (min-width: 768px){.VPTeamPageTitle+.VPTeamPageSection[data-v-c2f8e101-s]{margin-top:16px}.VPTeamPageSection+.VPTeamPageSection[data-v-c2f8e101-s],.VPTeamMembers+.VPTeamPageSection[data-v-c2f8e101-s]{margin-top:96px}}.VPTeamMembers[data-v-c2f8e101-s]{padding:0 24px}@media (min-width: 768px){.VPTeamMembers[data-v-c2f8e101-s]{padding:0 48px}}@media (min-width: 960px){.VPTeamMembers[data-v-c2f8e101-s]{padding:0 64px}}.VPTeamPageTitle[data-v-e277e15c]{padding:48px 32px;text-align:center}@media (min-width: 768px){.VPTeamPageTitle[data-v-e277e15c]{padding:64px 48px 48px}}@media (min-width: 960px){.VPTeamPageTitle[data-v-e277e15c]{padding:80px 64px 48px}}.title[data-v-e277e15c]{letter-spacing:0;line-height:44px;font-size:36px;font-weight:500}@media (min-width: 768px){.title[data-v-e277e15c]{letter-spacing:-.5px;line-height:56px;font-size:48px}}.lead[data-v-e277e15c]{margin:0 auto;max-width:512px;padding-top:12px;line-height:24px;font-size:16px;font-weight:500;color:var(--vp-c-text-2)}@media (min-width: 768px){.lead[data-v-e277e15c]{max-width:592px;letter-spacing:.15px;line-height:28px;font-size:20px}}.VPTeamPageSection[data-v-d43bc49d]{padding:0 32px}@media (min-width: 768px){.VPTeamPageSection[data-v-d43bc49d]{padding:0 48px}}@media (min-width: 960px){.VPTeamPageSection[data-v-d43bc49d]{padding:0 64px}}.title[data-v-d43bc49d]{position:relative;margin:0 auto;max-width:1152px;text-align:center;color:var(--vp-c-text-2)}.title-line[data-v-d43bc49d]{position:absolute;top:16px;left:0;width:100%;height:1px;background-color:var(--vp-c-divider)}.title-text[data-v-d43bc49d]{position:relative;display:inline-block;padding:0 24px;letter-spacing:0;line-height:32px;font-size:20px;font-weight:500;background-color:var(--vp-c-bg)}.lead[data-v-d43bc49d]{margin:0 auto;max-width:480px;padding-top:12px;text-align:center;line-height:24px;font-size:16px;font-weight:500;color:var(--vp-c-text-2)}.members[data-v-d43bc49d]{padding-top:40px}.VPTeamMembersItem[data-v-f9987cb6]{display:flex;flex-direction:column;gap:2px;border-radius:12px;width:100%;height:100%;overflow:hidden}.VPTeamMembersItem.small .profile[data-v-f9987cb6]{padding:32px}.VPTeamMembersItem.small .data[data-v-f9987cb6]{padding-top:20px}.VPTeamMembersItem.small .avatar[data-v-f9987cb6]{width:64px;height:64px}.VPTeamMembersItem.small .name[data-v-f9987cb6]{line-height:24px;font-size:16px}.VPTeamMembersItem.small .affiliation[data-v-f9987cb6]{padding-top:4px;line-height:20px;font-size:14px}.VPTeamMembersItem.small .desc[data-v-f9987cb6]{padding-top:12px;line-height:20px;font-size:14px}.VPTeamMembersItem.small .links[data-v-f9987cb6]{margin:0 -16px -20px;padding:10px 0 0}.VPTeamMembersItem.medium .profile[data-v-f9987cb6]{padding:48px 32px}.VPTeamMembersItem.medium .data[data-v-f9987cb6]{padding-top:24px;text-align:center}.VPTeamMembersItem.medium .avatar[data-v-f9987cb6]{width:96px;height:96px}.VPTeamMembersItem.medium .name[data-v-f9987cb6]{letter-spacing:.15px;line-height:28px;font-size:20px}.VPTeamMembersItem.medium .affiliation[data-v-f9987cb6]{padding-top:4px;font-size:16px}.VPTeamMembersItem.medium .desc[data-v-f9987cb6]{padding-top:16px;max-width:288px;font-size:16px}.VPTeamMembersItem.medium .links[data-v-f9987cb6]{margin:0 -16px -12px;padding:16px 12px 0}.profile[data-v-f9987cb6]{flex-grow:1;background-color:var(--vp-c-bg-soft)}.data[data-v-f9987cb6]{text-align:center}.avatar[data-v-f9987cb6]{position:relative;flex-shrink:0;margin:0 auto;border-radius:50%;box-shadow:var(--vp-shadow-3)}.avatar-img[data-v-f9987cb6]{position:absolute;top:0;right:0;bottom:0;left:0;border-radius:50%;object-fit:cover}.name[data-v-f9987cb6]{margin:0;font-weight:600}.affiliation[data-v-f9987cb6]{margin:0;font-weight:500;color:var(--vp-c-text-2)}.org.link[data-v-f9987cb6]{color:var(--vp-c-text-2);transition:color .25s}.org.link[data-v-f9987cb6]:hover{color:var(--vp-c-brand-1)}.desc[data-v-f9987cb6]{margin:0 auto}.desc[data-v-f9987cb6] a{font-weight:500;color:var(--vp-c-brand-1);text-decoration-style:dotted;transition:color .25s}.links[data-v-f9987cb6]{display:flex;justify-content:center;height:56px}.sp-link[data-v-f9987cb6]{display:flex;justify-content:center;align-items:center;text-align:center;padding:16px;font-size:14px;font-weight:500;color:var(--vp-c-sponsor);background-color:var(--vp-c-bg-soft);transition:color .25s,background-color .25s}.sp .sp-link.link[data-v-f9987cb6]:hover,.sp .sp-link.link[data-v-f9987cb6]:focus{outline:none;color:var(--vp-c-white);background-color:var(--vp-c-sponsor)}.sp-icon[data-v-f9987cb6]{margin-right:8px;font-size:16px}.VPTeamMembers.small .container[data-v-fba19bad]{grid-template-columns:repeat(auto-fit,minmax(224px,1fr))}.VPTeamMembers.small.count-1 .container[data-v-fba19bad]{max-width:276px}.VPTeamMembers.small.count-2 .container[data-v-fba19bad]{max-width:576px}.VPTeamMembers.small.count-3 .container[data-v-fba19bad]{max-width:876px}.VPTeamMembers.medium .container[data-v-fba19bad]{grid-template-columns:repeat(auto-fit,minmax(256px,1fr))}@media (min-width: 375px){.VPTeamMembers.medium .container[data-v-fba19bad]{grid-template-columns:repeat(auto-fit,minmax(288px,1fr))}}.VPTeamMembers.medium.count-1 .container[data-v-fba19bad]{max-width:368px}.VPTeamMembers.medium.count-2 .container[data-v-fba19bad]{max-width:760px}.container[data-v-fba19bad]{display:grid;gap:24px;margin:0 auto;max-width:1152px}.enjoyer{margin-top:.5rem;margin-bottom:0rem;border-radius:14px;padding-top:.2rem;padding-bottom:.2rem;position:relative;font-size:.9rem;font-weight:700;line-height:1.1rem;display:flex;align-items:center;justify-content:center;width:100%;gap:1rem;background-color:var(--vp-c-bg-alt);border:2px solid var(--vp-c-bg-alt);transition:border-color .5s}.enjoyer:hover{border:2px solid var(--vp-c-brand-light)}.enjoyer img{transition:transform .5s;transform:scale(1.25)}.enjoyer:hover img{transform:scale(1.75)}.enjoyer .heading{background-image:linear-gradient(120deg,#b047ff 16%,var(--vp-c-brand-lighter),var(--vp-c-brand-lighter));background-clip:text;-webkit-background-clip:text;-webkit-text-fill-color:transparent}.enjoyer .extra-info{color:var(--vp-c-text-1);opacity:0;font-size:.7rem;padding-left:.1rem;transition:opacity .5s}.enjoyer:hover .extra-info{opacity:.9}.VPVersionPicker[data-v-d483b3a6] button .text{color:var(--vp-c-text-1)!important}.VPVersionPicker[data-v-d483b3a6]:hover button .text{color:var(--vp-c-text-2)!important}:root{--vp-plugin-tabs-tab-text-color: var(--vp-c-text-2);--vp-plugin-tabs-tab-active-text-color: var(--vp-c-text-1);--vp-plugin-tabs-tab-hover-text-color: var(--vp-c-text-1);--vp-plugin-tabs-tab-bg: var(--vp-c-bg-soft);--vp-plugin-tabs-tab-divider: var(--vp-c-divider);--vp-plugin-tabs-tab-active-bar-color: var(--vp-c-brand-1)}.plugin-tabs{margin:16px 0;background-color:var(--vp-plugin-tabs-tab-bg);border-radius:8px}.plugin-tabs--tab-list{position:relative;padding:0 12px;overflow-x:auto;overflow-y:hidden}.plugin-tabs--tab-list:after{content:"";position:absolute;bottom:0;left:0;right:0;height:2px;background-color:var(--vp-plugin-tabs-tab-divider)}.plugin-tabs--tab{position:relative;padding:0 12px;line-height:48px;border-bottom:2px solid transparent;color:var(--vp-plugin-tabs-tab-text-color);font-size:14px;font-weight:500;white-space:nowrap;transition:color .25s}.plugin-tabs--tab[aria-selected=true]{color:var(--vp-plugin-tabs-tab-active-text-color)}.plugin-tabs--tab:hover{color:var(--vp-plugin-tabs-tab-hover-text-color)}.plugin-tabs--tab:after{content:"";position:absolute;bottom:-2px;left:8px;right:8px;height:2px;background-color:transparent;transition:background-color .25s;z-index:1}.plugin-tabs--tab[aria-selected=true]:after{background-color:var(--vp-plugin-tabs-tab-active-bar-color)}.plugin-tabs--content[data-v-9b0d03d2]{padding:16px}.plugin-tabs--content[data-v-9b0d03d2]>:first-child:first-child{margin-top:0}.plugin-tabs--content[data-v-9b0d03d2]>:last-child:last-child{margin-bottom:0}.plugin-tabs--content[data-v-9b0d03d2]>div[class*=language-]{border-radius:8px;margin:16px 0}:root:not(.dark) .plugin-tabs--content[data-v-9b0d03d2] div[class*=language-]{background-color:var(--vp-c-bg)}.VPHero .clip{white-space:pre;max-width:500px}@font-face{font-family:JuliaMono-Regular;src:url(https://cdn.jsdelivr.net/gh/cormullion/juliamono/webfonts/JuliaMono-Regular.woff2)}:root{--vp-font-family-base: "Barlow", "Inter var experimental", "Inter var", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif;--vp-font-family-mono: JuliaMono-Regular, monospace;font-feature-settings:"calt" 0}.mono pre,.mono code{font-family:JuliaMono-Light}:root{--julia-blue: #4063d8;--julia-purple: #9558b2;--julia-red: #cb3c33;--julia-green: #389826;--vp-c-brand: #389826;--vp-c-brand-light: #3dd027;--vp-c-brand-lighter: #9499ff;--vp-c-brand-lightest: #bcc0ff;--vp-c-brand-dark: #535bf2;--vp-c-brand-darker: #454ce1;--vp-c-brand-dimm: #212425}:root{--vp-button-brand-border: var(--vp-c-brand-light);--vp-button-brand-text: var(--vp-c-white);--vp-button-brand-bg: var(--vp-c-brand);--vp-button-brand-hover-border: var(--vp-c-brand-light);--vp-button-brand-hover-text: var(--vp-c-white);--vp-button-brand-hover-bg: var(--vp-c-brand-light);--vp-button-brand-active-border: var(--vp-c-brand-light);--vp-button-brand-active-text: var(--vp-c-white);--vp-button-brand-active-bg: var(--vp-button-brand-bg)}:root{--vp-home-hero-name-color: transparent;--vp-home-hero-name-background: -webkit-linear-gradient( 120deg, #9558b2 30%, #cb3c33 );--vp-home-hero-image-background-image: linear-gradient( -45deg, #9558b2 30%, #389826 30%, #cb3c33 );--vp-home-hero-image-filter: blur(40px)}@media (min-width: 640px){:root{--vp-home-hero-image-filter: blur(56px)}}@media (min-width: 960px){:root{--vp-home-hero-image-filter: blur(72px)}}:root.dark{--vp-custom-block-tip-border: var(--vp-c-brand);--vp-custom-block-tip-text: var(--vp-c-brand-lightest);--vp-custom-block-tip-bg: var(--vp-c-brand-dimm);--vp-c-black: hsl(220 20% 9%);--vp-c-black-pure: hsl(220, 24%, 4%);--vp-c-black-soft: hsl(220 16% 13%);--vp-c-black-mute: hsl(220 14% 17%);--vp-c-gray: hsl(220 8% 56%);--vp-c-gray-dark-1: hsl(220 10% 39%);--vp-c-gray-dark-2: hsl(220 12% 28%);--vp-c-gray-dark-3: hsl(220 12% 23%);--vp-c-gray-dark-4: hsl(220 14% 17%);--vp-c-gray-dark-5: hsl(220 16% 13%);--vp-custom-block-info-bg: hsl(220 14% 17%)}.DocSearch{--docsearch-primary-color: var(--vp-c-brand) !important}mjx-container>svg{display:block;margin:auto}mjx-container{padding:.5rem 0}mjx-container{display:inline;margin:auto 2px -2px}mjx-container>svg{margin:auto;display:inline}:root{--vp-c-brand-1: #cb3c33;--vp-c-brand-2: #cb3c33;--vp-c-brand-3: #cb3c33;--vp-c-sponsor: #ca2971;--vitest-c-sponsor-hover: #c13071}.dark{--vp-c-brand-1: #91dd33;--vp-c-brand-2: #91dd33;--vp-c-brand-3: #91dd33;--vp-c-sponsor: #91dd33;--vitest-c-sponsor-hover: #e51370}:root:not(.dark) .dark-only{display:none}:root:is(.dark) .light-only{display:none}@media (min-width: 1440px){.VPSidebar{padding-left:20px!important;width:250px!important}.VPNavBar .title{padding-left:15px!important;width:230px!important}.VPContent.has-sidebar{padding-left:250px!important;padding-right:5vw!important}.VPNavBar .curtain{width:100%!important}.VPDoc{padding:32px 0 0!important}.VPNavBar.has-sidebar .content{padding-left:250px!important;padding-right:20px!important}.VPNavBar .divider{padding-left:250px!important}}@media (min-width: 960px){.VPDoc{padding:32px 32px 0 10!important}.VPContent.has-sidebar{padding-left:255px!important}}.VPNavBar{padding-right:0!important}.VPDoc.has-aside .content-container{max-width:100%!important}.aside{max-width:200px!important;padding-left:0!important}.VPDocOutlineItem li{text-overflow:ellipsis;overflow:hidden;white-space:nowrap;max-width:200px}.VPNavBar .title{text-overflow:ellipsis;overflow:hidden;white-space:nowrap}.jldocstring.custom-block{border:1px solid var(--vp-c-gray-2);color:var(--vp-c-text-1)}.jldocstring.custom-block summary{font-weight:700;cursor:pointer;-webkit-user-select:none;user-select:none;margin:0 0 8px}.img-box[data-v-06a0366f]{box-sizing:content-box;border-radius:14px;margin:20px;height:350px;width:250px;overflow:hidden;display:inline-block;color:#fff;position:relative;background-color:transparent;border:2px solid var(--vp-c-bg-alt)}.img-box h2[data-v-06a0366f]{border-top:0}.img-box img[data-v-06a0366f]{height:100%;width:100%;object-fit:cover;opacity:.3;transition:transform .3s ease,opacity .3s ease}.caption[data-v-06a0366f]{position:absolute;bottom:30px;color:var(--vp-c-text-1);left:10px;opacity:1;transition:transform .3s ease,opacity .3s ease}.subcaption[data-v-06a0366f]{position:absolute;bottom:5px;color:var(--vp-c-text-1);left:10px;opacity:0;transition:transform .3s ease,opacity .3s ease}.transparent-box1[data-v-06a0366f]{height:250px;width:250px;background-color:transparent;position:absolute;top:0;left:0;transition:background-color .3s ease}.transparent-box2[data-v-06a0366f]{height:100px;width:250px;background-color:transparent;position:absolute;top:250px;left:0;transition:background-color .3s ease}.img-box:hover img[data-v-06a0366f]{transform:scale(1.1)}.img-box:hover .transparent-box1[data-v-06a0366f],.img-box:hover .transparent-box2[data-v-06a0366f]{background-color:var(--vp-c-bg-alt)}.img-box:hover .caption[data-v-06a0366f],.img-box:hover .subcaption[data-v-06a0366f]{transform:translateY(-20px);opacity:1}.img-box[data-v-06a0366f]:hover{border:2px solid var(--vp-c-brand-light);cursor:pointer}.caption>p[data-v-06a0366f]:nth-child(2){font-size:.8em}.subcaption>p[data-v-06a0366f]:nth-child(2){font-size:.8em}.opacity-low[data-v-06a0366f]{opacity:.85}.heading[data-v-578d61bc]{text-align:center;font-size:2em;letter-spacing:1px;padding:40px;color:#fff}.gallery-image[data-v-578d61bc]{padding:20px;display:flex;flex-wrap:wrap;justify-content:center}.gallery-image[data-v-578d61bc] img{height:350px;width:250px;transform:scale(1);transition:transform .4s ease}.VPLocalSearchBox[data-v-42e65fb9]{position:fixed;z-index:100;top:0;right:0;bottom:0;left:0;display:flex}.backdrop[data-v-42e65fb9]{position:absolute;top:0;right:0;bottom:0;left:0;background:var(--vp-backdrop-bg-color);transition:opacity .5s}.shell[data-v-42e65fb9]{position:relative;padding:12px;margin:64px auto;display:flex;flex-direction:column;gap:16px;background:var(--vp-local-search-bg);width:min(100vw - 60px,900px);height:min-content;max-height:min(100vh - 128px,900px);border-radius:6px}@media (max-width: 767px){.shell[data-v-42e65fb9]{margin:0;width:100vw;height:100vh;max-height:none;border-radius:0}}.search-bar[data-v-42e65fb9]{border:1px solid var(--vp-c-divider);border-radius:4px;display:flex;align-items:center;padding:0 12px;cursor:text}@media (max-width: 767px){.search-bar[data-v-42e65fb9]{padding:0 8px}}.search-bar[data-v-42e65fb9]:focus-within{border-color:var(--vp-c-brand-1)}.local-search-icon[data-v-42e65fb9]{display:block;font-size:18px}.navigate-icon[data-v-42e65fb9]{display:block;font-size:14px}.search-icon[data-v-42e65fb9]{margin:8px}@media (max-width: 767px){.search-icon[data-v-42e65fb9]{display:none}}.search-input[data-v-42e65fb9]{padding:6px 12px;font-size:inherit;width:100%}@media (max-width: 767px){.search-input[data-v-42e65fb9]{padding:6px 4px}}.search-actions[data-v-42e65fb9]{display:flex;gap:4px}@media (any-pointer: coarse){.search-actions[data-v-42e65fb9]{gap:8px}}@media (min-width: 769px){.search-actions.before[data-v-42e65fb9]{display:none}}.search-actions button[data-v-42e65fb9]{padding:8px}.search-actions button[data-v-42e65fb9]:not([disabled]):hover,.toggle-layout-button.detailed-list[data-v-42e65fb9]{color:var(--vp-c-brand-1)}.search-actions button.clear-button[data-v-42e65fb9]:disabled{opacity:.37}.search-keyboard-shortcuts[data-v-42e65fb9]{font-size:.8rem;opacity:75%;display:flex;flex-wrap:wrap;gap:16px;line-height:14px}.search-keyboard-shortcuts span[data-v-42e65fb9]{display:flex;align-items:center;gap:4px}@media (max-width: 767px){.search-keyboard-shortcuts[data-v-42e65fb9]{display:none}}.search-keyboard-shortcuts kbd[data-v-42e65fb9]{background:#8080801a;border-radius:4px;padding:3px 6px;min-width:24px;display:inline-block;text-align:center;vertical-align:middle;border:1px solid rgba(128,128,128,.15);box-shadow:0 2px 2px #0000001a}.results[data-v-42e65fb9]{display:flex;flex-direction:column;gap:6px;overflow-x:hidden;overflow-y:auto;overscroll-behavior:contain}.result[data-v-42e65fb9]{display:flex;align-items:center;gap:8px;border-radius:4px;transition:none;line-height:1rem;border:solid 2px var(--vp-local-search-result-border);outline:none}.result>div[data-v-42e65fb9]{margin:12px;width:100%;overflow:hidden}@media (max-width: 767px){.result>div[data-v-42e65fb9]{margin:8px}}.titles[data-v-42e65fb9]{display:flex;flex-wrap:wrap;gap:4px;position:relative;z-index:1001;padding:2px 0}.title[data-v-42e65fb9]{display:flex;align-items:center;gap:4px}.title.main[data-v-42e65fb9]{font-weight:500}.title-icon[data-v-42e65fb9]{opacity:.5;font-weight:500;color:var(--vp-c-brand-1)}.title svg[data-v-42e65fb9]{opacity:.5}.result.selected[data-v-42e65fb9]{--vp-local-search-result-bg: var(--vp-local-search-result-selected-bg);border-color:var(--vp-local-search-result-selected-border)}.excerpt-wrapper[data-v-42e65fb9]{position:relative}.excerpt[data-v-42e65fb9]{opacity:50%;pointer-events:none;max-height:140px;overflow:hidden;position:relative;margin-top:4px}.result.selected .excerpt[data-v-42e65fb9]{opacity:1}.excerpt[data-v-42e65fb9] *{font-size:.8rem!important;line-height:130%!important}.titles[data-v-42e65fb9] mark,.excerpt[data-v-42e65fb9] mark{background-color:var(--vp-local-search-highlight-bg);color:var(--vp-local-search-highlight-text);border-radius:2px;padding:0 2px}.excerpt[data-v-42e65fb9] .vp-code-group .tabs{display:none}.excerpt[data-v-42e65fb9] .vp-code-group div[class*=language-]{border-radius:8px!important}.excerpt-gradient-bottom[data-v-42e65fb9]{position:absolute;bottom:-1px;left:0;width:100%;height:8px;background:linear-gradient(transparent,var(--vp-local-search-result-bg));z-index:1000}.excerpt-gradient-top[data-v-42e65fb9]{position:absolute;top:-1px;left:0;width:100%;height:8px;background:linear-gradient(var(--vp-local-search-result-bg),transparent);z-index:1000}.result.selected .titles[data-v-42e65fb9],.result.selected .title-icon[data-v-42e65fb9]{color:var(--vp-c-brand-1)!important}.no-results[data-v-42e65fb9]{font-size:.9rem;text-align:center;padding:12px}svg[data-v-42e65fb9]{flex:none}
+@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-cyrillic-ext.BBPuwvHQ.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C88,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-cyrillic.C5lxZ8CY.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-greek-ext.CqjqNYQ-.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-greek.BBVDIX6e.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-vietnamese.BjW4sHH5.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-latin-ext.4ZJIpNVo.woff2) format("woff2");unicode-range:U+0100-02AF,U+0304,U+0308,U+0329,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-roman-latin.Di8DUHzh.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-cyrillic-ext.r48I6akx.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C88,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-cyrillic.By2_1cv3.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-greek-ext.1u6EdAuj.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-greek.DJ8dCoTZ.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-vietnamese.BSbpV94h.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-latin-ext.CN1xVJS-.woff2) format("woff2");unicode-range:U+0100-02AF,U+0304,U+0308,U+0329,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:italic;font-weight:100 900;font-display:swap;src:url(/dev/assets/inter-italic-latin.C2AdPX0b.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+2074,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Punctuation SC;font-weight:400;src:local("PingFang SC Regular"),local("Noto Sans CJK SC"),local("Microsoft YaHei");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}@font-face{font-family:Punctuation SC;font-weight:500;src:local("PingFang SC Medium"),local("Noto Sans CJK SC"),local("Microsoft YaHei");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}@font-face{font-family:Punctuation SC;font-weight:600;src:local("PingFang SC Semibold"),local("Noto Sans CJK SC Bold"),local("Microsoft YaHei Bold");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}@font-face{font-family:Punctuation SC;font-weight:700;src:local("PingFang SC Semibold"),local("Noto Sans CJK SC Bold"),local("Microsoft YaHei Bold");unicode-range:U+201C,U+201D,U+2018,U+2019,U+2E3A,U+2014,U+2013,U+2026,U+00B7,U+007E,U+002F}:root{--vp-c-white: #ffffff;--vp-c-black: #000000;--vp-c-neutral: var(--vp-c-black);--vp-c-neutral-inverse: var(--vp-c-white)}.dark{--vp-c-neutral: var(--vp-c-white);--vp-c-neutral-inverse: var(--vp-c-black)}:root{--vp-c-gray-1: #dddde3;--vp-c-gray-2: #e4e4e9;--vp-c-gray-3: #ebebef;--vp-c-gray-soft: rgba(142, 150, 170, .14);--vp-c-indigo-1: #3451b2;--vp-c-indigo-2: #3a5ccc;--vp-c-indigo-3: #5672cd;--vp-c-indigo-soft: rgba(100, 108, 255, .14);--vp-c-purple-1: #6f42c1;--vp-c-purple-2: #7e4cc9;--vp-c-purple-3: #8e5cd9;--vp-c-purple-soft: rgba(159, 122, 234, .14);--vp-c-green-1: #18794e;--vp-c-green-2: #299764;--vp-c-green-3: #30a46c;--vp-c-green-soft: rgba(16, 185, 129, .14);--vp-c-yellow-1: #915930;--vp-c-yellow-2: #946300;--vp-c-yellow-3: #9f6a00;--vp-c-yellow-soft: rgba(234, 179, 8, .14);--vp-c-red-1: #b8272c;--vp-c-red-2: #d5393e;--vp-c-red-3: #e0575b;--vp-c-red-soft: rgba(244, 63, 94, .14);--vp-c-sponsor: #db2777}.dark{--vp-c-gray-1: #515c67;--vp-c-gray-2: #414853;--vp-c-gray-3: #32363f;--vp-c-gray-soft: rgba(101, 117, 133, .16);--vp-c-indigo-1: #a8b1ff;--vp-c-indigo-2: #5c73e7;--vp-c-indigo-3: #3e63dd;--vp-c-indigo-soft: rgba(100, 108, 255, .16);--vp-c-purple-1: #c8abfa;--vp-c-purple-2: #a879e6;--vp-c-purple-3: #8e5cd9;--vp-c-purple-soft: rgba(159, 122, 234, .16);--vp-c-green-1: #3dd68c;--vp-c-green-2: #30a46c;--vp-c-green-3: #298459;--vp-c-green-soft: rgba(16, 185, 129, .16);--vp-c-yellow-1: #f9b44e;--vp-c-yellow-2: #da8b17;--vp-c-yellow-3: #a46a0a;--vp-c-yellow-soft: rgba(234, 179, 8, .16);--vp-c-red-1: #f66f81;--vp-c-red-2: #f14158;--vp-c-red-3: #b62a3c;--vp-c-red-soft: rgba(244, 63, 94, .16)}:root{--vp-c-bg: #ffffff;--vp-c-bg-alt: #f6f6f7;--vp-c-bg-elv: #ffffff;--vp-c-bg-soft: #f6f6f7}.dark{--vp-c-bg: #1b1b1f;--vp-c-bg-alt: #161618;--vp-c-bg-elv: #202127;--vp-c-bg-soft: #202127}:root{--vp-c-border: #c2c2c4;--vp-c-divider: #e2e2e3;--vp-c-gutter: #e2e2e3}.dark{--vp-c-border: #3c3f44;--vp-c-divider: #2e2e32;--vp-c-gutter: #000000}:root{--vp-c-text-1: rgba(60, 60, 67);--vp-c-text-2: rgba(60, 60, 67, .78);--vp-c-text-3: rgba(60, 60, 67, .56)}.dark{--vp-c-text-1: rgba(255, 255, 245, .86);--vp-c-text-2: rgba(235, 235, 245, .6);--vp-c-text-3: rgba(235, 235, 245, .38)}:root{--vp-c-default-1: var(--vp-c-gray-1);--vp-c-default-2: var(--vp-c-gray-2);--vp-c-default-3: var(--vp-c-gray-3);--vp-c-default-soft: var(--vp-c-gray-soft);--vp-c-brand-1: var(--vp-c-indigo-1);--vp-c-brand-2: var(--vp-c-indigo-2);--vp-c-brand-3: var(--vp-c-indigo-3);--vp-c-brand-soft: var(--vp-c-indigo-soft);--vp-c-brand: var(--vp-c-brand-1);--vp-c-tip-1: var(--vp-c-brand-1);--vp-c-tip-2: var(--vp-c-brand-2);--vp-c-tip-3: var(--vp-c-brand-3);--vp-c-tip-soft: var(--vp-c-brand-soft);--vp-c-note-1: var(--vp-c-brand-1);--vp-c-note-2: var(--vp-c-brand-2);--vp-c-note-3: var(--vp-c-brand-3);--vp-c-note-soft: var(--vp-c-brand-soft);--vp-c-success-1: var(--vp-c-green-1);--vp-c-success-2: var(--vp-c-green-2);--vp-c-success-3: var(--vp-c-green-3);--vp-c-success-soft: var(--vp-c-green-soft);--vp-c-important-1: var(--vp-c-purple-1);--vp-c-important-2: var(--vp-c-purple-2);--vp-c-important-3: var(--vp-c-purple-3);--vp-c-important-soft: var(--vp-c-purple-soft);--vp-c-warning-1: var(--vp-c-yellow-1);--vp-c-warning-2: var(--vp-c-yellow-2);--vp-c-warning-3: var(--vp-c-yellow-3);--vp-c-warning-soft: var(--vp-c-yellow-soft);--vp-c-danger-1: var(--vp-c-red-1);--vp-c-danger-2: var(--vp-c-red-2);--vp-c-danger-3: var(--vp-c-red-3);--vp-c-danger-soft: var(--vp-c-red-soft);--vp-c-caution-1: var(--vp-c-red-1);--vp-c-caution-2: var(--vp-c-red-2);--vp-c-caution-3: var(--vp-c-red-3);--vp-c-caution-soft: var(--vp-c-red-soft)}:root{--vp-font-family-base: "Inter", ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";--vp-font-family-mono: ui-monospace, "Menlo", "Monaco", "Consolas", "Liberation Mono", "Courier New", monospace;font-optical-sizing:auto}:root:where(:lang(zh)){--vp-font-family-base: "Punctuation SC", "Inter", ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"}:root{--vp-shadow-1: 0 1px 2px rgba(0, 0, 0, .04), 0 1px 2px rgba(0, 0, 0, .06);--vp-shadow-2: 0 3px 12px rgba(0, 0, 0, .07), 0 1px 4px rgba(0, 0, 0, .07);--vp-shadow-3: 0 12px 32px rgba(0, 0, 0, .1), 0 2px 6px rgba(0, 0, 0, .08);--vp-shadow-4: 0 14px 44px rgba(0, 0, 0, .12), 0 3px 9px rgba(0, 0, 0, .12);--vp-shadow-5: 0 18px 56px rgba(0, 0, 0, .16), 0 4px 12px rgba(0, 0, 0, .16)}:root{--vp-z-index-footer: 10;--vp-z-index-local-nav: 20;--vp-z-index-nav: 30;--vp-z-index-layout-top: 40;--vp-z-index-backdrop: 50;--vp-z-index-sidebar: 60}@media (min-width: 960px){:root{--vp-z-index-sidebar: 25}}:root{--vp-layout-max-width: 1440px}:root{--vp-header-anchor-symbol: "#"}:root{--vp-code-line-height: 1.7;--vp-code-font-size: .875em;--vp-code-color: var(--vp-c-brand-1);--vp-code-link-color: var(--vp-c-brand-1);--vp-code-link-hover-color: var(--vp-c-brand-2);--vp-code-bg: var(--vp-c-default-soft);--vp-code-block-color: var(--vp-c-text-2);--vp-code-block-bg: var(--vp-c-bg-alt);--vp-code-block-divider-color: var(--vp-c-gutter);--vp-code-lang-color: var(--vp-c-text-3);--vp-code-line-highlight-color: var(--vp-c-default-soft);--vp-code-line-number-color: var(--vp-c-text-3);--vp-code-line-diff-add-color: var(--vp-c-success-soft);--vp-code-line-diff-add-symbol-color: var(--vp-c-success-1);--vp-code-line-diff-remove-color: var(--vp-c-danger-soft);--vp-code-line-diff-remove-symbol-color: var(--vp-c-danger-1);--vp-code-line-warning-color: var(--vp-c-warning-soft);--vp-code-line-error-color: var(--vp-c-danger-soft);--vp-code-copy-code-border-color: var(--vp-c-divider);--vp-code-copy-code-bg: var(--vp-c-bg-soft);--vp-code-copy-code-hover-border-color: var(--vp-c-divider);--vp-code-copy-code-hover-bg: var(--vp-c-bg);--vp-code-copy-code-active-text: var(--vp-c-text-2);--vp-code-copy-copied-text-content: "Copied";--vp-code-tab-divider: var(--vp-code-block-divider-color);--vp-code-tab-text-color: var(--vp-c-text-2);--vp-code-tab-bg: var(--vp-code-block-bg);--vp-code-tab-hover-text-color: var(--vp-c-text-1);--vp-code-tab-active-text-color: var(--vp-c-text-1);--vp-code-tab-active-bar-color: var(--vp-c-brand-1)}:root{--vp-button-brand-border: transparent;--vp-button-brand-text: var(--vp-c-white);--vp-button-brand-bg: var(--vp-c-brand-3);--vp-button-brand-hover-border: transparent;--vp-button-brand-hover-text: var(--vp-c-white);--vp-button-brand-hover-bg: var(--vp-c-brand-2);--vp-button-brand-active-border: transparent;--vp-button-brand-active-text: var(--vp-c-white);--vp-button-brand-active-bg: var(--vp-c-brand-1);--vp-button-alt-border: transparent;--vp-button-alt-text: var(--vp-c-text-1);--vp-button-alt-bg: var(--vp-c-default-3);--vp-button-alt-hover-border: transparent;--vp-button-alt-hover-text: var(--vp-c-text-1);--vp-button-alt-hover-bg: var(--vp-c-default-2);--vp-button-alt-active-border: transparent;--vp-button-alt-active-text: var(--vp-c-text-1);--vp-button-alt-active-bg: var(--vp-c-default-1);--vp-button-sponsor-border: var(--vp-c-text-2);--vp-button-sponsor-text: var(--vp-c-text-2);--vp-button-sponsor-bg: transparent;--vp-button-sponsor-hover-border: var(--vp-c-sponsor);--vp-button-sponsor-hover-text: var(--vp-c-sponsor);--vp-button-sponsor-hover-bg: transparent;--vp-button-sponsor-active-border: var(--vp-c-sponsor);--vp-button-sponsor-active-text: var(--vp-c-sponsor);--vp-button-sponsor-active-bg: transparent}:root{--vp-custom-block-font-size: 14px;--vp-custom-block-code-font-size: 13px;--vp-custom-block-info-border: transparent;--vp-custom-block-info-text: var(--vp-c-text-1);--vp-custom-block-info-bg: var(--vp-c-default-soft);--vp-custom-block-info-code-bg: var(--vp-c-default-soft);--vp-custom-block-note-border: transparent;--vp-custom-block-note-text: var(--vp-c-text-1);--vp-custom-block-note-bg: var(--vp-c-default-soft);--vp-custom-block-note-code-bg: var(--vp-c-default-soft);--vp-custom-block-tip-border: transparent;--vp-custom-block-tip-text: var(--vp-c-text-1);--vp-custom-block-tip-bg: var(--vp-c-tip-soft);--vp-custom-block-tip-code-bg: var(--vp-c-tip-soft);--vp-custom-block-important-border: transparent;--vp-custom-block-important-text: var(--vp-c-text-1);--vp-custom-block-important-bg: var(--vp-c-important-soft);--vp-custom-block-important-code-bg: var(--vp-c-important-soft);--vp-custom-block-warning-border: transparent;--vp-custom-block-warning-text: var(--vp-c-text-1);--vp-custom-block-warning-bg: var(--vp-c-warning-soft);--vp-custom-block-warning-code-bg: var(--vp-c-warning-soft);--vp-custom-block-danger-border: transparent;--vp-custom-block-danger-text: var(--vp-c-text-1);--vp-custom-block-danger-bg: var(--vp-c-danger-soft);--vp-custom-block-danger-code-bg: var(--vp-c-danger-soft);--vp-custom-block-caution-border: transparent;--vp-custom-block-caution-text: var(--vp-c-text-1);--vp-custom-block-caution-bg: var(--vp-c-caution-soft);--vp-custom-block-caution-code-bg: var(--vp-c-caution-soft);--vp-custom-block-details-border: var(--vp-custom-block-info-border);--vp-custom-block-details-text: var(--vp-custom-block-info-text);--vp-custom-block-details-bg: var(--vp-custom-block-info-bg);--vp-custom-block-details-code-bg: var(--vp-custom-block-info-code-bg)}:root{--vp-input-border-color: var(--vp-c-border);--vp-input-bg-color: var(--vp-c-bg-alt);--vp-input-switch-bg-color: var(--vp-c-default-soft)}:root{--vp-nav-height: 64px;--vp-nav-bg-color: var(--vp-c-bg);--vp-nav-screen-bg-color: var(--vp-c-bg);--vp-nav-logo-height: 24px}.hide-nav{--vp-nav-height: 0px}.hide-nav .VPSidebar{--vp-nav-height: 22px}:root{--vp-local-nav-bg-color: var(--vp-c-bg)}:root{--vp-sidebar-width: 272px;--vp-sidebar-bg-color: var(--vp-c-bg-alt)}:root{--vp-backdrop-bg-color: rgba(0, 0, 0, .6)}:root{--vp-home-hero-name-color: var(--vp-c-brand-1);--vp-home-hero-name-background: transparent;--vp-home-hero-image-background-image: none;--vp-home-hero-image-filter: none}:root{--vp-badge-info-border: transparent;--vp-badge-info-text: var(--vp-c-text-2);--vp-badge-info-bg: var(--vp-c-default-soft);--vp-badge-tip-border: transparent;--vp-badge-tip-text: var(--vp-c-tip-1);--vp-badge-tip-bg: var(--vp-c-tip-soft);--vp-badge-warning-border: transparent;--vp-badge-warning-text: var(--vp-c-warning-1);--vp-badge-warning-bg: var(--vp-c-warning-soft);--vp-badge-danger-border: transparent;--vp-badge-danger-text: var(--vp-c-danger-1);--vp-badge-danger-bg: var(--vp-c-danger-soft)}:root{--vp-carbon-ads-text-color: var(--vp-c-text-1);--vp-carbon-ads-poweredby-color: var(--vp-c-text-2);--vp-carbon-ads-bg-color: var(--vp-c-bg-soft);--vp-carbon-ads-hover-text-color: var(--vp-c-brand-1);--vp-carbon-ads-hover-poweredby-color: var(--vp-c-text-1)}:root{--vp-local-search-bg: var(--vp-c-bg);--vp-local-search-result-bg: var(--vp-c-bg);--vp-local-search-result-border: var(--vp-c-divider);--vp-local-search-result-selected-bg: var(--vp-c-bg);--vp-local-search-result-selected-border: var(--vp-c-brand-1);--vp-local-search-highlight-bg: var(--vp-c-brand-1);--vp-local-search-highlight-text: var(--vp-c-neutral-inverse)}@media (prefers-reduced-motion: reduce){*,:before,:after{animation-delay:-1ms!important;animation-duration:1ms!important;animation-iteration-count:1!important;background-attachment:initial!important;scroll-behavior:auto!important;transition-duration:0s!important;transition-delay:0s!important}}*,:before,:after{box-sizing:border-box}html{line-height:1.4;font-size:16px;-webkit-text-size-adjust:100%}html.dark{color-scheme:dark}body{margin:0;width:100%;min-width:320px;min-height:100vh;line-height:24px;font-family:var(--vp-font-family-base);font-size:16px;font-weight:400;color:var(--vp-c-text-1);background-color:var(--vp-c-bg);font-synthesis:style;text-rendering:optimizeLegibility;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}main{display:block}h1,h2,h3,h4,h5,h6{margin:0;line-height:24px;font-size:16px;font-weight:400}p{margin:0}strong,b{font-weight:600}a,area,button,[role=button],input,label,select,summary,textarea{touch-action:manipulation}a{color:inherit;text-decoration:inherit}ol,ul{list-style:none;margin:0;padding:0}blockquote{margin:0}pre,code,kbd,samp{font-family:var(--vp-font-family-mono)}img,svg,video,canvas,audio,iframe,embed,object{display:block}figure{margin:0}img,video{max-width:100%;height:auto}button,input,optgroup,select,textarea{border:0;padding:0;line-height:inherit;color:inherit}button{padding:0;font-family:inherit;background-color:transparent;background-image:none}button:enabled,[role=button]:enabled{cursor:pointer}button:focus,button:focus-visible{outline:1px dotted;outline:4px auto -webkit-focus-ring-color}button:focus:not(:focus-visible){outline:none!important}input:focus,textarea:focus,select:focus{outline:none}table{border-collapse:collapse}input{background-color:transparent}input:-ms-input-placeholder,textarea:-ms-input-placeholder{color:var(--vp-c-text-3)}input::-ms-input-placeholder,textarea::-ms-input-placeholder{color:var(--vp-c-text-3)}input::placeholder,textarea::placeholder{color:var(--vp-c-text-3)}input::-webkit-outer-spin-button,input::-webkit-inner-spin-button{-webkit-appearance:none;margin:0}input[type=number]{-moz-appearance:textfield}textarea{resize:vertical}select{-webkit-appearance:none}fieldset{margin:0;padding:0}h1,h2,h3,h4,h5,h6,li,p{overflow-wrap:break-word}vite-error-overlay{z-index:9999}mjx-container{overflow-x:auto}mjx-container>svg{display:inline-block;margin:auto}[class^=vpi-],[class*=" vpi-"],.vp-icon{width:1em;height:1em}[class^=vpi-].bg,[class*=" vpi-"].bg,.vp-icon.bg{background-size:100% 100%;background-color:transparent}[class^=vpi-]:not(.bg),[class*=" vpi-"]:not(.bg),.vp-icon:not(.bg){-webkit-mask:var(--icon) no-repeat;mask:var(--icon) no-repeat;-webkit-mask-size:100% 100%;mask-size:100% 100%;background-color:currentColor;color:inherit}.vpi-align-left{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M21 6H3M15 12H3M17 18H3'/%3E%3C/svg%3E")}.vpi-arrow-right,.vpi-arrow-down,.vpi-arrow-left,.vpi-arrow-up{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M5 12h14M12 5l7 7-7 7'/%3E%3C/svg%3E")}.vpi-chevron-right,.vpi-chevron-down,.vpi-chevron-left,.vpi-chevron-up{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='m9 18 6-6-6-6'/%3E%3C/svg%3E")}.vpi-chevron-down,.vpi-arrow-down{transform:rotate(90deg)}.vpi-chevron-left,.vpi-arrow-left{transform:rotate(180deg)}.vpi-chevron-up,.vpi-arrow-up{transform:rotate(-90deg)}.vpi-square-pen{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M12 3H5a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h14a2 2 0 0 0 2-2v-7'/%3E%3Cpath d='M18.375 2.625a2.121 2.121 0 1 1 3 3L12 15l-4 1 1-4Z'/%3E%3C/svg%3E")}.vpi-plus{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M5 12h14M12 5v14'/%3E%3C/svg%3E")}.vpi-sun{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Ccircle cx='12' cy='12' r='4'/%3E%3Cpath d='M12 2v2M12 20v2M4.93 4.93l1.41 1.41M17.66 17.66l1.41 1.41M2 12h2M20 12h2M6.34 17.66l-1.41 1.41M19.07 4.93l-1.41 1.41'/%3E%3C/svg%3E")}.vpi-moon{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M12 3a6 6 0 0 0 9 9 9 9 0 1 1-9-9Z'/%3E%3C/svg%3E")}.vpi-more-horizontal{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Ccircle cx='12' cy='12' r='1'/%3E%3Ccircle cx='19' cy='12' r='1'/%3E%3Ccircle cx='5' cy='12' r='1'/%3E%3C/svg%3E")}.vpi-languages{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='m5 8 6 6M4 14l6-6 2-3M2 5h12M7 2h1M22 22l-5-10-5 10M14 18h6'/%3E%3C/svg%3E")}.vpi-heart{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M19 14c1.49-1.46 3-3.21 3-5.5A5.5 5.5 0 0 0 16.5 3c-1.76 0-3 .5-4.5 2-1.5-1.5-2.74-2-4.5-2A5.5 5.5 0 0 0 2 8.5c0 2.3 1.5 4.05 3 5.5l7 7Z'/%3E%3C/svg%3E")}.vpi-search{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Ccircle cx='11' cy='11' r='8'/%3E%3Cpath d='m21 21-4.3-4.3'/%3E%3C/svg%3E")}.vpi-layout-list{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Crect width='7' height='7' x='3' y='3' rx='1'/%3E%3Crect width='7' height='7' x='3' y='14' rx='1'/%3E%3Cpath d='M14 4h7M14 9h7M14 15h7M14 20h7'/%3E%3C/svg%3E")}.vpi-delete{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='M20 5H9l-7 7 7 7h11a2 2 0 0 0 2-2V7a2 2 0 0 0-2-2ZM18 9l-6 6M12 9l6 6'/%3E%3C/svg%3E")}.vpi-corner-down-left{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Cpath d='m9 10-5 5 5 5'/%3E%3Cpath d='M20 4v7a4 4 0 0 1-4 4H4'/%3E%3C/svg%3E")}:root{--vp-icon-copy: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='rgba(128,128,128,1)' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Crect width='8' height='4' x='8' y='2' rx='1' ry='1'/%3E%3Cpath d='M16 4h2a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2'/%3E%3C/svg%3E");--vp-icon-copied: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' stroke='rgba(128,128,128,1)' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' viewBox='0 0 24 24'%3E%3Crect width='8' height='4' x='8' y='2' rx='1' ry='1'/%3E%3Cpath d='M16 4h2a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h2'/%3E%3Cpath d='m9 14 2 2 4-4'/%3E%3C/svg%3E")}.vpi-social-discord{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M20.317 4.37a19.791 19.791 0 0 0-4.885-1.515.074.074 0 0 0-.079.037c-.21.375-.444.864-.608 1.25a18.27 18.27 0 0 0-5.487 0 12.64 12.64 0 0 0-.617-1.25.077.077 0 0 0-.079-.037A19.736 19.736 0 0 0 3.677 4.37a.07.07 0 0 0-.032.027C.533 9.046-.32 13.58.099 18.057a.082.082 0 0 0 .031.057 19.9 19.9 0 0 0 5.993 3.03.078.078 0 0 0 .084-.028c.462-.63.874-1.295 1.226-1.994a.076.076 0 0 0-.041-.106 13.107 13.107 0 0 1-1.872-.892.077.077 0 0 1-.008-.128 10.2 10.2 0 0 0 .372-.292.074.074 0 0 1 .077-.01c3.928 1.793 8.18 1.793 12.062 0a.074.074 0 0 1 .078.01c.12.098.246.198.373.292a.077.077 0 0 1-.006.127 12.299 12.299 0 0 1-1.873.892.077.077 0 0 0-.041.107c.36.698.772 1.362 1.225 1.993a.076.076 0 0 0 .084.028 19.839 19.839 0 0 0 6.002-3.03.077.077 0 0 0 .032-.054c.5-5.177-.838-9.674-3.549-13.66a.061.061 0 0 0-.031-.03zM8.02 15.33c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.956-2.419 2.157-2.419 1.21 0 2.176 1.096 2.157 2.42 0 1.333-.956 2.418-2.157 2.418zm7.975 0c-1.183 0-2.157-1.085-2.157-2.419 0-1.333.955-2.419 2.157-2.419 1.21 0 2.176 1.096 2.157 2.42 0 1.333-.946 2.418-2.157 2.418Z'/%3E%3C/svg%3E")}.vpi-social-facebook{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M9.101 23.691v-7.98H6.627v-3.667h2.474v-1.58c0-4.085 1.848-5.978 5.858-5.978.401 0 .955.042 1.468.103a8.68 8.68 0 0 1 1.141.195v3.325a8.623 8.623 0 0 0-.653-.036 26.805 26.805 0 0 0-.733-.009c-.707 0-1.259.096-1.675.309a1.686 1.686 0 0 0-.679.622c-.258.42-.374.995-.374 1.752v1.297h3.919l-.386 2.103-.287 1.564h-3.246v8.245C19.396 23.238 24 18.179 24 12.044c0-6.627-5.373-12-12-12s-12 5.373-12 12c0 5.628 3.874 10.35 9.101 11.647Z'/%3E%3C/svg%3E")}.vpi-social-github{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12'/%3E%3C/svg%3E")}.vpi-social-instagram{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M7.03.084c-1.277.06-2.149.264-2.91.563a5.874 5.874 0 0 0-2.124 1.388 5.878 5.878 0 0 0-1.38 2.127C.321 4.926.12 5.8.064 7.076.008 8.354-.005 8.764.001 12.023c.007 3.259.021 3.667.083 4.947.061 1.277.264 2.149.563 2.911.308.789.72 1.457 1.388 2.123a5.872 5.872 0 0 0 2.129 1.38c.763.295 1.636.496 2.913.552 1.278.056 1.689.069 4.947.063 3.257-.007 3.668-.021 4.947-.082 1.28-.06 2.147-.265 2.91-.563a5.881 5.881 0 0 0 2.123-1.388 5.881 5.881 0 0 0 1.38-2.129c.295-.763.496-1.636.551-2.912.056-1.28.07-1.69.063-4.948-.006-3.258-.02-3.667-.081-4.947-.06-1.28-.264-2.148-.564-2.911a5.892 5.892 0 0 0-1.387-2.123 5.857 5.857 0 0 0-2.128-1.38C19.074.322 18.202.12 16.924.066 15.647.009 15.236-.006 11.977 0 8.718.008 8.31.021 7.03.084m.14 21.693c-1.17-.05-1.805-.245-2.228-.408a3.736 3.736 0 0 1-1.382-.895 3.695 3.695 0 0 1-.9-1.378c-.165-.423-.363-1.058-.417-2.228-.06-1.264-.072-1.644-.08-4.848-.006-3.204.006-3.583.061-4.848.05-1.169.246-1.805.408-2.228.216-.561.477-.96.895-1.382a3.705 3.705 0 0 1 1.379-.9c.423-.165 1.057-.361 2.227-.417 1.265-.06 1.644-.072 4.848-.08 3.203-.006 3.583.006 4.85.062 1.168.05 1.804.244 2.227.408.56.216.96.475 1.382.895.421.42.681.817.9 1.378.165.422.362 1.056.417 2.227.06 1.265.074 1.645.08 4.848.005 3.203-.006 3.583-.061 4.848-.051 1.17-.245 1.805-.408 2.23-.216.56-.477.96-.896 1.38a3.705 3.705 0 0 1-1.378.9c-.422.165-1.058.362-2.226.418-1.266.06-1.645.072-4.85.079-3.204.007-3.582-.006-4.848-.06m9.783-16.192a1.44 1.44 0 1 0 1.437-1.442 1.44 1.44 0 0 0-1.437 1.442M5.839 12.012a6.161 6.161 0 1 0 12.323-.024 6.162 6.162 0 0 0-12.323.024M8 12.008A4 4 0 1 1 12.008 16 4 4 0 0 1 8 12.008'/%3E%3C/svg%3E")}.vpi-social-linkedin{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M20.447 20.452h-3.554v-5.569c0-1.328-.027-3.037-1.852-3.037-1.853 0-2.136 1.445-2.136 2.939v5.667H9.351V9h3.414v1.561h.046c.477-.9 1.637-1.85 3.37-1.85 3.601 0 4.267 2.37 4.267 5.455v6.286zM5.337 7.433a2.062 2.062 0 0 1-2.063-2.065 2.064 2.064 0 1 1 2.063 2.065zm1.782 13.019H3.555V9h3.564v11.452zM22.225 0H1.771C.792 0 0 .774 0 1.729v20.542C0 23.227.792 24 1.771 24h20.451C23.2 24 24 23.227 24 22.271V1.729C24 .774 23.2 0 22.222 0h.003z'/%3E%3C/svg%3E")}.vpi-social-mastodon{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M23.268 5.313c-.35-2.578-2.617-4.61-5.304-5.004C17.51.242 15.792 0 11.813 0h-.03c-3.98 0-4.835.242-5.288.309C3.882.692 1.496 2.518.917 5.127.64 6.412.61 7.837.661 9.143c.074 1.874.088 3.745.26 5.611.118 1.24.325 2.47.62 3.68.55 2.237 2.777 4.098 4.96 4.857 2.336.792 4.849.923 7.256.38.265-.061.527-.132.786-.213.585-.184 1.27-.39 1.774-.753a.057.057 0 0 0 .023-.043v-1.809a.052.052 0 0 0-.02-.041.053.053 0 0 0-.046-.01 20.282 20.282 0 0 1-4.709.545c-2.73 0-3.463-1.284-3.674-1.818a5.593 5.593 0 0 1-.319-1.433.053.053 0 0 1 .066-.054c1.517.363 3.072.546 4.632.546.376 0 .75 0 1.125-.01 1.57-.044 3.224-.124 4.768-.422.038-.008.077-.015.11-.024 2.435-.464 4.753-1.92 4.989-5.604.008-.145.03-1.52.03-1.67.002-.512.167-3.63-.024-5.545zm-3.748 9.195h-2.561V8.29c0-1.309-.55-1.976-1.67-1.976-1.23 0-1.846.79-1.846 2.35v3.403h-2.546V8.663c0-1.56-.617-2.35-1.848-2.35-1.112 0-1.668.668-1.67 1.977v6.218H4.822V8.102c0-1.31.337-2.35 1.011-3.12.696-.77 1.608-1.164 2.74-1.164 1.311 0 2.302.5 2.962 1.498l.638 1.06.638-1.06c.66-.999 1.65-1.498 2.96-1.498 1.13 0 2.043.395 2.74 1.164.675.77 1.012 1.81 1.012 3.12z'/%3E%3C/svg%3E")}.vpi-social-npm{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M1.763 0C.786 0 0 .786 0 1.763v20.474C0 23.214.786 24 1.763 24h20.474c.977 0 1.763-.786 1.763-1.763V1.763C24 .786 23.214 0 22.237 0zM5.13 5.323l13.837.019-.009 13.836h-3.464l.01-10.382h-3.456L12.04 19.17H5.113z'/%3E%3C/svg%3E")}.vpi-social-slack{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M5.042 15.165a2.528 2.528 0 0 1-2.52 2.523A2.528 2.528 0 0 1 0 15.165a2.527 2.527 0 0 1 2.522-2.52h2.52v2.52zm1.271 0a2.527 2.527 0 0 1 2.521-2.52 2.527 2.527 0 0 1 2.521 2.52v6.313A2.528 2.528 0 0 1 8.834 24a2.528 2.528 0 0 1-2.521-2.522v-6.313zM8.834 5.042a2.528 2.528 0 0 1-2.521-2.52A2.528 2.528 0 0 1 8.834 0a2.528 2.528 0 0 1 2.521 2.522v2.52H8.834zm0 1.271a2.528 2.528 0 0 1 2.521 2.521 2.528 2.528 0 0 1-2.521 2.521H2.522A2.528 2.528 0 0 1 0 8.834a2.528 2.528 0 0 1 2.522-2.521h6.312zm10.122 2.521a2.528 2.528 0 0 1 2.522-2.521A2.528 2.528 0 0 1 24 8.834a2.528 2.528 0 0 1-2.522 2.521h-2.522V8.834zm-1.268 0a2.528 2.528 0 0 1-2.523 2.521 2.527 2.527 0 0 1-2.52-2.521V2.522A2.527 2.527 0 0 1 15.165 0a2.528 2.528 0 0 1 2.523 2.522v6.312zm-2.523 10.122a2.528 2.528 0 0 1 2.523 2.522A2.528 2.528 0 0 1 15.165 24a2.527 2.527 0 0 1-2.52-2.522v-2.522h2.52zm0-1.268a2.527 2.527 0 0 1-2.52-2.523 2.526 2.526 0 0 1 2.52-2.52h6.313A2.527 2.527 0 0 1 24 15.165a2.528 2.528 0 0 1-2.522 2.523h-6.313z'/%3E%3C/svg%3E")}.vpi-social-twitter,.vpi-social-x{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M18.901 1.153h3.68l-8.04 9.19L24 22.846h-7.406l-5.8-7.584-6.638 7.584H.474l8.6-9.83L0 1.154h7.594l5.243 6.932ZM17.61 20.644h2.039L6.486 3.24H4.298Z'/%3E%3C/svg%3E")}.vpi-social-youtube{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath d='M23.498 6.186a3.016 3.016 0 0 0-2.122-2.136C19.505 3.545 12 3.545 12 3.545s-7.505 0-9.377.505A3.017 3.017 0 0 0 .502 6.186C0 8.07 0 12 0 12s0 3.93.502 5.814a3.016 3.016 0 0 0 2.122 2.136c1.871.505 9.376.505 9.376.505s7.505 0 9.377-.505a3.015 3.015 0 0 0 2.122-2.136C24 15.93 24 12 24 12s0-3.93-.502-5.814zM9.545 15.568V8.432L15.818 12l-6.273 3.568z'/%3E%3C/svg%3E")}.visually-hidden{position:absolute;width:1px;height:1px;white-space:nowrap;clip:rect(0 0 0 0);clip-path:inset(50%);overflow:hidden}.custom-block{border:1px solid transparent;border-radius:8px;padding:16px 16px 8px;line-height:24px;font-size:var(--vp-custom-block-font-size);color:var(--vp-c-text-2)}.custom-block.info{border-color:var(--vp-custom-block-info-border);color:var(--vp-custom-block-info-text);background-color:var(--vp-custom-block-info-bg)}.custom-block.info a,.custom-block.info code{color:var(--vp-c-brand-1)}.custom-block.info a:hover,.custom-block.info a:hover>code{color:var(--vp-c-brand-2)}.custom-block.info code{background-color:var(--vp-custom-block-info-code-bg)}.custom-block.note{border-color:var(--vp-custom-block-note-border);color:var(--vp-custom-block-note-text);background-color:var(--vp-custom-block-note-bg)}.custom-block.note a,.custom-block.note code{color:var(--vp-c-brand-1)}.custom-block.note a:hover,.custom-block.note a:hover>code{color:var(--vp-c-brand-2)}.custom-block.note code{background-color:var(--vp-custom-block-note-code-bg)}.custom-block.tip{border-color:var(--vp-custom-block-tip-border);color:var(--vp-custom-block-tip-text);background-color:var(--vp-custom-block-tip-bg)}.custom-block.tip a,.custom-block.tip code{color:var(--vp-c-tip-1)}.custom-block.tip a:hover,.custom-block.tip a:hover>code{color:var(--vp-c-tip-2)}.custom-block.tip code{background-color:var(--vp-custom-block-tip-code-bg)}.custom-block.important{border-color:var(--vp-custom-block-important-border);color:var(--vp-custom-block-important-text);background-color:var(--vp-custom-block-important-bg)}.custom-block.important a,.custom-block.important code{color:var(--vp-c-important-1)}.custom-block.important a:hover,.custom-block.important a:hover>code{color:var(--vp-c-important-2)}.custom-block.important code{background-color:var(--vp-custom-block-important-code-bg)}.custom-block.warning{border-color:var(--vp-custom-block-warning-border);color:var(--vp-custom-block-warning-text);background-color:var(--vp-custom-block-warning-bg)}.custom-block.warning a,.custom-block.warning code{color:var(--vp-c-warning-1)}.custom-block.warning a:hover,.custom-block.warning a:hover>code{color:var(--vp-c-warning-2)}.custom-block.warning code{background-color:var(--vp-custom-block-warning-code-bg)}.custom-block.danger{border-color:var(--vp-custom-block-danger-border);color:var(--vp-custom-block-danger-text);background-color:var(--vp-custom-block-danger-bg)}.custom-block.danger a,.custom-block.danger code{color:var(--vp-c-danger-1)}.custom-block.danger a:hover,.custom-block.danger a:hover>code{color:var(--vp-c-danger-2)}.custom-block.danger code{background-color:var(--vp-custom-block-danger-code-bg)}.custom-block.caution{border-color:var(--vp-custom-block-caution-border);color:var(--vp-custom-block-caution-text);background-color:var(--vp-custom-block-caution-bg)}.custom-block.caution a,.custom-block.caution code{color:var(--vp-c-caution-1)}.custom-block.caution a:hover,.custom-block.caution a:hover>code{color:var(--vp-c-caution-2)}.custom-block.caution code{background-color:var(--vp-custom-block-caution-code-bg)}.custom-block.details{border-color:var(--vp-custom-block-details-border);color:var(--vp-custom-block-details-text);background-color:var(--vp-custom-block-details-bg)}.custom-block.details a{color:var(--vp-c-brand-1)}.custom-block.details a:hover,.custom-block.details a:hover>code{color:var(--vp-c-brand-2)}.custom-block.details code{background-color:var(--vp-custom-block-details-code-bg)}.custom-block-title{font-weight:600}.custom-block p+p{margin:8px 0}.custom-block.details summary{margin:0 0 8px;font-weight:700;cursor:pointer;-webkit-user-select:none;user-select:none}.custom-block.details summary+p{margin:8px 0}.custom-block a{color:inherit;font-weight:600;text-decoration:underline;text-underline-offset:2px;transition:opacity .25s}.custom-block a:hover{opacity:.75}.custom-block code{font-size:var(--vp-custom-block-code-font-size)}.custom-block.custom-block th,.custom-block.custom-block blockquote>p{font-size:var(--vp-custom-block-font-size);color:inherit}.dark .vp-code span{color:var(--shiki-dark, inherit)}html:not(.dark) .vp-code span{color:var(--shiki-light, inherit)}.vp-code-group{margin-top:16px}.vp-code-group .tabs{position:relative;display:flex;margin-right:-24px;margin-left:-24px;padding:0 12px;background-color:var(--vp-code-tab-bg);overflow-x:auto;overflow-y:hidden;box-shadow:inset 0 -1px var(--vp-code-tab-divider)}@media (min-width: 640px){.vp-code-group .tabs{margin-right:0;margin-left:0;border-radius:8px 8px 0 0}}.vp-code-group .tabs input{position:fixed;opacity:0;pointer-events:none}.vp-code-group .tabs label{position:relative;display:inline-block;border-bottom:1px solid transparent;padding:0 12px;line-height:48px;font-size:14px;font-weight:500;color:var(--vp-code-tab-text-color);white-space:nowrap;cursor:pointer;transition:color .25s}.vp-code-group .tabs label:after{position:absolute;right:8px;bottom:-1px;left:8px;z-index:1;height:2px;border-radius:2px;content:"";background-color:transparent;transition:background-color .25s}.vp-code-group label:hover{color:var(--vp-code-tab-hover-text-color)}.vp-code-group input:checked+label{color:var(--vp-code-tab-active-text-color)}.vp-code-group input:checked+label:after{background-color:var(--vp-code-tab-active-bar-color)}.vp-code-group div[class*=language-],.vp-block{display:none;margin-top:0!important;border-top-left-radius:0!important;border-top-right-radius:0!important}.vp-code-group div[class*=language-].active,.vp-block.active{display:block}.vp-block{padding:20px 24px}.vp-doc h1,.vp-doc h2,.vp-doc h3,.vp-doc h4,.vp-doc h5,.vp-doc h6{position:relative;font-weight:600;outline:none}.vp-doc h1{letter-spacing:-.02em;line-height:40px;font-size:28px}.vp-doc h2{margin:48px 0 16px;border-top:1px solid var(--vp-c-divider);padding-top:24px;letter-spacing:-.02em;line-height:32px;font-size:24px}.vp-doc h3{margin:32px 0 0;letter-spacing:-.01em;line-height:28px;font-size:20px}.vp-doc h4{margin:24px 0 0;letter-spacing:-.01em;line-height:24px;font-size:18px}.vp-doc .header-anchor{position:absolute;top:0;left:0;margin-left:-.87em;font-weight:500;-webkit-user-select:none;user-select:none;opacity:0;text-decoration:none;transition:color .25s,opacity .25s}.vp-doc .header-anchor:before{content:var(--vp-header-anchor-symbol)}.vp-doc h1:hover .header-anchor,.vp-doc h1 .header-anchor:focus,.vp-doc h2:hover .header-anchor,.vp-doc h2 .header-anchor:focus,.vp-doc h3:hover .header-anchor,.vp-doc h3 .header-anchor:focus,.vp-doc h4:hover .header-anchor,.vp-doc h4 .header-anchor:focus,.vp-doc h5:hover .header-anchor,.vp-doc h5 .header-anchor:focus,.vp-doc h6:hover .header-anchor,.vp-doc h6 .header-anchor:focus{opacity:1}@media (min-width: 768px){.vp-doc h1{letter-spacing:-.02em;line-height:40px;font-size:32px}}.vp-doc h2 .header-anchor{top:24px}.vp-doc p,.vp-doc summary{margin:16px 0}.vp-doc p{line-height:28px}.vp-doc blockquote{margin:16px 0;border-left:2px solid var(--vp-c-divider);padding-left:16px;transition:border-color .5s;color:var(--vp-c-text-2)}.vp-doc blockquote>p{margin:0;font-size:16px;transition:color .5s}.vp-doc a{font-weight:500;color:var(--vp-c-brand-1);text-decoration:underline;text-underline-offset:2px;transition:color .25s,opacity .25s}.vp-doc a:hover{color:var(--vp-c-brand-2)}.vp-doc strong{font-weight:600}.vp-doc ul,.vp-doc ol{padding-left:1.25rem;margin:16px 0}.vp-doc ul{list-style:disc}.vp-doc ol{list-style:decimal}.vp-doc li+li{margin-top:8px}.vp-doc li>ol,.vp-doc li>ul{margin:8px 0 0}.vp-doc table{display:block;border-collapse:collapse;margin:20px 0;overflow-x:auto}.vp-doc tr{background-color:var(--vp-c-bg);border-top:1px solid var(--vp-c-divider);transition:background-color .5s}.vp-doc tr:nth-child(2n){background-color:var(--vp-c-bg-soft)}.vp-doc th,.vp-doc td{border:1px solid var(--vp-c-divider);padding:8px 16px}.vp-doc th{text-align:left;font-size:14px;font-weight:600;color:var(--vp-c-text-2);background-color:var(--vp-c-bg-soft)}.vp-doc td{font-size:14px}.vp-doc hr{margin:16px 0;border:none;border-top:1px solid var(--vp-c-divider)}.vp-doc .custom-block{margin:16px 0}.vp-doc .custom-block p{margin:8px 0;line-height:24px}.vp-doc .custom-block p:first-child{margin:0}.vp-doc .custom-block div[class*=language-]{margin:8px 0;border-radius:8px}.vp-doc .custom-block div[class*=language-] code{font-weight:400;background-color:transparent}.vp-doc .custom-block .vp-code-group .tabs{margin:0;border-radius:8px 8px 0 0}.vp-doc :not(pre,h1,h2,h3,h4,h5,h6)>code{font-size:var(--vp-code-font-size);color:var(--vp-code-color)}.vp-doc :not(pre)>code{border-radius:4px;padding:3px 6px;background-color:var(--vp-code-bg);transition:color .25s,background-color .5s}.vp-doc a>code{color:var(--vp-code-link-color)}.vp-doc a:hover>code{color:var(--vp-code-link-hover-color)}.vp-doc h1>code,.vp-doc h2>code,.vp-doc h3>code,.vp-doc h4>code{font-size:.9em}.vp-doc div[class*=language-],.vp-block{position:relative;margin:16px -24px;background-color:var(--vp-code-block-bg);overflow-x:auto;transition:background-color .5s}@media (min-width: 640px){.vp-doc div[class*=language-],.vp-block{border-radius:8px;margin:16px 0}}@media (max-width: 639px){.vp-doc li div[class*=language-]{border-radius:8px 0 0 8px}}.vp-doc div[class*=language-]+div[class*=language-],.vp-doc div[class$=-api]+div[class*=language-],.vp-doc div[class*=language-]+div[class$=-api]>div[class*=language-]{margin-top:-8px}.vp-doc [class*=language-] pre,.vp-doc [class*=language-] code{direction:ltr;text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none}.vp-doc [class*=language-] pre{position:relative;z-index:1;margin:0;padding:20px 0;background:transparent;overflow-x:auto}.vp-doc [class*=language-] code{display:block;padding:0 24px;width:fit-content;min-width:100%;line-height:var(--vp-code-line-height);font-size:var(--vp-code-font-size);color:var(--vp-code-block-color);transition:color .5s}.vp-doc [class*=language-] code .highlighted{background-color:var(--vp-code-line-highlight-color);transition:background-color .5s;margin:0 -24px;padding:0 24px;width:calc(100% + 48px);display:inline-block}.vp-doc [class*=language-] code .highlighted.error{background-color:var(--vp-code-line-error-color)}.vp-doc [class*=language-] code .highlighted.warning{background-color:var(--vp-code-line-warning-color)}.vp-doc [class*=language-] code .diff{transition:background-color .5s;margin:0 -24px;padding:0 24px;width:calc(100% + 48px);display:inline-block}.vp-doc [class*=language-] code .diff:before{position:absolute;left:10px}.vp-doc [class*=language-] .has-focused-lines .line:not(.has-focus){filter:blur(.095rem);opacity:.4;transition:filter .35s,opacity .35s}.vp-doc [class*=language-] .has-focused-lines .line:not(.has-focus){opacity:.7;transition:filter .35s,opacity .35s}.vp-doc [class*=language-]:hover .has-focused-lines .line:not(.has-focus){filter:blur(0);opacity:1}.vp-doc [class*=language-] code .diff.remove{background-color:var(--vp-code-line-diff-remove-color);opacity:.7}.vp-doc [class*=language-] code .diff.remove:before{content:"-";color:var(--vp-code-line-diff-remove-symbol-color)}.vp-doc [class*=language-] code .diff.add{background-color:var(--vp-code-line-diff-add-color)}.vp-doc [class*=language-] code .diff.add:before{content:"+";color:var(--vp-code-line-diff-add-symbol-color)}.vp-doc div[class*=language-].line-numbers-mode{padding-left:32px}.vp-doc .line-numbers-wrapper{position:absolute;top:0;bottom:0;left:0;z-index:3;border-right:1px solid var(--vp-code-block-divider-color);padding-top:20px;width:32px;text-align:center;font-family:var(--vp-font-family-mono);line-height:var(--vp-code-line-height);font-size:var(--vp-code-font-size);color:var(--vp-code-line-number-color);transition:border-color .5s,color .5s}.vp-doc [class*=language-]>button.copy{direction:ltr;position:absolute;top:12px;right:12px;z-index:3;border:1px solid var(--vp-code-copy-code-border-color);border-radius:4px;width:40px;height:40px;background-color:var(--vp-code-copy-code-bg);opacity:0;cursor:pointer;background-image:var(--vp-icon-copy);background-position:50%;background-size:20px;background-repeat:no-repeat;transition:border-color .25s,background-color .25s,opacity .25s}.vp-doc [class*=language-]:hover>button.copy,.vp-doc [class*=language-]>button.copy:focus{opacity:1}.vp-doc [class*=language-]>button.copy:hover,.vp-doc [class*=language-]>button.copy.copied{border-color:var(--vp-code-copy-code-hover-border-color);background-color:var(--vp-code-copy-code-hover-bg)}.vp-doc [class*=language-]>button.copy.copied,.vp-doc [class*=language-]>button.copy:hover.copied{border-radius:0 4px 4px 0;background-color:var(--vp-code-copy-code-hover-bg);background-image:var(--vp-icon-copied)}.vp-doc [class*=language-]>button.copy.copied:before,.vp-doc [class*=language-]>button.copy:hover.copied:before{position:relative;top:-1px;transform:translate(calc(-100% - 1px));display:flex;justify-content:center;align-items:center;border:1px solid var(--vp-code-copy-code-hover-border-color);border-right:0;border-radius:4px 0 0 4px;padding:0 10px;width:fit-content;height:40px;text-align:center;font-size:12px;font-weight:500;color:var(--vp-code-copy-code-active-text);background-color:var(--vp-code-copy-code-hover-bg);white-space:nowrap;content:var(--vp-code-copy-copied-text-content)}.vp-doc [class*=language-]>span.lang{position:absolute;top:2px;right:8px;z-index:2;font-size:12px;font-weight:500;-webkit-user-select:none;user-select:none;color:var(--vp-code-lang-color);transition:color .4s,opacity .4s}.vp-doc [class*=language-]:hover>button.copy+span.lang,.vp-doc [class*=language-]>button.copy:focus+span.lang{opacity:0}.vp-doc .VPTeamMembers{margin-top:24px}.vp-doc .VPTeamMembers.small.count-1 .container{margin:0!important;max-width:calc((100% - 24px)/2)!important}.vp-doc .VPTeamMembers.small.count-2 .container,.vp-doc .VPTeamMembers.small.count-3 .container{max-width:100%!important}.vp-doc .VPTeamMembers.medium.count-1 .container{margin:0!important;max-width:calc((100% - 24px)/2)!important}:is(.vp-external-link-icon,.vp-doc a[href*="://"],.vp-doc a[target=_blank]):not(.no-icon):after{display:inline-block;margin-top:-1px;margin-left:4px;width:11px;height:11px;background:currentColor;color:var(--vp-c-text-3);flex-shrink:0;--icon: url("data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24' %3E%3Cpath d='M0 0h24v24H0V0z' fill='none' /%3E%3Cpath d='M9 5v2h6.59L4 18.59 5.41 20 17 8.41V15h2V5H9z' /%3E%3C/svg%3E");-webkit-mask-image:var(--icon);mask-image:var(--icon)}.vp-external-link-icon:after{content:""}.external-link-icon-enabled :is(.vp-doc a[href*="://"],.vp-doc a[target=_blank]):after{content:"";color:currentColor}.vp-sponsor{border-radius:16px;overflow:hidden}.vp-sponsor.aside{border-radius:12px}.vp-sponsor-section+.vp-sponsor-section{margin-top:4px}.vp-sponsor-tier{margin:0 0 4px!important;text-align:center;letter-spacing:1px!important;line-height:24px;width:100%;font-weight:600;color:var(--vp-c-text-2);background-color:var(--vp-c-bg-soft)}.vp-sponsor.normal .vp-sponsor-tier{padding:13px 0 11px;font-size:14px}.vp-sponsor.aside .vp-sponsor-tier{padding:9px 0 7px;font-size:12px}.vp-sponsor-grid+.vp-sponsor-tier{margin-top:4px}.vp-sponsor-grid{display:flex;flex-wrap:wrap;gap:4px}.vp-sponsor-grid.xmini .vp-sponsor-grid-link{height:64px}.vp-sponsor-grid.xmini .vp-sponsor-grid-image{max-width:64px;max-height:22px}.vp-sponsor-grid.mini .vp-sponsor-grid-link{height:72px}.vp-sponsor-grid.mini .vp-sponsor-grid-image{max-width:96px;max-height:24px}.vp-sponsor-grid.small .vp-sponsor-grid-link{height:96px}.vp-sponsor-grid.small .vp-sponsor-grid-image{max-width:96px;max-height:24px}.vp-sponsor-grid.medium .vp-sponsor-grid-link{height:112px}.vp-sponsor-grid.medium .vp-sponsor-grid-image{max-width:120px;max-height:36px}.vp-sponsor-grid.big .vp-sponsor-grid-link{height:184px}.vp-sponsor-grid.big .vp-sponsor-grid-image{max-width:192px;max-height:56px}.vp-sponsor-grid[data-vp-grid="2"] .vp-sponsor-grid-item{width:calc((100% - 4px)/2)}.vp-sponsor-grid[data-vp-grid="3"] .vp-sponsor-grid-item{width:calc((100% - 4px * 2) / 3)}.vp-sponsor-grid[data-vp-grid="4"] .vp-sponsor-grid-item{width:calc((100% - 12px)/4)}.vp-sponsor-grid[data-vp-grid="5"] .vp-sponsor-grid-item{width:calc((100% - 16px)/5)}.vp-sponsor-grid[data-vp-grid="6"] .vp-sponsor-grid-item{width:calc((100% - 4px * 5) / 6)}.vp-sponsor-grid-item{flex-shrink:0;width:100%;background-color:var(--vp-c-bg-soft);transition:background-color .25s}.vp-sponsor-grid-item:hover{background-color:var(--vp-c-default-soft)}.vp-sponsor-grid-item:hover .vp-sponsor-grid-image{filter:grayscale(0) invert(0)}.vp-sponsor-grid-item.empty:hover{background-color:var(--vp-c-bg-soft)}.dark .vp-sponsor-grid-item:hover{background-color:var(--vp-c-white)}.dark .vp-sponsor-grid-item.empty:hover{background-color:var(--vp-c-bg-soft)}.vp-sponsor-grid-link{display:flex}.vp-sponsor-grid-box{display:flex;justify-content:center;align-items:center;width:100%}.vp-sponsor-grid-image{max-width:100%;filter:grayscale(1);transition:filter .25s}.dark .vp-sponsor-grid-image{filter:grayscale(1) invert(1)}.VPBadge{display:inline-block;margin-left:2px;border:1px solid transparent;border-radius:12px;padding:0 10px;line-height:22px;font-size:12px;font-weight:500;transform:translateY(-2px)}.VPBadge.small{padding:0 6px;line-height:18px;font-size:10px;transform:translateY(-8px)}.VPDocFooter .VPBadge{display:none}.vp-doc h1>.VPBadge{margin-top:4px;vertical-align:top}.vp-doc h2>.VPBadge{margin-top:3px;padding:0 8px;vertical-align:top}.vp-doc h3>.VPBadge{vertical-align:middle}.vp-doc h4>.VPBadge,.vp-doc h5>.VPBadge,.vp-doc h6>.VPBadge{vertical-align:middle;line-height:18px}.VPBadge.info{border-color:var(--vp-badge-info-border);color:var(--vp-badge-info-text);background-color:var(--vp-badge-info-bg)}.VPBadge.tip{border-color:var(--vp-badge-tip-border);color:var(--vp-badge-tip-text);background-color:var(--vp-badge-tip-bg)}.VPBadge.warning{border-color:var(--vp-badge-warning-border);color:var(--vp-badge-warning-text);background-color:var(--vp-badge-warning-bg)}.VPBadge.danger{border-color:var(--vp-badge-danger-border);color:var(--vp-badge-danger-text);background-color:var(--vp-badge-danger-bg)}.VPBackdrop[data-v-b06cdb19]{position:fixed;top:0;right:0;bottom:0;left:0;z-index:var(--vp-z-index-backdrop);background:var(--vp-backdrop-bg-color);transition:opacity .5s}.VPBackdrop.fade-enter-from[data-v-b06cdb19],.VPBackdrop.fade-leave-to[data-v-b06cdb19]{opacity:0}.VPBackdrop.fade-leave-active[data-v-b06cdb19]{transition-duration:.25s}@media (min-width: 1280px){.VPBackdrop[data-v-b06cdb19]{display:none}}.NotFound[data-v-951cab6c]{padding:64px 24px 96px;text-align:center}@media (min-width: 768px){.NotFound[data-v-951cab6c]{padding:96px 32px 168px}}.code[data-v-951cab6c]{line-height:64px;font-size:64px;font-weight:600}.title[data-v-951cab6c]{padding-top:12px;letter-spacing:2px;line-height:20px;font-size:20px;font-weight:700}.divider[data-v-951cab6c]{margin:24px auto 18px;width:64px;height:1px;background-color:var(--vp-c-divider)}.quote[data-v-951cab6c]{margin:0 auto;max-width:256px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}.action[data-v-951cab6c]{padding-top:20px}.link[data-v-951cab6c]{display:inline-block;border:1px solid var(--vp-c-brand-1);border-radius:16px;padding:3px 16px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1);transition:border-color .25s,color .25s}.link[data-v-951cab6c]:hover{border-color:var(--vp-c-brand-2);color:var(--vp-c-brand-2)}.root[data-v-3f927ebe]{position:relative;z-index:1}.nested[data-v-3f927ebe]{padding-right:16px;padding-left:16px}.outline-link[data-v-3f927ebe]{display:block;line-height:32px;font-size:14px;font-weight:400;color:var(--vp-c-text-2);white-space:nowrap;overflow:hidden;text-overflow:ellipsis;transition:color .5s}.outline-link[data-v-3f927ebe]:hover,.outline-link.active[data-v-3f927ebe]{color:var(--vp-c-text-1);transition:color .25s}.outline-link.nested[data-v-3f927ebe]{padding-left:13px}.VPDocAsideOutline[data-v-b38bf2ff]{display:none}.VPDocAsideOutline.has-outline[data-v-b38bf2ff]{display:block}.content[data-v-b38bf2ff]{position:relative;border-left:1px solid var(--vp-c-divider);padding-left:16px;font-size:13px;font-weight:500}.outline-marker[data-v-b38bf2ff]{position:absolute;top:32px;left:-1px;z-index:0;opacity:0;width:2px;border-radius:2px;height:18px;background-color:var(--vp-c-brand-1);transition:top .25s cubic-bezier(0,1,.5,1),background-color .5s,opacity .25s}.outline-title[data-v-b38bf2ff]{line-height:32px;font-size:14px;font-weight:600}.VPDocAside[data-v-6d7b3c46]{display:flex;flex-direction:column;flex-grow:1}.spacer[data-v-6d7b3c46]{flex-grow:1}.VPDocAside[data-v-6d7b3c46] .spacer+.VPDocAsideSponsors,.VPDocAside[data-v-6d7b3c46] .spacer+.VPDocAsideCarbonAds{margin-top:24px}.VPDocAside[data-v-6d7b3c46] .VPDocAsideSponsors+.VPDocAsideCarbonAds{margin-top:16px}.VPLastUpdated[data-v-475f71b8]{line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}@media (min-width: 640px){.VPLastUpdated[data-v-475f71b8]{line-height:32px;font-size:14px;font-weight:500}}.VPDocFooter[data-v-4f9813fa]{margin-top:64px}.edit-info[data-v-4f9813fa]{padding-bottom:18px}@media (min-width: 640px){.edit-info[data-v-4f9813fa]{display:flex;justify-content:space-between;align-items:center;padding-bottom:14px}}.edit-link-button[data-v-4f9813fa]{display:flex;align-items:center;border:0;line-height:32px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1);transition:color .25s}.edit-link-button[data-v-4f9813fa]:hover{color:var(--vp-c-brand-2)}.edit-link-icon[data-v-4f9813fa]{margin-right:8px}.prev-next[data-v-4f9813fa]{border-top:1px solid var(--vp-c-divider);padding-top:24px;display:grid;grid-row-gap:8px}@media (min-width: 640px){.prev-next[data-v-4f9813fa]{grid-template-columns:repeat(2,1fr);grid-column-gap:16px}}.pager-link[data-v-4f9813fa]{display:block;border:1px solid var(--vp-c-divider);border-radius:8px;padding:11px 16px 13px;width:100%;height:100%;transition:border-color .25s}.pager-link[data-v-4f9813fa]:hover{border-color:var(--vp-c-brand-1)}.pager-link.next[data-v-4f9813fa]{margin-left:auto;text-align:right}.desc[data-v-4f9813fa]{display:block;line-height:20px;font-size:12px;font-weight:500;color:var(--vp-c-text-2)}.title[data-v-4f9813fa]{display:block;line-height:20px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1);transition:color .25s}.VPDoc[data-v-83890dd9]{padding:32px 24px 96px;width:100%}@media (min-width: 768px){.VPDoc[data-v-83890dd9]{padding:48px 32px 128px}}@media (min-width: 960px){.VPDoc[data-v-83890dd9]{padding:48px 32px 0}.VPDoc:not(.has-sidebar) .container[data-v-83890dd9]{display:flex;justify-content:center;max-width:992px}.VPDoc:not(.has-sidebar) .content[data-v-83890dd9]{max-width:752px}}@media (min-width: 1280px){.VPDoc .container[data-v-83890dd9]{display:flex;justify-content:center}.VPDoc .aside[data-v-83890dd9]{display:block}}@media (min-width: 1440px){.VPDoc:not(.has-sidebar) .content[data-v-83890dd9]{max-width:784px}.VPDoc:not(.has-sidebar) .container[data-v-83890dd9]{max-width:1104px}}.container[data-v-83890dd9]{margin:0 auto;width:100%}.aside[data-v-83890dd9]{position:relative;display:none;order:2;flex-grow:1;padding-left:32px;width:100%;max-width:256px}.left-aside[data-v-83890dd9]{order:1;padding-left:unset;padding-right:32px}.aside-container[data-v-83890dd9]{position:fixed;top:0;padding-top:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + var(--vp-doc-top-height, 0px) + 48px);width:224px;height:100vh;overflow-x:hidden;overflow-y:auto;scrollbar-width:none}.aside-container[data-v-83890dd9]::-webkit-scrollbar{display:none}.aside-curtain[data-v-83890dd9]{position:fixed;bottom:0;z-index:10;width:224px;height:32px;background:linear-gradient(transparent,var(--vp-c-bg) 70%)}.aside-content[data-v-83890dd9]{display:flex;flex-direction:column;min-height:calc(100vh - (var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 48px));padding-bottom:32px}.content[data-v-83890dd9]{position:relative;margin:0 auto;width:100%}@media (min-width: 960px){.content[data-v-83890dd9]{padding:0 32px 128px}}@media (min-width: 1280px){.content[data-v-83890dd9]{order:1;margin:0;min-width:640px}}.content-container[data-v-83890dd9]{margin:0 auto}.VPDoc.has-aside .content-container[data-v-83890dd9]{max-width:688px}.VPButton[data-v-906d7fb4]{display:inline-block;border:1px solid transparent;text-align:center;font-weight:600;white-space:nowrap;transition:color .25s,border-color .25s,background-color .25s}.VPButton[data-v-906d7fb4]:active{transition:color .1s,border-color .1s,background-color .1s}.VPButton.medium[data-v-906d7fb4]{border-radius:20px;padding:0 20px;line-height:38px;font-size:14px}.VPButton.big[data-v-906d7fb4]{border-radius:24px;padding:0 24px;line-height:46px;font-size:16px}.VPButton.brand[data-v-906d7fb4]{border-color:var(--vp-button-brand-border);color:var(--vp-button-brand-text);background-color:var(--vp-button-brand-bg)}.VPButton.brand[data-v-906d7fb4]:hover{border-color:var(--vp-button-brand-hover-border);color:var(--vp-button-brand-hover-text);background-color:var(--vp-button-brand-hover-bg)}.VPButton.brand[data-v-906d7fb4]:active{border-color:var(--vp-button-brand-active-border);color:var(--vp-button-brand-active-text);background-color:var(--vp-button-brand-active-bg)}.VPButton.alt[data-v-906d7fb4]{border-color:var(--vp-button-alt-border);color:var(--vp-button-alt-text);background-color:var(--vp-button-alt-bg)}.VPButton.alt[data-v-906d7fb4]:hover{border-color:var(--vp-button-alt-hover-border);color:var(--vp-button-alt-hover-text);background-color:var(--vp-button-alt-hover-bg)}.VPButton.alt[data-v-906d7fb4]:active{border-color:var(--vp-button-alt-active-border);color:var(--vp-button-alt-active-text);background-color:var(--vp-button-alt-active-bg)}.VPButton.sponsor[data-v-906d7fb4]{border-color:var(--vp-button-sponsor-border);color:var(--vp-button-sponsor-text);background-color:var(--vp-button-sponsor-bg)}.VPButton.sponsor[data-v-906d7fb4]:hover{border-color:var(--vp-button-sponsor-hover-border);color:var(--vp-button-sponsor-hover-text);background-color:var(--vp-button-sponsor-hover-bg)}.VPButton.sponsor[data-v-906d7fb4]:active{border-color:var(--vp-button-sponsor-active-border);color:var(--vp-button-sponsor-active-text);background-color:var(--vp-button-sponsor-active-bg)}html:not(.dark) .VPImage.dark[data-v-35a7d0b8]{display:none}.dark .VPImage.light[data-v-35a7d0b8]{display:none}.VPHero[data-v-955009fc]{margin-top:calc((var(--vp-nav-height) + var(--vp-layout-top-height, 0px)) * -1);padding:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 48px) 24px 48px}@media (min-width: 640px){.VPHero[data-v-955009fc]{padding:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 80px) 48px 64px}}@media (min-width: 960px){.VPHero[data-v-955009fc]{padding:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px) + 80px) 64px 64px}}.container[data-v-955009fc]{display:flex;flex-direction:column;margin:0 auto;max-width:1152px}@media (min-width: 960px){.container[data-v-955009fc]{flex-direction:row}}.main[data-v-955009fc]{position:relative;z-index:10;order:2;flex-grow:1;flex-shrink:0}.VPHero.has-image .container[data-v-955009fc]{text-align:center}@media (min-width: 960px){.VPHero.has-image .container[data-v-955009fc]{text-align:left}}@media (min-width: 960px){.main[data-v-955009fc]{order:1;width:calc((100% / 3) * 2)}.VPHero.has-image .main[data-v-955009fc]{max-width:592px}}.name[data-v-955009fc],.text[data-v-955009fc]{max-width:392px;letter-spacing:-.4px;line-height:40px;font-size:32px;font-weight:700;white-space:pre-wrap}.VPHero.has-image .name[data-v-955009fc],.VPHero.has-image .text[data-v-955009fc]{margin:0 auto}.name[data-v-955009fc]{color:var(--vp-home-hero-name-color)}.clip[data-v-955009fc]{background:var(--vp-home-hero-name-background);-webkit-background-clip:text;background-clip:text;-webkit-text-fill-color:var(--vp-home-hero-name-color)}@media (min-width: 640px){.name[data-v-955009fc],.text[data-v-955009fc]{max-width:576px;line-height:56px;font-size:48px}}@media (min-width: 960px){.name[data-v-955009fc],.text[data-v-955009fc]{line-height:64px;font-size:56px}.VPHero.has-image .name[data-v-955009fc],.VPHero.has-image .text[data-v-955009fc]{margin:0}}.tagline[data-v-955009fc]{padding-top:8px;max-width:392px;line-height:28px;font-size:18px;font-weight:500;white-space:pre-wrap;color:var(--vp-c-text-2)}.VPHero.has-image .tagline[data-v-955009fc]{margin:0 auto}@media (min-width: 640px){.tagline[data-v-955009fc]{padding-top:12px;max-width:576px;line-height:32px;font-size:20px}}@media (min-width: 960px){.tagline[data-v-955009fc]{line-height:36px;font-size:24px}.VPHero.has-image .tagline[data-v-955009fc]{margin:0}}.actions[data-v-955009fc]{display:flex;flex-wrap:wrap;margin:-6px;padding-top:24px}.VPHero.has-image .actions[data-v-955009fc]{justify-content:center}@media (min-width: 640px){.actions[data-v-955009fc]{padding-top:32px}}@media (min-width: 960px){.VPHero.has-image .actions[data-v-955009fc]{justify-content:flex-start}}.action[data-v-955009fc]{flex-shrink:0;padding:6px}.image[data-v-955009fc]{order:1;margin:-76px -24px -48px}@media (min-width: 640px){.image[data-v-955009fc]{margin:-108px -24px -48px}}@media (min-width: 960px){.image[data-v-955009fc]{flex-grow:1;order:2;margin:0;min-height:100%}}.image-container[data-v-955009fc]{position:relative;margin:0 auto;width:320px;height:320px}@media (min-width: 640px){.image-container[data-v-955009fc]{width:392px;height:392px}}@media (min-width: 960px){.image-container[data-v-955009fc]{display:flex;justify-content:center;align-items:center;width:100%;height:100%;transform:translate(-32px,-32px)}}.image-bg[data-v-955009fc]{position:absolute;top:50%;left:50%;border-radius:50%;width:192px;height:192px;background-image:var(--vp-home-hero-image-background-image);filter:var(--vp-home-hero-image-filter);transform:translate(-50%,-50%)}@media (min-width: 640px){.image-bg[data-v-955009fc]{width:256px;height:256px}}@media (min-width: 960px){.image-bg[data-v-955009fc]{width:320px;height:320px}}[data-v-955009fc] .image-src{position:absolute;top:50%;left:50%;max-width:192px;max-height:192px;transform:translate(-50%,-50%)}@media (min-width: 640px){[data-v-955009fc] .image-src{max-width:256px;max-height:256px}}@media (min-width: 960px){[data-v-955009fc] .image-src{max-width:320px;max-height:320px}}.VPFeature[data-v-f5e9645b]{display:block;border:1px solid var(--vp-c-bg-soft);border-radius:12px;height:100%;background-color:var(--vp-c-bg-soft);transition:border-color .25s,background-color .25s}.VPFeature.link[data-v-f5e9645b]:hover{border-color:var(--vp-c-brand-1)}.box[data-v-f5e9645b]{display:flex;flex-direction:column;padding:24px;height:100%}.box[data-v-f5e9645b]>.VPImage{margin-bottom:20px}.icon[data-v-f5e9645b]{display:flex;justify-content:center;align-items:center;margin-bottom:20px;border-radius:6px;background-color:var(--vp-c-default-soft);width:48px;height:48px;font-size:24px;transition:background-color .25s}.title[data-v-f5e9645b]{line-height:24px;font-size:16px;font-weight:600}.details[data-v-f5e9645b]{flex-grow:1;padding-top:8px;line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}.link-text[data-v-f5e9645b]{padding-top:8px}.link-text-value[data-v-f5e9645b]{display:flex;align-items:center;font-size:14px;font-weight:500;color:var(--vp-c-brand-1)}.link-text-icon[data-v-f5e9645b]{margin-left:6px}.VPFeatures[data-v-d0a190d7]{position:relative;padding:0 24px}@media (min-width: 640px){.VPFeatures[data-v-d0a190d7]{padding:0 48px}}@media (min-width: 960px){.VPFeatures[data-v-d0a190d7]{padding:0 64px}}.container[data-v-d0a190d7]{margin:0 auto;max-width:1152px}.items[data-v-d0a190d7]{display:flex;flex-wrap:wrap;margin:-8px}.item[data-v-d0a190d7]{padding:8px;width:100%}@media (min-width: 640px){.item.grid-2[data-v-d0a190d7],.item.grid-4[data-v-d0a190d7],.item.grid-6[data-v-d0a190d7]{width:50%}}@media (min-width: 768px){.item.grid-2[data-v-d0a190d7],.item.grid-4[data-v-d0a190d7]{width:50%}.item.grid-3[data-v-d0a190d7],.item.grid-6[data-v-d0a190d7]{width:calc(100% / 3)}}@media (min-width: 960px){.item.grid-4[data-v-d0a190d7]{width:25%}}.container[data-v-7a48a447]{margin:auto;width:100%;max-width:1280px;padding:0 24px}@media (min-width: 640px){.container[data-v-7a48a447]{padding:0 48px}}@media (min-width: 960px){.container[data-v-7a48a447]{width:100%;padding:0 64px}}.vp-doc[data-v-7a48a447] .VPHomeSponsors,.vp-doc[data-v-7a48a447] .VPTeamPage{margin-left:var(--vp-offset, calc(50% - 50vw) );margin-right:var(--vp-offset, calc(50% - 50vw) )}.vp-doc[data-v-7a48a447] .VPHomeSponsors h2{border-top:none;letter-spacing:normal}.vp-doc[data-v-7a48a447] .VPHomeSponsors a,.vp-doc[data-v-7a48a447] .VPTeamPage a{text-decoration:none}.VPHome[data-v-cbb6ec48]{margin-bottom:96px}@media (min-width: 768px){.VPHome[data-v-cbb6ec48]{margin-bottom:128px}}.VPContent[data-v-91765379]{flex-grow:1;flex-shrink:0;margin:var(--vp-layout-top-height, 0px) auto 0;width:100%}.VPContent.is-home[data-v-91765379]{width:100%;max-width:100%}.VPContent.has-sidebar[data-v-91765379]{margin:0}@media (min-width: 960px){.VPContent[data-v-91765379]{padding-top:var(--vp-nav-height)}.VPContent.has-sidebar[data-v-91765379]{margin:var(--vp-layout-top-height, 0px) 0 0;padding-left:var(--vp-sidebar-width)}}@media (min-width: 1440px){.VPContent.has-sidebar[data-v-91765379]{padding-right:calc((100vw - var(--vp-layout-max-width)) / 2);padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.VPFooter[data-v-c970a860]{position:relative;z-index:var(--vp-z-index-footer);border-top:1px solid var(--vp-c-gutter);padding:32px 24px;background-color:var(--vp-c-bg)}.VPFooter.has-sidebar[data-v-c970a860]{display:none}.VPFooter[data-v-c970a860] a{text-decoration-line:underline;text-underline-offset:2px;transition:color .25s}.VPFooter[data-v-c970a860] a:hover{color:var(--vp-c-text-1)}@media (min-width: 768px){.VPFooter[data-v-c970a860]{padding:32px}}.container[data-v-c970a860]{margin:0 auto;max-width:var(--vp-layout-max-width);text-align:center}.message[data-v-c970a860],.copyright[data-v-c970a860]{line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-2)}.VPLocalNavOutlineDropdown[data-v-bc9dc845]{padding:12px 20px 11px}@media (min-width: 960px){.VPLocalNavOutlineDropdown[data-v-bc9dc845]{padding:12px 36px 11px}}.VPLocalNavOutlineDropdown button[data-v-bc9dc845]{display:block;font-size:12px;font-weight:500;line-height:24px;color:var(--vp-c-text-2);transition:color .5s;position:relative}.VPLocalNavOutlineDropdown button[data-v-bc9dc845]:hover{color:var(--vp-c-text-1);transition:color .25s}.VPLocalNavOutlineDropdown button.open[data-v-bc9dc845]{color:var(--vp-c-text-1)}.icon[data-v-bc9dc845]{display:inline-block;vertical-align:middle;margin-left:2px;font-size:14px;transform:rotate(0);transition:transform .25s}@media (min-width: 960px){.VPLocalNavOutlineDropdown button[data-v-bc9dc845]{font-size:14px}.icon[data-v-bc9dc845]{font-size:16px}}.open>.icon[data-v-bc9dc845]{transform:rotate(90deg)}.items[data-v-bc9dc845]{position:absolute;top:40px;right:16px;left:16px;display:grid;gap:1px;border:1px solid var(--vp-c-border);border-radius:8px;background-color:var(--vp-c-gutter);max-height:calc(var(--vp-vh, 100vh) - 86px);overflow:hidden auto;box-shadow:var(--vp-shadow-3)}@media (min-width: 960px){.items[data-v-bc9dc845]{right:auto;left:calc(var(--vp-sidebar-width) + 32px);width:320px}}.header[data-v-bc9dc845]{background-color:var(--vp-c-bg-soft)}.top-link[data-v-bc9dc845]{display:block;padding:0 16px;line-height:48px;font-size:14px;font-weight:500;color:var(--vp-c-brand-1)}.outline[data-v-bc9dc845]{padding:8px 0;background-color:var(--vp-c-bg-soft)}.flyout-enter-active[data-v-bc9dc845]{transition:all .2s ease-out}.flyout-leave-active[data-v-bc9dc845]{transition:all .15s ease-in}.flyout-enter-from[data-v-bc9dc845],.flyout-leave-to[data-v-bc9dc845]{opacity:0;transform:translateY(-16px)}.VPLocalNav[data-v-070ab83d]{position:sticky;top:0;left:0;z-index:var(--vp-z-index-local-nav);border-bottom:1px solid var(--vp-c-gutter);padding-top:var(--vp-layout-top-height, 0px);width:100%;background-color:var(--vp-local-nav-bg-color)}.VPLocalNav.fixed[data-v-070ab83d]{position:fixed}@media (min-width: 960px){.VPLocalNav[data-v-070ab83d]{top:var(--vp-nav-height)}.VPLocalNav.has-sidebar[data-v-070ab83d]{padding-left:var(--vp-sidebar-width)}.VPLocalNav.empty[data-v-070ab83d]{display:none}}@media (min-width: 1280px){.VPLocalNav[data-v-070ab83d]{display:none}}@media (min-width: 1440px){.VPLocalNav.has-sidebar[data-v-070ab83d]{padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.container[data-v-070ab83d]{display:flex;justify-content:space-between;align-items:center}.menu[data-v-070ab83d]{display:flex;align-items:center;padding:12px 24px 11px;line-height:24px;font-size:12px;font-weight:500;color:var(--vp-c-text-2);transition:color .5s}.menu[data-v-070ab83d]:hover{color:var(--vp-c-text-1);transition:color .25s}@media (min-width: 768px){.menu[data-v-070ab83d]{padding:0 32px}}@media (min-width: 960px){.menu[data-v-070ab83d]{display:none}}.menu-icon[data-v-070ab83d]{margin-right:8px;font-size:14px}.VPOutlineDropdown[data-v-070ab83d]{padding:12px 24px 11px}@media (min-width: 768px){.VPOutlineDropdown[data-v-070ab83d]{padding:12px 32px 11px}}.VPSwitch[data-v-4a1c76db]{position:relative;border-radius:11px;display:block;width:40px;height:22px;flex-shrink:0;border:1px solid var(--vp-input-border-color);background-color:var(--vp-input-switch-bg-color);transition:border-color .25s!important}.VPSwitch[data-v-4a1c76db]:hover{border-color:var(--vp-c-brand-1)}.check[data-v-4a1c76db]{position:absolute;top:1px;left:1px;width:18px;height:18px;border-radius:50%;background-color:var(--vp-c-neutral-inverse);box-shadow:var(--vp-shadow-1);transition:transform .25s!important}.icon[data-v-4a1c76db]{position:relative;display:block;width:18px;height:18px;border-radius:50%;overflow:hidden}.icon[data-v-4a1c76db] [class^=vpi-]{position:absolute;top:3px;left:3px;width:12px;height:12px;color:var(--vp-c-text-2)}.dark .icon[data-v-4a1c76db] [class^=vpi-]{color:var(--vp-c-text-1);transition:opacity .25s!important}.sun[data-v-e40a8bb6]{opacity:1}.moon[data-v-e40a8bb6],.dark .sun[data-v-e40a8bb6]{opacity:0}.dark .moon[data-v-e40a8bb6]{opacity:1}.dark .VPSwitchAppearance[data-v-e40a8bb6] .check{transform:translate(18px)}.VPNavBarAppearance[data-v-af096f4a]{display:none}@media (min-width: 1280px){.VPNavBarAppearance[data-v-af096f4a]{display:flex;align-items:center}}.VPMenuGroup+.VPMenuLink[data-v-acbfed09]{margin:12px -12px 0;border-top:1px solid var(--vp-c-divider);padding:12px 12px 0}.link[data-v-acbfed09]{display:block;border-radius:6px;padding:0 12px;line-height:32px;font-size:14px;font-weight:500;color:var(--vp-c-text-1);white-space:nowrap;transition:background-color .25s,color .25s}.link[data-v-acbfed09]:hover{color:var(--vp-c-brand-1);background-color:var(--vp-c-default-soft)}.link.active[data-v-acbfed09]{color:var(--vp-c-brand-1)}.VPMenuGroup[data-v-48c802d0]{margin:12px -12px 0;border-top:1px solid var(--vp-c-divider);padding:12px 12px 0}.VPMenuGroup[data-v-48c802d0]:first-child{margin-top:0;border-top:0;padding-top:0}.VPMenuGroup+.VPMenuGroup[data-v-48c802d0]{margin-top:12px;border-top:1px solid var(--vp-c-divider)}.title[data-v-48c802d0]{padding:0 12px;line-height:32px;font-size:14px;font-weight:600;color:var(--vp-c-text-2);white-space:nowrap;transition:color .25s}.VPMenu[data-v-7dd3104a]{border-radius:12px;padding:12px;min-width:128px;border:1px solid var(--vp-c-divider);background-color:var(--vp-c-bg-elv);box-shadow:var(--vp-shadow-3);transition:background-color .5s;max-height:calc(100vh - var(--vp-nav-height));overflow-y:auto}.VPMenu[data-v-7dd3104a] .group{margin:0 -12px;padding:0 12px 12px}.VPMenu[data-v-7dd3104a] .group+.group{border-top:1px solid var(--vp-c-divider);padding:11px 12px 12px}.VPMenu[data-v-7dd3104a] .group:last-child{padding-bottom:0}.VPMenu[data-v-7dd3104a] .group+.item{border-top:1px solid var(--vp-c-divider);padding:11px 16px 0}.VPMenu[data-v-7dd3104a] .item{padding:0 16px;white-space:nowrap}.VPMenu[data-v-7dd3104a] .label{flex-grow:1;line-height:28px;font-size:12px;font-weight:500;color:var(--vp-c-text-2);transition:color .5s}.VPMenu[data-v-7dd3104a] .action{padding-left:24px}.VPFlyout[data-v-04f5c5e9]{position:relative}.VPFlyout[data-v-04f5c5e9]:hover{color:var(--vp-c-brand-1);transition:color .25s}.VPFlyout:hover .text[data-v-04f5c5e9]{color:var(--vp-c-text-2)}.VPFlyout:hover .icon[data-v-04f5c5e9]{fill:var(--vp-c-text-2)}.VPFlyout.active .text[data-v-04f5c5e9]{color:var(--vp-c-brand-1)}.VPFlyout.active:hover .text[data-v-04f5c5e9]{color:var(--vp-c-brand-2)}.button[aria-expanded=false]+.menu[data-v-04f5c5e9]{opacity:0;visibility:hidden;transform:translateY(0)}.VPFlyout:hover .menu[data-v-04f5c5e9],.button[aria-expanded=true]+.menu[data-v-04f5c5e9]{opacity:1;visibility:visible;transform:translateY(0)}.button[data-v-04f5c5e9]{display:flex;align-items:center;padding:0 12px;height:var(--vp-nav-height);color:var(--vp-c-text-1);transition:color .5s}.text[data-v-04f5c5e9]{display:flex;align-items:center;line-height:var(--vp-nav-height);font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:color .25s}.option-icon[data-v-04f5c5e9]{margin-right:0;font-size:16px}.text-icon[data-v-04f5c5e9]{margin-left:4px;font-size:14px}.icon[data-v-04f5c5e9]{font-size:20px;transition:fill .25s}.menu[data-v-04f5c5e9]{position:absolute;top:calc(var(--vp-nav-height) / 2 + 20px);right:0;opacity:0;visibility:hidden;transition:opacity .25s,visibility .25s,transform .25s}.VPSocialLink[data-v-717b8b75]{display:flex;justify-content:center;align-items:center;width:36px;height:36px;color:var(--vp-c-text-2);transition:color .5s}.VPSocialLink[data-v-717b8b75]:hover{color:var(--vp-c-text-1);transition:color .25s}.VPSocialLink[data-v-717b8b75]>svg,.VPSocialLink[data-v-717b8b75]>[class^=vpi-social-]{width:20px;height:20px;fill:currentColor}.VPSocialLinks[data-v-ee7a9424]{display:flex;justify-content:center}.VPNavBarExtra[data-v-925effce]{display:none;margin-right:-12px}@media (min-width: 768px){.VPNavBarExtra[data-v-925effce]{display:block}}@media (min-width: 1280px){.VPNavBarExtra[data-v-925effce]{display:none}}.trans-title[data-v-925effce]{padding:0 24px 0 12px;line-height:32px;font-size:14px;font-weight:700;color:var(--vp-c-text-1)}.item.appearance[data-v-925effce],.item.social-links[data-v-925effce]{display:flex;align-items:center;padding:0 12px}.item.appearance[data-v-925effce]{min-width:176px}.appearance-action[data-v-925effce]{margin-right:-2px}.social-links-list[data-v-925effce]{margin:-4px -8px}.VPNavBarHamburger[data-v-5dea55bf]{display:flex;justify-content:center;align-items:center;width:48px;height:var(--vp-nav-height)}@media (min-width: 768px){.VPNavBarHamburger[data-v-5dea55bf]{display:none}}.container[data-v-5dea55bf]{position:relative;width:16px;height:14px;overflow:hidden}.VPNavBarHamburger:hover .top[data-v-5dea55bf]{top:0;left:0;transform:translate(4px)}.VPNavBarHamburger:hover .middle[data-v-5dea55bf]{top:6px;left:0;transform:translate(0)}.VPNavBarHamburger:hover .bottom[data-v-5dea55bf]{top:12px;left:0;transform:translate(8px)}.VPNavBarHamburger.active .top[data-v-5dea55bf]{top:6px;transform:translate(0) rotate(225deg)}.VPNavBarHamburger.active .middle[data-v-5dea55bf]{top:6px;transform:translate(16px)}.VPNavBarHamburger.active .bottom[data-v-5dea55bf]{top:6px;transform:translate(0) rotate(135deg)}.VPNavBarHamburger.active:hover .top[data-v-5dea55bf],.VPNavBarHamburger.active:hover .middle[data-v-5dea55bf],.VPNavBarHamburger.active:hover .bottom[data-v-5dea55bf]{background-color:var(--vp-c-text-2);transition:top .25s,background-color .25s,transform .25s}.top[data-v-5dea55bf],.middle[data-v-5dea55bf],.bottom[data-v-5dea55bf]{position:absolute;width:16px;height:2px;background-color:var(--vp-c-text-1);transition:top .25s,background-color .5s,transform .25s}.top[data-v-5dea55bf]{top:0;left:0;transform:translate(0)}.middle[data-v-5dea55bf]{top:6px;left:0;transform:translate(8px)}.bottom[data-v-5dea55bf]{top:12px;left:0;transform:translate(4px)}.VPNavBarMenuLink[data-v-956ec74c]{display:flex;align-items:center;padding:0 12px;line-height:var(--vp-nav-height);font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:color .25s}.VPNavBarMenuLink.active[data-v-956ec74c],.VPNavBarMenuLink[data-v-956ec74c]:hover{color:var(--vp-c-brand-1)}.VPNavBarMenu[data-v-e6d46098]{display:none}@media (min-width: 768px){.VPNavBarMenu[data-v-e6d46098]{display:flex}}/*! @docsearch/css 3.6.3 | MIT License | © Algolia, Inc. and contributors | https://docsearch.algolia.com */:root{--docsearch-primary-color:#5468ff;--docsearch-text-color:#1c1e21;--docsearch-spacing:12px;--docsearch-icon-stroke-width:1.4;--docsearch-highlight-color:var(--docsearch-primary-color);--docsearch-muted-color:#969faf;--docsearch-container-background:rgba(101,108,133,.8);--docsearch-logo-color:#5468ff;--docsearch-modal-width:560px;--docsearch-modal-height:600px;--docsearch-modal-background:#f5f6f7;--docsearch-modal-shadow:inset 1px 1px 0 0 hsla(0,0%,100%,.5),0 3px 8px 0 #555a64;--docsearch-searchbox-height:56px;--docsearch-searchbox-background:#ebedf0;--docsearch-searchbox-focus-background:#fff;--docsearch-searchbox-shadow:inset 0 0 0 2px var(--docsearch-primary-color);--docsearch-hit-height:56px;--docsearch-hit-color:#444950;--docsearch-hit-active-color:#fff;--docsearch-hit-background:#fff;--docsearch-hit-shadow:0 1px 3px 0 #d4d9e1;--docsearch-key-gradient:linear-gradient(-225deg,#d5dbe4,#f8f8f8);--docsearch-key-shadow:inset 0 -2px 0 0 #cdcde6,inset 0 0 1px 1px #fff,0 1px 2px 1px rgba(30,35,90,.4);--docsearch-key-pressed-shadow:inset 0 -2px 0 0 #cdcde6,inset 0 0 1px 1px #fff,0 1px 1px 0 rgba(30,35,90,.4);--docsearch-footer-height:44px;--docsearch-footer-background:#fff;--docsearch-footer-shadow:0 -1px 0 0 #e0e3e8,0 -3px 6px 0 rgba(69,98,155,.12)}html[data-theme=dark]{--docsearch-text-color:#f5f6f7;--docsearch-container-background:rgba(9,10,17,.8);--docsearch-modal-background:#15172a;--docsearch-modal-shadow:inset 1px 1px 0 0 #2c2e40,0 3px 8px 0 #000309;--docsearch-searchbox-background:#090a11;--docsearch-searchbox-focus-background:#000;--docsearch-hit-color:#bec3c9;--docsearch-hit-shadow:none;--docsearch-hit-background:#090a11;--docsearch-key-gradient:linear-gradient(-26.5deg,#565872,#31355b);--docsearch-key-shadow:inset 0 -2px 0 0 #282d55,inset 0 0 1px 1px #51577d,0 2px 2px 0 rgba(3,4,9,.3);--docsearch-key-pressed-shadow:inset 0 -2px 0 0 #282d55,inset 0 0 1px 1px #51577d,0 1px 1px 0 rgba(3,4,9,.30196078431372547);--docsearch-footer-background:#1e2136;--docsearch-footer-shadow:inset 0 1px 0 0 rgba(73,76,106,.5),0 -4px 8px 0 rgba(0,0,0,.2);--docsearch-logo-color:#fff;--docsearch-muted-color:#7f8497}.DocSearch-Button{align-items:center;background:var(--docsearch-searchbox-background);border:0;border-radius:40px;color:var(--docsearch-muted-color);cursor:pointer;display:flex;font-weight:500;height:36px;justify-content:space-between;margin:0 0 0 16px;padding:0 8px;-webkit-user-select:none;user-select:none}.DocSearch-Button:active,.DocSearch-Button:focus,.DocSearch-Button:hover{background:var(--docsearch-searchbox-focus-background);box-shadow:var(--docsearch-searchbox-shadow);color:var(--docsearch-text-color);outline:none}.DocSearch-Button-Container{align-items:center;display:flex}.DocSearch-Search-Icon{stroke-width:1.6}.DocSearch-Button .DocSearch-Search-Icon{color:var(--docsearch-text-color)}.DocSearch-Button-Placeholder{font-size:1rem;padding:0 12px 0 6px}.DocSearch-Button-Keys{display:flex;min-width:calc(40px + .8em)}.DocSearch-Button-Key{align-items:center;background:var(--docsearch-key-gradient);border-radius:3px;box-shadow:var(--docsearch-key-shadow);color:var(--docsearch-muted-color);display:flex;height:18px;justify-content:center;margin-right:.4em;position:relative;padding:0 0 2px;border:0;top:-1px;width:20px}.DocSearch-Button-Key--pressed{transform:translate3d(0,1px,0);box-shadow:var(--docsearch-key-pressed-shadow)}@media (max-width:768px){.DocSearch-Button-Keys,.DocSearch-Button-Placeholder{display:none}}.DocSearch--active{overflow:hidden!important}.DocSearch-Container,.DocSearch-Container *{box-sizing:border-box}.DocSearch-Container{background-color:var(--docsearch-container-background);height:100vh;left:0;position:fixed;top:0;width:100vw;z-index:200}.DocSearch-Container a{text-decoration:none}.DocSearch-Link{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;color:var(--docsearch-highlight-color);cursor:pointer;font:inherit;margin:0;padding:0}.DocSearch-Modal{background:var(--docsearch-modal-background);border-radius:6px;box-shadow:var(--docsearch-modal-shadow);flex-direction:column;margin:60px auto auto;max-width:var(--docsearch-modal-width);position:relative}.DocSearch-SearchBar{display:flex;padding:var(--docsearch-spacing) var(--docsearch-spacing) 0}.DocSearch-Form{align-items:center;background:var(--docsearch-searchbox-focus-background);border-radius:4px;box-shadow:var(--docsearch-searchbox-shadow);display:flex;height:var(--docsearch-searchbox-height);margin:0;padding:0 var(--docsearch-spacing);position:relative;width:100%}.DocSearch-Input{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:transparent;border:0;color:var(--docsearch-text-color);flex:1;font:inherit;font-size:1.2em;height:100%;outline:none;padding:0 0 0 8px;width:80%}.DocSearch-Input::placeholder{color:var(--docsearch-muted-color);opacity:1}.DocSearch-Input::-webkit-search-cancel-button,.DocSearch-Input::-webkit-search-decoration,.DocSearch-Input::-webkit-search-results-button,.DocSearch-Input::-webkit-search-results-decoration{display:none}.DocSearch-LoadingIndicator,.DocSearch-MagnifierLabel,.DocSearch-Reset{margin:0;padding:0}.DocSearch-MagnifierLabel,.DocSearch-Reset{align-items:center;color:var(--docsearch-highlight-color);display:flex;justify-content:center}.DocSearch-Container--Stalled .DocSearch-MagnifierLabel,.DocSearch-LoadingIndicator{display:none}.DocSearch-Container--Stalled .DocSearch-LoadingIndicator{align-items:center;color:var(--docsearch-highlight-color);display:flex;justify-content:center}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Reset{animation:none;-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:50%;color:var(--docsearch-icon-color);cursor:pointer;right:0;stroke-width:var(--docsearch-icon-stroke-width)}}.DocSearch-Reset{animation:fade-in .1s ease-in forwards;-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:50%;color:var(--docsearch-icon-color);cursor:pointer;padding:2px;right:0;stroke-width:var(--docsearch-icon-stroke-width)}.DocSearch-Reset[hidden]{display:none}.DocSearch-Reset:hover{color:var(--docsearch-highlight-color)}.DocSearch-LoadingIndicator svg,.DocSearch-MagnifierLabel svg{height:24px;width:24px}.DocSearch-Cancel{display:none}.DocSearch-Dropdown{max-height:calc(var(--docsearch-modal-height) - var(--docsearch-searchbox-height) - var(--docsearch-spacing) - var(--docsearch-footer-height));min-height:var(--docsearch-spacing);overflow-y:auto;overflow-y:overlay;padding:0 var(--docsearch-spacing);scrollbar-color:var(--docsearch-muted-color) var(--docsearch-modal-background);scrollbar-width:thin}.DocSearch-Dropdown::-webkit-scrollbar{width:12px}.DocSearch-Dropdown::-webkit-scrollbar-track{background:transparent}.DocSearch-Dropdown::-webkit-scrollbar-thumb{background-color:var(--docsearch-muted-color);border:3px solid var(--docsearch-modal-background);border-radius:20px}.DocSearch-Dropdown ul{list-style:none;margin:0;padding:0}.DocSearch-Label{font-size:.75em;line-height:1.6em}.DocSearch-Help,.DocSearch-Label{color:var(--docsearch-muted-color)}.DocSearch-Help{font-size:.9em;margin:0;-webkit-user-select:none;user-select:none}.DocSearch-Title{font-size:1.2em}.DocSearch-Logo a{display:flex}.DocSearch-Logo svg{color:var(--docsearch-logo-color);margin-left:8px}.DocSearch-Hits:last-of-type{margin-bottom:24px}.DocSearch-Hits mark{background:none;color:var(--docsearch-highlight-color)}.DocSearch-HitsFooter{color:var(--docsearch-muted-color);display:flex;font-size:.85em;justify-content:center;margin-bottom:var(--docsearch-spacing);padding:var(--docsearch-spacing)}.DocSearch-HitsFooter a{border-bottom:1px solid;color:inherit}.DocSearch-Hit{border-radius:4px;display:flex;padding-bottom:4px;position:relative}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit--deleting{transition:none}}.DocSearch-Hit--deleting{opacity:0;transition:all .25s linear}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit--favoriting{transition:none}}.DocSearch-Hit--favoriting{transform:scale(0);transform-origin:top center;transition:all .25s linear;transition-delay:.25s}.DocSearch-Hit a{background:var(--docsearch-hit-background);border-radius:4px;box-shadow:var(--docsearch-hit-shadow);display:block;padding-left:var(--docsearch-spacing);width:100%}.DocSearch-Hit-source{background:var(--docsearch-modal-background);color:var(--docsearch-highlight-color);font-size:.85em;font-weight:600;line-height:32px;margin:0 -4px;padding:8px 4px 0;position:sticky;top:0;z-index:10}.DocSearch-Hit-Tree{color:var(--docsearch-muted-color);height:var(--docsearch-hit-height);opacity:.5;stroke-width:var(--docsearch-icon-stroke-width);width:24px}.DocSearch-Hit[aria-selected=true] a{background-color:var(--docsearch-highlight-color)}.DocSearch-Hit[aria-selected=true] mark{text-decoration:underline}.DocSearch-Hit-Container{align-items:center;color:var(--docsearch-hit-color);display:flex;flex-direction:row;height:var(--docsearch-hit-height);padding:0 var(--docsearch-spacing) 0 0}.DocSearch-Hit-icon{height:20px;width:20px}.DocSearch-Hit-action,.DocSearch-Hit-icon{color:var(--docsearch-muted-color);stroke-width:var(--docsearch-icon-stroke-width)}.DocSearch-Hit-action{align-items:center;display:flex;height:22px;width:22px}.DocSearch-Hit-action svg{display:block;height:18px;width:18px}.DocSearch-Hit-action+.DocSearch-Hit-action{margin-left:6px}.DocSearch-Hit-action-button{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:50%;color:inherit;cursor:pointer;padding:2px}svg.DocSearch-Hit-Select-Icon{display:none}.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-Select-Icon{display:block}.DocSearch-Hit-action-button:focus,.DocSearch-Hit-action-button:hover{background:#0003;transition:background-color .1s ease-in}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit-action-button:focus,.DocSearch-Hit-action-button:hover{transition:none}}.DocSearch-Hit-action-button:focus path,.DocSearch-Hit-action-button:hover path{fill:#fff}.DocSearch-Hit-content-wrapper{display:flex;flex:1 1 auto;flex-direction:column;font-weight:500;justify-content:center;line-height:1.2em;margin:0 8px;overflow-x:hidden;position:relative;text-overflow:ellipsis;white-space:nowrap;width:80%}.DocSearch-Hit-title{font-size:.9em}.DocSearch-Hit-path{color:var(--docsearch-muted-color);font-size:.75em}.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-action,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-icon,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-path,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-text,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-title,.DocSearch-Hit[aria-selected=true] .DocSearch-Hit-Tree,.DocSearch-Hit[aria-selected=true] mark{color:var(--docsearch-hit-active-color)!important}@media screen and (prefers-reduced-motion:reduce){.DocSearch-Hit-action-button:focus,.DocSearch-Hit-action-button:hover{background:#0003;transition:none}}.DocSearch-ErrorScreen,.DocSearch-NoResults,.DocSearch-StartScreen{font-size:.9em;margin:0 auto;padding:36px 0;text-align:center;width:80%}.DocSearch-Screen-Icon{color:var(--docsearch-muted-color);padding-bottom:12px}.DocSearch-NoResults-Prefill-List{display:inline-block;padding-bottom:24px;text-align:left}.DocSearch-NoResults-Prefill-List ul{display:inline-block;padding:8px 0 0}.DocSearch-NoResults-Prefill-List li{list-style-position:inside;list-style-type:"» "}.DocSearch-Prefill{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;border-radius:1em;color:var(--docsearch-highlight-color);cursor:pointer;display:inline-block;font-size:1em;font-weight:700;padding:0}.DocSearch-Prefill:focus,.DocSearch-Prefill:hover{outline:none;text-decoration:underline}.DocSearch-Footer{align-items:center;background:var(--docsearch-footer-background);border-radius:0 0 8px 8px;box-shadow:var(--docsearch-footer-shadow);display:flex;flex-direction:row-reverse;flex-shrink:0;height:var(--docsearch-footer-height);justify-content:space-between;padding:0 var(--docsearch-spacing);position:relative;-webkit-user-select:none;user-select:none;width:100%;z-index:300}.DocSearch-Commands{color:var(--docsearch-muted-color);display:flex;list-style:none;margin:0;padding:0}.DocSearch-Commands li{align-items:center;display:flex}.DocSearch-Commands li:not(:last-of-type){margin-right:.8em}.DocSearch-Commands-Key{align-items:center;background:var(--docsearch-key-gradient);border-radius:2px;box-shadow:var(--docsearch-key-shadow);display:flex;height:18px;justify-content:center;margin-right:.4em;padding:0 0 1px;color:var(--docsearch-muted-color);border:0;width:20px}.DocSearch-VisuallyHiddenForAccessibility{clip:rect(0 0 0 0);clip-path:inset(50%);height:1px;overflow:hidden;position:absolute;white-space:nowrap;width:1px}@media (max-width:768px){:root{--docsearch-spacing:10px;--docsearch-footer-height:40px}.DocSearch-Dropdown{height:100%}.DocSearch-Container{height:100vh;height:-webkit-fill-available;height:calc(var(--docsearch-vh, 1vh)*100);position:absolute}.DocSearch-Footer{border-radius:0;bottom:0;position:absolute}.DocSearch-Hit-content-wrapper{display:flex;position:relative;width:80%}.DocSearch-Modal{border-radius:0;box-shadow:none;height:100vh;height:-webkit-fill-available;height:calc(var(--docsearch-vh, 1vh)*100);margin:0;max-width:100%;width:100%}.DocSearch-Dropdown{max-height:calc(var(--docsearch-vh, 1vh)*100 - var(--docsearch-searchbox-height) - var(--docsearch-spacing) - var(--docsearch-footer-height))}.DocSearch-Cancel{-webkit-appearance:none;-moz-appearance:none;appearance:none;background:none;border:0;color:var(--docsearch-highlight-color);cursor:pointer;display:inline-block;flex:none;font:inherit;font-size:1em;font-weight:500;margin-left:var(--docsearch-spacing);outline:none;overflow:hidden;padding:0;-webkit-user-select:none;user-select:none;white-space:nowrap}.DocSearch-Commands,.DocSearch-Hit-Tree{display:none}}@keyframes fade-in{0%{opacity:0}to{opacity:1}}[class*=DocSearch]{--docsearch-primary-color: var(--vp-c-brand-1);--docsearch-highlight-color: var(--docsearch-primary-color);--docsearch-text-color: var(--vp-c-text-1);--docsearch-muted-color: var(--vp-c-text-2);--docsearch-searchbox-shadow: none;--docsearch-searchbox-background: transparent;--docsearch-searchbox-focus-background: transparent;--docsearch-key-gradient: transparent;--docsearch-key-shadow: none;--docsearch-modal-background: var(--vp-c-bg-soft);--docsearch-footer-background: var(--vp-c-bg)}.dark [class*=DocSearch]{--docsearch-modal-shadow: none;--docsearch-footer-shadow: none;--docsearch-logo-color: var(--vp-c-text-2);--docsearch-hit-background: var(--vp-c-default-soft);--docsearch-hit-color: var(--vp-c-text-2);--docsearch-hit-shadow: none}.DocSearch-Button{display:flex;justify-content:center;align-items:center;margin:0;padding:0;width:48px;height:55px;background:transparent;transition:border-color .25s}.DocSearch-Button:hover{background:transparent}.DocSearch-Button:focus{outline:1px dotted;outline:5px auto -webkit-focus-ring-color}.DocSearch-Button-Key--pressed{transform:none;box-shadow:none}.DocSearch-Button:focus:not(:focus-visible){outline:none!important}@media (min-width: 768px){.DocSearch-Button{justify-content:flex-start;border:1px solid transparent;border-radius:8px;padding:0 10px 0 12px;width:100%;height:40px;background-color:var(--vp-c-bg-alt)}.DocSearch-Button:hover{border-color:var(--vp-c-brand-1);background:var(--vp-c-bg-alt)}}.DocSearch-Button .DocSearch-Button-Container{display:flex;align-items:center}.DocSearch-Button .DocSearch-Search-Icon{position:relative;width:16px;height:16px;color:var(--vp-c-text-1);fill:currentColor;transition:color .5s}.DocSearch-Button:hover .DocSearch-Search-Icon{color:var(--vp-c-text-1)}@media (min-width: 768px){.DocSearch-Button .DocSearch-Search-Icon{top:1px;margin-right:8px;width:14px;height:14px;color:var(--vp-c-text-2)}}.DocSearch-Button .DocSearch-Button-Placeholder{display:none;margin-top:2px;padding:0 16px 0 0;font-size:13px;font-weight:500;color:var(--vp-c-text-2);transition:color .5s}.DocSearch-Button:hover .DocSearch-Button-Placeholder{color:var(--vp-c-text-1)}@media (min-width: 768px){.DocSearch-Button .DocSearch-Button-Placeholder{display:inline-block}}.DocSearch-Button .DocSearch-Button-Keys{direction:ltr;display:none;min-width:auto}@media (min-width: 768px){.DocSearch-Button .DocSearch-Button-Keys{display:flex;align-items:center}}.DocSearch-Button .DocSearch-Button-Key{display:block;margin:2px 0 0;border:1px solid var(--vp-c-divider);border-right:none;border-radius:4px 0 0 4px;padding-left:6px;min-width:0;width:auto;height:22px;line-height:22px;font-family:var(--vp-font-family-base);font-size:12px;font-weight:500;transition:color .5s,border-color .5s}.DocSearch-Button .DocSearch-Button-Key+.DocSearch-Button-Key{border-right:1px solid var(--vp-c-divider);border-left:none;border-radius:0 4px 4px 0;padding-left:2px;padding-right:6px}.DocSearch-Button .DocSearch-Button-Key:first-child{font-size:0!important}.DocSearch-Button .DocSearch-Button-Key:first-child:after{content:"Ctrl";font-size:12px;letter-spacing:normal;color:var(--docsearch-muted-color)}.mac .DocSearch-Button .DocSearch-Button-Key:first-child:after{content:"⌘"}.DocSearch-Button .DocSearch-Button-Key:first-child>*{display:none}.DocSearch-Search-Icon{--icon: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' stroke-width='1.6' viewBox='0 0 20 20'%3E%3Cpath fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' d='m14.386 14.386 4.088 4.088-4.088-4.088A7.533 7.533 0 1 1 3.733 3.733a7.533 7.533 0 0 1 10.653 10.653z'/%3E%3C/svg%3E")}.VPNavBarSearch{display:flex;align-items:center}@media (min-width: 768px){.VPNavBarSearch{flex-grow:1;padding-left:24px}}@media (min-width: 960px){.VPNavBarSearch{padding-left:32px}}.dark .DocSearch-Footer{border-top:1px solid var(--vp-c-divider)}.DocSearch-Form{border:1px solid var(--vp-c-brand-1);background-color:var(--vp-c-white)}.dark .DocSearch-Form{background-color:var(--vp-c-default-soft)}.DocSearch-Screen-Icon>svg{margin:auto}.VPNavBarSocialLinks[data-v-164c457f]{display:none}@media (min-width: 1280px){.VPNavBarSocialLinks[data-v-164c457f]{display:flex;align-items:center}}.title[data-v-0f4f798b]{display:flex;align-items:center;border-bottom:1px solid transparent;width:100%;height:var(--vp-nav-height);font-size:16px;font-weight:600;color:var(--vp-c-text-1);transition:opacity .25s}@media (min-width: 960px){.title[data-v-0f4f798b]{flex-shrink:0}.VPNavBarTitle.has-sidebar .title[data-v-0f4f798b]{border-bottom-color:var(--vp-c-divider)}}[data-v-0f4f798b] .logo{margin-right:8px;height:var(--vp-nav-logo-height)}.VPNavBarTranslations[data-v-c80d9ad0]{display:none}@media (min-width: 1280px){.VPNavBarTranslations[data-v-c80d9ad0]{display:flex;align-items:center}}.title[data-v-c80d9ad0]{padding:0 24px 0 12px;line-height:32px;font-size:14px;font-weight:700;color:var(--vp-c-text-1)}.VPNavBar[data-v-822684d1]{position:relative;height:var(--vp-nav-height);pointer-events:none;white-space:nowrap;transition:background-color .25s}.VPNavBar.screen-open[data-v-822684d1]{transition:none;background-color:var(--vp-nav-bg-color);border-bottom:1px solid var(--vp-c-divider)}.VPNavBar[data-v-822684d1]:not(.home){background-color:var(--vp-nav-bg-color)}@media (min-width: 960px){.VPNavBar[data-v-822684d1]:not(.home){background-color:transparent}.VPNavBar[data-v-822684d1]:not(.has-sidebar):not(.home.top){background-color:var(--vp-nav-bg-color)}}.wrapper[data-v-822684d1]{padding:0 8px 0 24px}@media (min-width: 768px){.wrapper[data-v-822684d1]{padding:0 32px}}@media (min-width: 960px){.VPNavBar.has-sidebar .wrapper[data-v-822684d1]{padding:0}}.container[data-v-822684d1]{display:flex;justify-content:space-between;margin:0 auto;max-width:calc(var(--vp-layout-max-width) - 64px);height:var(--vp-nav-height);pointer-events:none}.container>.title[data-v-822684d1],.container>.content[data-v-822684d1]{pointer-events:none}.container[data-v-822684d1] *{pointer-events:auto}@media (min-width: 960px){.VPNavBar.has-sidebar .container[data-v-822684d1]{max-width:100%}}.title[data-v-822684d1]{flex-shrink:0;height:calc(var(--vp-nav-height) - 1px);transition:background-color .5s}@media (min-width: 960px){.VPNavBar.has-sidebar .title[data-v-822684d1]{position:absolute;top:0;left:0;z-index:2;padding:0 32px;width:var(--vp-sidebar-width);height:var(--vp-nav-height);background-color:transparent}}@media (min-width: 1440px){.VPNavBar.has-sidebar .title[data-v-822684d1]{padding-left:max(32px,calc((100% - (var(--vp-layout-max-width) - 64px)) / 2));width:calc((100% - (var(--vp-layout-max-width) - 64px)) / 2 + var(--vp-sidebar-width) - 32px)}}.content[data-v-822684d1]{flex-grow:1}@media (min-width: 960px){.VPNavBar.has-sidebar .content[data-v-822684d1]{position:relative;z-index:1;padding-right:32px;padding-left:var(--vp-sidebar-width)}}@media (min-width: 1440px){.VPNavBar.has-sidebar .content[data-v-822684d1]{padding-right:calc((100vw - var(--vp-layout-max-width)) / 2 + 32px);padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.content-body[data-v-822684d1]{display:flex;justify-content:flex-end;align-items:center;height:var(--vp-nav-height);transition:background-color .5s}@media (min-width: 960px){.VPNavBar:not(.home.top) .content-body[data-v-822684d1]{position:relative;background-color:var(--vp-nav-bg-color)}.VPNavBar:not(.has-sidebar):not(.home.top) .content-body[data-v-822684d1]{background-color:transparent}}@media (max-width: 767px){.content-body[data-v-822684d1]{column-gap:.5rem}}.menu+.translations[data-v-822684d1]:before,.menu+.appearance[data-v-822684d1]:before,.menu+.social-links[data-v-822684d1]:before,.translations+.appearance[data-v-822684d1]:before,.appearance+.social-links[data-v-822684d1]:before{margin-right:8px;margin-left:8px;width:1px;height:24px;background-color:var(--vp-c-divider);content:""}.menu+.appearance[data-v-822684d1]:before,.translations+.appearance[data-v-822684d1]:before{margin-right:16px}.appearance+.social-links[data-v-822684d1]:before{margin-left:16px}.social-links[data-v-822684d1]{margin-right:-8px}.divider[data-v-822684d1]{width:100%;height:1px}@media (min-width: 960px){.VPNavBar.has-sidebar .divider[data-v-822684d1]{padding-left:var(--vp-sidebar-width)}}@media (min-width: 1440px){.VPNavBar.has-sidebar .divider[data-v-822684d1]{padding-left:calc((100vw - var(--vp-layout-max-width)) / 2 + var(--vp-sidebar-width))}}.divider-line[data-v-822684d1]{width:100%;height:1px;transition:background-color .5s}.VPNavBar:not(.home) .divider-line[data-v-822684d1]{background-color:var(--vp-c-gutter)}@media (min-width: 960px){.VPNavBar:not(.home.top) .divider-line[data-v-822684d1]{background-color:var(--vp-c-gutter)}.VPNavBar:not(.has-sidebar):not(.home.top) .divider[data-v-822684d1]{background-color:var(--vp-c-gutter)}}.VPNavScreenAppearance[data-v-ffb44008]{display:flex;justify-content:space-between;align-items:center;border-radius:8px;padding:12px 14px 12px 16px;background-color:var(--vp-c-bg-soft)}.text[data-v-ffb44008]{line-height:24px;font-size:12px;font-weight:500;color:var(--vp-c-text-2)}.VPNavScreenMenuLink[data-v-735512b8]{display:block;border-bottom:1px solid var(--vp-c-divider);padding:12px 0 11px;line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:border-color .25s,color .25s}.VPNavScreenMenuLink[data-v-735512b8]:hover{color:var(--vp-c-brand-1)}.VPNavScreenMenuGroupLink[data-v-372ae7c0]{display:block;margin-left:12px;line-height:32px;font-size:14px;font-weight:400;color:var(--vp-c-text-1);transition:color .25s}.VPNavScreenMenuGroupLink[data-v-372ae7c0]:hover{color:var(--vp-c-brand-1)}.VPNavScreenMenuGroupSection[data-v-4b8941ac]{display:block}.title[data-v-4b8941ac]{line-height:32px;font-size:13px;font-weight:700;color:var(--vp-c-text-2);transition:color .25s}.VPNavScreenMenuGroup[data-v-875057a5]{border-bottom:1px solid var(--vp-c-divider);height:48px;overflow:hidden;transition:border-color .5s}.VPNavScreenMenuGroup .items[data-v-875057a5]{visibility:hidden}.VPNavScreenMenuGroup.open .items[data-v-875057a5]{visibility:visible}.VPNavScreenMenuGroup.open[data-v-875057a5]{padding-bottom:10px;height:auto}.VPNavScreenMenuGroup.open .button[data-v-875057a5]{padding-bottom:6px;color:var(--vp-c-brand-1)}.VPNavScreenMenuGroup.open .button-icon[data-v-875057a5]{transform:rotate(45deg)}.button[data-v-875057a5]{display:flex;justify-content:space-between;align-items:center;padding:12px 4px 11px 0;width:100%;line-height:24px;font-size:14px;font-weight:500;color:var(--vp-c-text-1);transition:color .25s}.button[data-v-875057a5]:hover{color:var(--vp-c-brand-1)}.button-icon[data-v-875057a5]{transition:transform .25s}.group[data-v-875057a5]:first-child{padding-top:0}.group+.group[data-v-875057a5],.group+.item[data-v-875057a5]{padding-top:4px}.VPNavScreenTranslations[data-v-362991c2]{height:24px;overflow:hidden}.VPNavScreenTranslations.open[data-v-362991c2]{height:auto}.title[data-v-362991c2]{display:flex;align-items:center;font-size:14px;font-weight:500;color:var(--vp-c-text-1)}.icon[data-v-362991c2]{font-size:16px}.icon.lang[data-v-362991c2]{margin-right:8px}.icon.chevron[data-v-362991c2]{margin-left:4px}.list[data-v-362991c2]{padding:4px 0 0 24px}.link[data-v-362991c2]{line-height:32px;font-size:13px;color:var(--vp-c-text-1)}.VPNavScreen[data-v-833aabba]{position:fixed;top:calc(var(--vp-nav-height) + var(--vp-layout-top-height, 0px));right:0;bottom:0;left:0;padding:0 32px;width:100%;background-color:var(--vp-nav-screen-bg-color);overflow-y:auto;transition:background-color .25s;pointer-events:auto}.VPNavScreen.fade-enter-active[data-v-833aabba],.VPNavScreen.fade-leave-active[data-v-833aabba]{transition:opacity .25s}.VPNavScreen.fade-enter-active .container[data-v-833aabba],.VPNavScreen.fade-leave-active .container[data-v-833aabba]{transition:transform .25s ease}.VPNavScreen.fade-enter-from[data-v-833aabba],.VPNavScreen.fade-leave-to[data-v-833aabba]{opacity:0}.VPNavScreen.fade-enter-from .container[data-v-833aabba],.VPNavScreen.fade-leave-to .container[data-v-833aabba]{transform:translateY(-8px)}@media (min-width: 768px){.VPNavScreen[data-v-833aabba]{display:none}}.container[data-v-833aabba]{margin:0 auto;padding:24px 0 96px;max-width:288px}.menu+.translations[data-v-833aabba],.menu+.appearance[data-v-833aabba],.translations+.appearance[data-v-833aabba]{margin-top:24px}.menu+.social-links[data-v-833aabba]{margin-top:16px}.appearance+.social-links[data-v-833aabba]{margin-top:16px}.VPNav[data-v-f1e365da]{position:relative;top:var(--vp-layout-top-height, 0px);left:0;z-index:var(--vp-z-index-nav);width:100%;pointer-events:none;transition:background-color .5s}@media (min-width: 960px){.VPNav[data-v-f1e365da]{position:fixed}}.VPSidebarItem.level-0[data-v-196b2e5f]{padding-bottom:24px}.VPSidebarItem.collapsed.level-0[data-v-196b2e5f]{padding-bottom:10px}.item[data-v-196b2e5f]{position:relative;display:flex;width:100%}.VPSidebarItem.collapsible>.item[data-v-196b2e5f]{cursor:pointer}.indicator[data-v-196b2e5f]{position:absolute;top:6px;bottom:6px;left:-17px;width:2px;border-radius:2px;transition:background-color .25s}.VPSidebarItem.level-2.is-active>.item>.indicator[data-v-196b2e5f],.VPSidebarItem.level-3.is-active>.item>.indicator[data-v-196b2e5f],.VPSidebarItem.level-4.is-active>.item>.indicator[data-v-196b2e5f],.VPSidebarItem.level-5.is-active>.item>.indicator[data-v-196b2e5f]{background-color:var(--vp-c-brand-1)}.link[data-v-196b2e5f]{display:flex;align-items:center;flex-grow:1}.text[data-v-196b2e5f]{flex-grow:1;padding:4px 0;line-height:24px;font-size:14px;transition:color .25s}.VPSidebarItem.level-0 .text[data-v-196b2e5f]{font-weight:700;color:var(--vp-c-text-1)}.VPSidebarItem.level-1 .text[data-v-196b2e5f],.VPSidebarItem.level-2 .text[data-v-196b2e5f],.VPSidebarItem.level-3 .text[data-v-196b2e5f],.VPSidebarItem.level-4 .text[data-v-196b2e5f],.VPSidebarItem.level-5 .text[data-v-196b2e5f]{font-weight:500;color:var(--vp-c-text-2)}.VPSidebarItem.level-0.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-1.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-2.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-3.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-4.is-link>.item>.link:hover .text[data-v-196b2e5f],.VPSidebarItem.level-5.is-link>.item>.link:hover .text[data-v-196b2e5f]{color:var(--vp-c-brand-1)}.VPSidebarItem.level-0.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-1.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-2.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-3.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-4.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-5.has-active>.item>.text[data-v-196b2e5f],.VPSidebarItem.level-0.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-1.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-2.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-3.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-4.has-active>.item>.link>.text[data-v-196b2e5f],.VPSidebarItem.level-5.has-active>.item>.link>.text[data-v-196b2e5f]{color:var(--vp-c-text-1)}.VPSidebarItem.level-0.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-1.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-2.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-3.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-4.is-active>.item .link>.text[data-v-196b2e5f],.VPSidebarItem.level-5.is-active>.item .link>.text[data-v-196b2e5f]{color:var(--vp-c-brand-1)}.caret[data-v-196b2e5f]{display:flex;justify-content:center;align-items:center;margin-right:-7px;width:32px;height:32px;color:var(--vp-c-text-3);cursor:pointer;transition:color .25s;flex-shrink:0}.item:hover .caret[data-v-196b2e5f]{color:var(--vp-c-text-2)}.item:hover .caret[data-v-196b2e5f]:hover{color:var(--vp-c-text-1)}.caret-icon[data-v-196b2e5f]{font-size:18px;transform:rotate(90deg);transition:transform .25s}.VPSidebarItem.collapsed .caret-icon[data-v-196b2e5f]{transform:rotate(0)}.VPSidebarItem.level-1 .items[data-v-196b2e5f],.VPSidebarItem.level-2 .items[data-v-196b2e5f],.VPSidebarItem.level-3 .items[data-v-196b2e5f],.VPSidebarItem.level-4 .items[data-v-196b2e5f],.VPSidebarItem.level-5 .items[data-v-196b2e5f]{border-left:1px solid var(--vp-c-divider);padding-left:16px}.VPSidebarItem.collapsed .items[data-v-196b2e5f]{display:none}.no-transition[data-v-9e426adc] .caret-icon{transition:none}.group+.group[data-v-9e426adc]{border-top:1px solid var(--vp-c-divider);padding-top:10px}@media (min-width: 960px){.group[data-v-9e426adc]{padding-top:10px;width:calc(var(--vp-sidebar-width) - 64px)}}.VPSidebar[data-v-18756405]{position:fixed;top:var(--vp-layout-top-height, 0px);bottom:0;left:0;z-index:var(--vp-z-index-sidebar);padding:32px 32px 96px;width:calc(100vw - 64px);max-width:320px;background-color:var(--vp-sidebar-bg-color);opacity:0;box-shadow:var(--vp-c-shadow-3);overflow-x:hidden;overflow-y:auto;transform:translate(-100%);transition:opacity .5s,transform .25s ease;overscroll-behavior:contain}.VPSidebar.open[data-v-18756405]{opacity:1;visibility:visible;transform:translate(0);transition:opacity .25s,transform .5s cubic-bezier(.19,1,.22,1)}.dark .VPSidebar[data-v-18756405]{box-shadow:var(--vp-shadow-1)}@media (min-width: 960px){.VPSidebar[data-v-18756405]{padding-top:var(--vp-nav-height);width:var(--vp-sidebar-width);max-width:100%;background-color:var(--vp-sidebar-bg-color);opacity:1;visibility:visible;box-shadow:none;transform:translate(0)}}@media (min-width: 1440px){.VPSidebar[data-v-18756405]{padding-left:max(32px,calc((100% - (var(--vp-layout-max-width) - 64px)) / 2));width:calc((100% - (var(--vp-layout-max-width) - 64px)) / 2 + var(--vp-sidebar-width) - 32px)}}@media (min-width: 960px){.curtain[data-v-18756405]{position:sticky;top:-64px;left:0;z-index:1;margin-top:calc(var(--vp-nav-height) * -1);margin-right:-32px;margin-left:-32px;height:var(--vp-nav-height);background-color:var(--vp-sidebar-bg-color)}}.nav[data-v-18756405]{outline:0}.VPSkipLink[data-v-c3508ec8]{top:8px;left:8px;padding:8px 16px;z-index:999;border-radius:8px;font-size:12px;font-weight:700;text-decoration:none;color:var(--vp-c-brand-1);box-shadow:var(--vp-shadow-3);background-color:var(--vp-c-bg)}.VPSkipLink[data-v-c3508ec8]:focus{height:auto;width:auto;clip:auto;clip-path:none}@media (min-width: 1280px){.VPSkipLink[data-v-c3508ec8]{top:14px;left:16px}}.Layout[data-v-a9a9e638]{display:flex;flex-direction:column;min-height:100vh}.VPHomeSponsors[data-v-db81191c]{border-top:1px solid var(--vp-c-gutter);padding-top:88px!important}.VPHomeSponsors[data-v-db81191c]{margin:96px 0}@media (min-width: 768px){.VPHomeSponsors[data-v-db81191c]{margin:128px 0}}.VPHomeSponsors[data-v-db81191c]{padding:0 24px}@media (min-width: 768px){.VPHomeSponsors[data-v-db81191c]{padding:0 48px}}@media (min-width: 960px){.VPHomeSponsors[data-v-db81191c]{padding:0 64px}}.container[data-v-db81191c]{margin:0 auto;max-width:1152px}.love[data-v-db81191c]{margin:0 auto;width:fit-content;font-size:28px;color:var(--vp-c-text-3)}.icon[data-v-db81191c]{display:inline-block}.message[data-v-db81191c]{margin:0 auto;padding-top:10px;max-width:320px;text-align:center;line-height:24px;font-size:16px;font-weight:500;color:var(--vp-c-text-2)}.sponsors[data-v-db81191c]{padding-top:32px}.action[data-v-db81191c]{padding-top:40px;text-align:center}.VPTeamPage[data-v-c2f8e101]{margin:96px 0}@media (min-width: 768px){.VPTeamPage[data-v-c2f8e101]{margin:128px 0}}.VPHome .VPTeamPageTitle[data-v-c2f8e101-s]{border-top:1px solid var(--vp-c-gutter);padding-top:88px!important}.VPTeamPageSection+.VPTeamPageSection[data-v-c2f8e101-s],.VPTeamMembers+.VPTeamPageSection[data-v-c2f8e101-s]{margin-top:64px}.VPTeamMembers+.VPTeamMembers[data-v-c2f8e101-s]{margin-top:24px}@media (min-width: 768px){.VPTeamPageTitle+.VPTeamPageSection[data-v-c2f8e101-s]{margin-top:16px}.VPTeamPageSection+.VPTeamPageSection[data-v-c2f8e101-s],.VPTeamMembers+.VPTeamPageSection[data-v-c2f8e101-s]{margin-top:96px}}.VPTeamMembers[data-v-c2f8e101-s]{padding:0 24px}@media (min-width: 768px){.VPTeamMembers[data-v-c2f8e101-s]{padding:0 48px}}@media (min-width: 960px){.VPTeamMembers[data-v-c2f8e101-s]{padding:0 64px}}.VPTeamPageTitle[data-v-e277e15c]{padding:48px 32px;text-align:center}@media (min-width: 768px){.VPTeamPageTitle[data-v-e277e15c]{padding:64px 48px 48px}}@media (min-width: 960px){.VPTeamPageTitle[data-v-e277e15c]{padding:80px 64px 48px}}.title[data-v-e277e15c]{letter-spacing:0;line-height:44px;font-size:36px;font-weight:500}@media (min-width: 768px){.title[data-v-e277e15c]{letter-spacing:-.5px;line-height:56px;font-size:48px}}.lead[data-v-e277e15c]{margin:0 auto;max-width:512px;padding-top:12px;line-height:24px;font-size:16px;font-weight:500;color:var(--vp-c-text-2)}@media (min-width: 768px){.lead[data-v-e277e15c]{max-width:592px;letter-spacing:.15px;line-height:28px;font-size:20px}}.VPTeamPageSection[data-v-d43bc49d]{padding:0 32px}@media (min-width: 768px){.VPTeamPageSection[data-v-d43bc49d]{padding:0 48px}}@media (min-width: 960px){.VPTeamPageSection[data-v-d43bc49d]{padding:0 64px}}.title[data-v-d43bc49d]{position:relative;margin:0 auto;max-width:1152px;text-align:center;color:var(--vp-c-text-2)}.title-line[data-v-d43bc49d]{position:absolute;top:16px;left:0;width:100%;height:1px;background-color:var(--vp-c-divider)}.title-text[data-v-d43bc49d]{position:relative;display:inline-block;padding:0 24px;letter-spacing:0;line-height:32px;font-size:20px;font-weight:500;background-color:var(--vp-c-bg)}.lead[data-v-d43bc49d]{margin:0 auto;max-width:480px;padding-top:12px;text-align:center;line-height:24px;font-size:16px;font-weight:500;color:var(--vp-c-text-2)}.members[data-v-d43bc49d]{padding-top:40px}.VPTeamMembersItem[data-v-f9987cb6]{display:flex;flex-direction:column;gap:2px;border-radius:12px;width:100%;height:100%;overflow:hidden}.VPTeamMembersItem.small .profile[data-v-f9987cb6]{padding:32px}.VPTeamMembersItem.small .data[data-v-f9987cb6]{padding-top:20px}.VPTeamMembersItem.small .avatar[data-v-f9987cb6]{width:64px;height:64px}.VPTeamMembersItem.small .name[data-v-f9987cb6]{line-height:24px;font-size:16px}.VPTeamMembersItem.small .affiliation[data-v-f9987cb6]{padding-top:4px;line-height:20px;font-size:14px}.VPTeamMembersItem.small .desc[data-v-f9987cb6]{padding-top:12px;line-height:20px;font-size:14px}.VPTeamMembersItem.small .links[data-v-f9987cb6]{margin:0 -16px -20px;padding:10px 0 0}.VPTeamMembersItem.medium .profile[data-v-f9987cb6]{padding:48px 32px}.VPTeamMembersItem.medium .data[data-v-f9987cb6]{padding-top:24px;text-align:center}.VPTeamMembersItem.medium .avatar[data-v-f9987cb6]{width:96px;height:96px}.VPTeamMembersItem.medium .name[data-v-f9987cb6]{letter-spacing:.15px;line-height:28px;font-size:20px}.VPTeamMembersItem.medium .affiliation[data-v-f9987cb6]{padding-top:4px;font-size:16px}.VPTeamMembersItem.medium .desc[data-v-f9987cb6]{padding-top:16px;max-width:288px;font-size:16px}.VPTeamMembersItem.medium .links[data-v-f9987cb6]{margin:0 -16px -12px;padding:16px 12px 0}.profile[data-v-f9987cb6]{flex-grow:1;background-color:var(--vp-c-bg-soft)}.data[data-v-f9987cb6]{text-align:center}.avatar[data-v-f9987cb6]{position:relative;flex-shrink:0;margin:0 auto;border-radius:50%;box-shadow:var(--vp-shadow-3)}.avatar-img[data-v-f9987cb6]{position:absolute;top:0;right:0;bottom:0;left:0;border-radius:50%;object-fit:cover}.name[data-v-f9987cb6]{margin:0;font-weight:600}.affiliation[data-v-f9987cb6]{margin:0;font-weight:500;color:var(--vp-c-text-2)}.org.link[data-v-f9987cb6]{color:var(--vp-c-text-2);transition:color .25s}.org.link[data-v-f9987cb6]:hover{color:var(--vp-c-brand-1)}.desc[data-v-f9987cb6]{margin:0 auto}.desc[data-v-f9987cb6] a{font-weight:500;color:var(--vp-c-brand-1);text-decoration-style:dotted;transition:color .25s}.links[data-v-f9987cb6]{display:flex;justify-content:center;height:56px}.sp-link[data-v-f9987cb6]{display:flex;justify-content:center;align-items:center;text-align:center;padding:16px;font-size:14px;font-weight:500;color:var(--vp-c-sponsor);background-color:var(--vp-c-bg-soft);transition:color .25s,background-color .25s}.sp .sp-link.link[data-v-f9987cb6]:hover,.sp .sp-link.link[data-v-f9987cb6]:focus{outline:none;color:var(--vp-c-white);background-color:var(--vp-c-sponsor)}.sp-icon[data-v-f9987cb6]{margin-right:8px;font-size:16px}.VPTeamMembers.small .container[data-v-fba19bad]{grid-template-columns:repeat(auto-fit,minmax(224px,1fr))}.VPTeamMembers.small.count-1 .container[data-v-fba19bad]{max-width:276px}.VPTeamMembers.small.count-2 .container[data-v-fba19bad]{max-width:576px}.VPTeamMembers.small.count-3 .container[data-v-fba19bad]{max-width:876px}.VPTeamMembers.medium .container[data-v-fba19bad]{grid-template-columns:repeat(auto-fit,minmax(256px,1fr))}@media (min-width: 375px){.VPTeamMembers.medium .container[data-v-fba19bad]{grid-template-columns:repeat(auto-fit,minmax(288px,1fr))}}.VPTeamMembers.medium.count-1 .container[data-v-fba19bad]{max-width:368px}.VPTeamMembers.medium.count-2 .container[data-v-fba19bad]{max-width:760px}.container[data-v-fba19bad]{display:grid;gap:24px;margin:0 auto;max-width:1152px}.enjoyer{margin-top:.5rem;margin-bottom:0rem;border-radius:14px;padding-top:.2rem;padding-bottom:.2rem;position:relative;font-size:.9rem;font-weight:700;line-height:1.1rem;display:flex;align-items:center;justify-content:center;width:100%;gap:1rem;background-color:var(--vp-c-bg-alt);border:2px solid var(--vp-c-bg-alt);transition:border-color .5s}.enjoyer:hover{border:2px solid var(--vp-c-brand-light)}.enjoyer img{transition:transform .5s;transform:scale(1.25)}.enjoyer:hover img{transform:scale(1.75)}.enjoyer .heading{background-image:linear-gradient(120deg,#b047ff 16%,var(--vp-c-brand-lighter),var(--vp-c-brand-lighter));background-clip:text;-webkit-background-clip:text;-webkit-text-fill-color:transparent}.enjoyer .extra-info{color:var(--vp-c-text-1);opacity:0;font-size:.7rem;padding-left:.1rem;transition:opacity .5s}.enjoyer:hover .extra-info{opacity:.9}.VPVersionPicker[data-v-d483b3a6] button .text{color:var(--vp-c-text-1)!important}.VPVersionPicker[data-v-d483b3a6]:hover button .text{color:var(--vp-c-text-2)!important}:root{--vp-plugin-tabs-tab-text-color: var(--vp-c-text-2);--vp-plugin-tabs-tab-active-text-color: var(--vp-c-text-1);--vp-plugin-tabs-tab-hover-text-color: var(--vp-c-text-1);--vp-plugin-tabs-tab-bg: var(--vp-c-bg-soft);--vp-plugin-tabs-tab-divider: var(--vp-c-divider);--vp-plugin-tabs-tab-active-bar-color: var(--vp-c-brand-1)}.plugin-tabs{margin:16px 0;background-color:var(--vp-plugin-tabs-tab-bg);border-radius:8px}.plugin-tabs--tab-list{position:relative;padding:0 12px;overflow-x:auto;overflow-y:hidden}.plugin-tabs--tab-list:after{content:"";position:absolute;bottom:0;left:0;right:0;height:2px;background-color:var(--vp-plugin-tabs-tab-divider)}.plugin-tabs--tab{position:relative;padding:0 12px;line-height:48px;border-bottom:2px solid transparent;color:var(--vp-plugin-tabs-tab-text-color);font-size:14px;font-weight:500;white-space:nowrap;transition:color .25s}.plugin-tabs--tab[aria-selected=true]{color:var(--vp-plugin-tabs-tab-active-text-color)}.plugin-tabs--tab:hover{color:var(--vp-plugin-tabs-tab-hover-text-color)}.plugin-tabs--tab:after{content:"";position:absolute;bottom:-2px;left:8px;right:8px;height:2px;background-color:transparent;transition:background-color .25s;z-index:1}.plugin-tabs--tab[aria-selected=true]:after{background-color:var(--vp-plugin-tabs-tab-active-bar-color)}.plugin-tabs--content[data-v-9b0d03d2]{padding:16px}.plugin-tabs--content[data-v-9b0d03d2]>:first-child:first-child{margin-top:0}.plugin-tabs--content[data-v-9b0d03d2]>:last-child:last-child{margin-bottom:0}.plugin-tabs--content[data-v-9b0d03d2]>div[class*=language-]{border-radius:8px;margin:16px 0}:root:not(.dark) .plugin-tabs--content[data-v-9b0d03d2] div[class*=language-]{background-color:var(--vp-c-bg)}.VPHero .clip{white-space:pre;max-width:500px}@font-face{font-family:JuliaMono-Regular;src:url(https://cdn.jsdelivr.net/gh/cormullion/juliamono/webfonts/JuliaMono-Regular.woff2)}:root{--vp-font-family-base: "Barlow", "Inter var experimental", "Inter var", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, Cantarell, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif;--vp-font-family-mono: JuliaMono-Regular, monospace;font-feature-settings:"calt" 0}.mono pre,.mono code{font-family:JuliaMono-Light}:root{--julia-blue: #4063d8;--julia-purple: #9558b2;--julia-red: #cb3c33;--julia-green: #389826;--vp-c-brand: #389826;--vp-c-brand-light: #3dd027;--vp-c-brand-lighter: #9499ff;--vp-c-brand-lightest: #bcc0ff;--vp-c-brand-dark: #535bf2;--vp-c-brand-darker: #454ce1;--vp-c-brand-dimm: #212425}:root{--vp-button-brand-border: var(--vp-c-brand-light);--vp-button-brand-text: var(--vp-c-white);--vp-button-brand-bg: var(--vp-c-brand);--vp-button-brand-hover-border: var(--vp-c-brand-light);--vp-button-brand-hover-text: var(--vp-c-white);--vp-button-brand-hover-bg: var(--vp-c-brand-light);--vp-button-brand-active-border: var(--vp-c-brand-light);--vp-button-brand-active-text: var(--vp-c-white);--vp-button-brand-active-bg: var(--vp-button-brand-bg)}:root{--vp-home-hero-name-color: transparent;--vp-home-hero-name-background: -webkit-linear-gradient( 120deg, #9558b2 30%, #cb3c33 );--vp-home-hero-image-background-image: linear-gradient( -45deg, #9558b2 30%, #389826 30%, #cb3c33 );--vp-home-hero-image-filter: blur(40px)}@media (min-width: 640px){:root{--vp-home-hero-image-filter: blur(56px)}}@media (min-width: 960px){:root{--vp-home-hero-image-filter: blur(72px)}}:root.dark{--vp-custom-block-tip-border: var(--vp-c-brand);--vp-custom-block-tip-text: var(--vp-c-brand-lightest);--vp-custom-block-tip-bg: var(--vp-c-brand-dimm);--vp-c-black: hsl(220 20% 9%);--vp-c-black-pure: hsl(220, 24%, 4%);--vp-c-black-soft: hsl(220 16% 13%);--vp-c-black-mute: hsl(220 14% 17%);--vp-c-gray: hsl(220 8% 56%);--vp-c-gray-dark-1: hsl(220 10% 39%);--vp-c-gray-dark-2: hsl(220 12% 28%);--vp-c-gray-dark-3: hsl(220 12% 23%);--vp-c-gray-dark-4: hsl(220 14% 17%);--vp-c-gray-dark-5: hsl(220 16% 13%);--vp-custom-block-info-bg: hsl(220 14% 17%)}.DocSearch{--docsearch-primary-color: var(--vp-c-brand) !important}mjx-container>svg{display:block;margin:auto}mjx-container{padding:.5rem 0}mjx-container{display:inline;margin:auto 2px -2px}mjx-container>svg{margin:auto;display:inline}:root{--vp-c-brand-1: #cb3c33;--vp-c-brand-2: #cb3c33;--vp-c-brand-3: #cb3c33;--vp-c-sponsor: #ca2971;--vitest-c-sponsor-hover: #c13071}.dark{--vp-c-brand-1: #91dd33;--vp-c-brand-2: #91dd33;--vp-c-brand-3: #91dd33;--vp-c-sponsor: #91dd33;--vitest-c-sponsor-hover: #e51370}:root:not(.dark) .dark-only{display:none}:root:is(.dark) .light-only{display:none}@media (min-width: 1440px){.VPSidebar{padding-left:20px!important;width:250px!important}.VPNavBar .title{padding-left:15px!important;width:230px!important}.VPContent.has-sidebar{padding-left:250px!important;padding-right:5vw!important}.VPNavBar .curtain{width:100%!important}.VPDoc{padding:32px 0 0!important}.VPNavBar.has-sidebar .content{padding-left:250px!important;padding-right:20px!important}.VPNavBar .divider{padding-left:250px!important}}@media (min-width: 960px){.VPDoc{padding:32px 32px 0 10!important}.VPContent.has-sidebar{padding-left:255px!important}}.VPNavBar{padding-right:0!important}.VPDoc.has-aside .content-container{max-width:100%!important}.aside{max-width:200px!important;padding-left:0!important}.VPDocOutlineItem li{text-overflow:ellipsis;overflow:hidden;white-space:nowrap;max-width:200px}.VPNavBar .title{text-overflow:ellipsis;overflow:hidden;white-space:nowrap}.jldocstring.custom-block{border:1px solid var(--vp-c-gray-2);color:var(--vp-c-text-1)}.jldocstring.custom-block summary{font-weight:700;cursor:pointer;-webkit-user-select:none;user-select:none;margin:0 0 8px}.img-box[data-v-06a0366f]{box-sizing:content-box;border-radius:14px;margin:20px;height:350px;width:250px;overflow:hidden;display:inline-block;color:#fff;position:relative;background-color:transparent;border:2px solid var(--vp-c-bg-alt)}.img-box h2[data-v-06a0366f]{border-top:0}.img-box img[data-v-06a0366f]{height:100%;width:100%;object-fit:cover;opacity:.3;transition:transform .3s ease,opacity .3s ease}.caption[data-v-06a0366f]{position:absolute;bottom:30px;color:var(--vp-c-text-1);left:10px;opacity:1;transition:transform .3s ease,opacity .3s ease}.subcaption[data-v-06a0366f]{position:absolute;bottom:5px;color:var(--vp-c-text-1);left:10px;opacity:0;transition:transform .3s ease,opacity .3s ease}.transparent-box1[data-v-06a0366f]{height:250px;width:250px;background-color:transparent;position:absolute;top:0;left:0;transition:background-color .3s ease}.transparent-box2[data-v-06a0366f]{height:100px;width:250px;background-color:transparent;position:absolute;top:250px;left:0;transition:background-color .3s ease}.img-box:hover img[data-v-06a0366f]{transform:scale(1.1)}.img-box:hover .transparent-box1[data-v-06a0366f],.img-box:hover .transparent-box2[data-v-06a0366f]{background-color:var(--vp-c-bg-alt)}.img-box:hover .caption[data-v-06a0366f],.img-box:hover .subcaption[data-v-06a0366f]{transform:translateY(-20px);opacity:1}.img-box[data-v-06a0366f]:hover{border:2px solid var(--vp-c-brand-light);cursor:pointer}.caption>p[data-v-06a0366f]:nth-child(2){font-size:.8em}.subcaption>p[data-v-06a0366f]:nth-child(2){font-size:.8em}.opacity-low[data-v-06a0366f]{opacity:.85}.heading[data-v-578d61bc]{text-align:center;font-size:2em;letter-spacing:1px;padding:40px;color:#fff}.gallery-image[data-v-578d61bc]{padding:20px;display:flex;flex-wrap:wrap;justify-content:center}.gallery-image[data-v-578d61bc] img{height:350px;width:250px;transform:scale(1);transition:transform .4s ease}.VPLocalSearchBox[data-v-42e65fb9]{position:fixed;z-index:100;top:0;right:0;bottom:0;left:0;display:flex}.backdrop[data-v-42e65fb9]{position:absolute;top:0;right:0;bottom:0;left:0;background:var(--vp-backdrop-bg-color);transition:opacity .5s}.shell[data-v-42e65fb9]{position:relative;padding:12px;margin:64px auto;display:flex;flex-direction:column;gap:16px;background:var(--vp-local-search-bg);width:min(100vw - 60px,900px);height:min-content;max-height:min(100vh - 128px,900px);border-radius:6px}@media (max-width: 767px){.shell[data-v-42e65fb9]{margin:0;width:100vw;height:100vh;max-height:none;border-radius:0}}.search-bar[data-v-42e65fb9]{border:1px solid var(--vp-c-divider);border-radius:4px;display:flex;align-items:center;padding:0 12px;cursor:text}@media (max-width: 767px){.search-bar[data-v-42e65fb9]{padding:0 8px}}.search-bar[data-v-42e65fb9]:focus-within{border-color:var(--vp-c-brand-1)}.local-search-icon[data-v-42e65fb9]{display:block;font-size:18px}.navigate-icon[data-v-42e65fb9]{display:block;font-size:14px}.search-icon[data-v-42e65fb9]{margin:8px}@media (max-width: 767px){.search-icon[data-v-42e65fb9]{display:none}}.search-input[data-v-42e65fb9]{padding:6px 12px;font-size:inherit;width:100%}@media (max-width: 767px){.search-input[data-v-42e65fb9]{padding:6px 4px}}.search-actions[data-v-42e65fb9]{display:flex;gap:4px}@media (any-pointer: coarse){.search-actions[data-v-42e65fb9]{gap:8px}}@media (min-width: 769px){.search-actions.before[data-v-42e65fb9]{display:none}}.search-actions button[data-v-42e65fb9]{padding:8px}.search-actions button[data-v-42e65fb9]:not([disabled]):hover,.toggle-layout-button.detailed-list[data-v-42e65fb9]{color:var(--vp-c-brand-1)}.search-actions button.clear-button[data-v-42e65fb9]:disabled{opacity:.37}.search-keyboard-shortcuts[data-v-42e65fb9]{font-size:.8rem;opacity:75%;display:flex;flex-wrap:wrap;gap:16px;line-height:14px}.search-keyboard-shortcuts span[data-v-42e65fb9]{display:flex;align-items:center;gap:4px}@media (max-width: 767px){.search-keyboard-shortcuts[data-v-42e65fb9]{display:none}}.search-keyboard-shortcuts kbd[data-v-42e65fb9]{background:#8080801a;border-radius:4px;padding:3px 6px;min-width:24px;display:inline-block;text-align:center;vertical-align:middle;border:1px solid rgba(128,128,128,.15);box-shadow:0 2px 2px #0000001a}.results[data-v-42e65fb9]{display:flex;flex-direction:column;gap:6px;overflow-x:hidden;overflow-y:auto;overscroll-behavior:contain}.result[data-v-42e65fb9]{display:flex;align-items:center;gap:8px;border-radius:4px;transition:none;line-height:1rem;border:solid 2px var(--vp-local-search-result-border);outline:none}.result>div[data-v-42e65fb9]{margin:12px;width:100%;overflow:hidden}@media (max-width: 767px){.result>div[data-v-42e65fb9]{margin:8px}}.titles[data-v-42e65fb9]{display:flex;flex-wrap:wrap;gap:4px;position:relative;z-index:1001;padding:2px 0}.title[data-v-42e65fb9]{display:flex;align-items:center;gap:4px}.title.main[data-v-42e65fb9]{font-weight:500}.title-icon[data-v-42e65fb9]{opacity:.5;font-weight:500;color:var(--vp-c-brand-1)}.title svg[data-v-42e65fb9]{opacity:.5}.result.selected[data-v-42e65fb9]{--vp-local-search-result-bg: var(--vp-local-search-result-selected-bg);border-color:var(--vp-local-search-result-selected-border)}.excerpt-wrapper[data-v-42e65fb9]{position:relative}.excerpt[data-v-42e65fb9]{opacity:50%;pointer-events:none;max-height:140px;overflow:hidden;position:relative;margin-top:4px}.result.selected .excerpt[data-v-42e65fb9]{opacity:1}.excerpt[data-v-42e65fb9] *{font-size:.8rem!important;line-height:130%!important}.titles[data-v-42e65fb9] mark,.excerpt[data-v-42e65fb9] mark{background-color:var(--vp-local-search-highlight-bg);color:var(--vp-local-search-highlight-text);border-radius:2px;padding:0 2px}.excerpt[data-v-42e65fb9] .vp-code-group .tabs{display:none}.excerpt[data-v-42e65fb9] .vp-code-group div[class*=language-]{border-radius:8px!important}.excerpt-gradient-bottom[data-v-42e65fb9]{position:absolute;bottom:-1px;left:0;width:100%;height:8px;background:linear-gradient(transparent,var(--vp-local-search-result-bg));z-index:1000}.excerpt-gradient-top[data-v-42e65fb9]{position:absolute;top:-1px;left:0;width:100%;height:8px;background:linear-gradient(var(--vp-local-search-result-bg),transparent);z-index:1000}.result.selected .titles[data-v-42e65fb9],.result.selected .title-icon[data-v-42e65fb9]{color:var(--vp-c-brand-1)!important}.no-results[data-v-42e65fb9]{font-size:.9rem;text-align:center;padding:12px}svg[data-v-42e65fb9]{flex:none}
diff --git a/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.BfLFXCWx.js b/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.BfLFXCWx.js
deleted file mode 100644
index 0af67503bc..0000000000
--- a/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.BfLFXCWx.js
+++ /dev/null
@@ -1,286 +0,0 @@
-import{_ as t,c as i,a2 as n,j as A,a as h,o as a}from"./chunks/framework.CCjWn1F9.js";const j=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),e={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},m={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},B={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},K={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},D={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function Z(X,s,R,x,w,O){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
-      OptimizationOptimJL, Printf, Random, SciMLSensitivity
-using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=h("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=h(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=h(", ")),A("mjx-container",r,[(a(),i("svg",d,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=h(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
-    M = m₁ + m₂
-    r₁ = m₂ / M .* path
-    r₂ = -m₁ / M .* path
-    return r₁, r₂
-end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=h("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",C,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
-    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
-
-    if size(soln, 1) == 2
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-
-        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
-        p, M, e = model_params
-    else
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-        p = soln[3, :]
-        e = soln[4, :]
-    end
-
-    r = p ./ (1 .+ e .* cos.(χ))
-    x = r .* cos.(ϕ)
-    y = r .* sin.(ϕ)
-
-    orbit = vcat(x', y')
-    return orbit
-end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
-    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
-    b = (v[3:end] .- v[1:(end - 2)]) / 2
-    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
-    return [a; b; c] / dt
-end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
-    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
-    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
-    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
-    return [a; b; c] / (dt^2)
-end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
-    x = orbit[1, :]
-    y = orbit[2, :]
-
-    Ixx = x .^ 2
-    Iyy = y .^ 2
-    Ixy = x .* y
-    trace = Ixx .+ Iyy
-
-    if component[1] == 1 && component[2] == 1
-        tmp = Ixx .- trace ./ 3
-    elseif component[1] == 2 && component[2] == 2
-        tmp = Iyy .- trace ./ 3
-    else
-        tmp = Ixy
-    end
-
-    return mass .* tmp
-end
-
-function h_22_quadrupole_components(dt, orbit, component, mass=1)
-    mtensor = orbit2tensor(orbit, component, mass)
-    mtensor_ddot = d2_dt2(mtensor, dt)
-    return 2 * mtensor_ddot
-end
-
-function h_22_quadrupole(dt, orbit, mass=1)
-    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
-    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
-    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
-    return h11, h12, h22
-end
-
-function h_22_strain_one_body(dt::T, orbit) where {T}
-    h11, h12, h22 = h_22_quadrupole(dt, orbit)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
-    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
-    h11 = h11_1 + h11_2
-    h12 = h12_1 + h12_2
-    h22 = h22_1 + h22_2
-    return h11, h12, h22
-end
-
-function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
-    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
-
-    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
-
-    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
-    @assert mass_ratio≤1 "mass_ratio must be <= 1"
-    @assert mass_ratio≥0 "mass_ratio must be non-negative"
-
-    orbit = soln2orbit(soln, model_params)
-    if mass_ratio > 0
-        m₂ = inv(T(1) + mass_ratio)
-        m₁ = mass_ratio * m₂
-
-        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
-        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
-    else
-        waveform = h_22_strain_one_body(dt, orbit)
-    end
-    return waveform
-end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",g,[(a(),i("svg",o,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=h("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=h(", ")),A("mjx-container",u,[(a(),i("svg",F,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=h(", and ")),A("mjx-container",c,[(a(),i("svg",b,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=h(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
-    χ, ϕ = u
-
-    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
-    denom = sqrt((p - 2)^2 - 4 * e^2)
-
-    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
-    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
-
-    return [χ̇, ϕ̇]
-end
-
-mass_ratio = 0.0         # test particle
-u0 = Float64[π, 0.0]     # initial conditions
-datasize = 250
-tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
-tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
-dt_data = tsteps[2] - tsteps[1]
-dt = 100.0
-const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
-soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
-waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
-
-    axislegend(ax, [[l, s]], ["Waveform Data"])
-
-    fig
-end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
-    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
-ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[3.189577f-5; -6.391393f-5; -3.3405635f-5; 4.4343187f-6; 0.000115499526; -0.0001018731; 0.00023788413; 0.00013548152; 6.836329f-5; -7.0168346f-5; -1.1637023f-5; 0.000107990956; 0.00010711853; -8.205676f-5; -3.5767007f-5; 0.0001381862; -4.684011f-5; 0.000164203; -3.8421153f-5; 5.4418593f-5; 9.1661295f-6; 2.394569f-5; -2.2449845f-5; 4.2435263f-6; 0.00014469436; -7.2968054f-5; 8.1639504f-5; -0.00010950737; 0.000111710426; -4.2097818f-5; -0.00023246874; -6.5241285f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-4.072868f-5 -5.0945444f-5 -0.0001259869 -2.931564f-5 0.000166747 7.638867f-5 -5.6004457f-5 -0.00014100932 -2.1586391f-5 -1.2109375f-5 -9.704629f-5 0.00017422912 0.00027027391 8.124638f-5 2.312072f-5 -1.3258273f-5 -6.2884916f-5 -5.023768f-5 -6.0434613f-5 -5.904619f-5 5.3575146f-5 8.3898245f-5 6.905054f-6 -9.50151f-7 -3.3204353f-5 -0.000321201 0.0001560693 6.604745f-5 -4.809239f-5 0.00012601794 -2.430164f-5 7.2888855f-5; 5.081071f-5 0.00017587443 -0.0001901438 -0.00016309925 -1.4872606f-5 0.00010889661 -9.051517f-5 -9.4335606f-5 0.00014208801 -4.8255f-5 -0.00015105841 3.318696f-5 -0.00010077599 0.00012599488 1.5010563f-5 8.972074f-5 -1.3046683f-5 -0.00015159532 0.00011351562 0.0001516642 0.00014350485 -0.00010061943 -0.00016779696 0.00016463436 6.636019f-5 0.00010201224 0.00011553895 -0.00013985121 9.045295f-5 3.6739464f-5 4.8656864f-5 0.000121753; 0.00022025035 -5.518999f-6 -9.600911f-5 0.00013558794 -5.8217807f-5 -7.169174f-5 8.9917674f-5 -0.00010023812 0.00017261699 -3.40887f-5 -0.00019824217 0.0001871318 -8.1081045f-5 3.311501f-5 -0.00010184848 -5.5567038f-5 -6.2686864f-5 -0.00011883248 0.0001333885 -2.6586393f-5 4.5853876f-6 -8.5089574f-5 -2.3479493f-5 9.173001f-6 7.11846f-5 0.000114722425 -1.7309303f-5 -8.865217f-5 5.1577266f-5 -9.788526f-5 5.233557f-5 -7.615518f-5; 0.00020963635 -1.8638202f-5 0.00011834405 -6.729295f-5 -0.00015167797 -7.0826565f-5 3.4370183f-5 -0.00016984323 9.748238f-5 6.600134f-5 -6.2498555f-5 -0.0001225902 -0.000192871 -0.00012371842 -0.00024446193 -3.0057638f-5 0.00016294271 9.313347f-5 -0.00013989632 -1.6167292f-5 -9.897745f-6 -3.6781636f-5 3.8724313f-5 0.000102805214 -6.287945f-5 3.452113f-5 3.6691803f-5 1.649534f-5 -1.0298279f-5 -3.4320224f-6 6.5838605f-5 -4.1367566f-6; 3.1436957f-5 0.00010077432 -0.0001992841 -8.427237f-6 2.0229614f-5 -1.8158896f-7 -4.3141594f-5 -6.0499427f-5 0.0001370642 -8.7448054f-5 -8.16965f-5 1.0385415f-5 -1.1151988f-5 2.0163849f-5 -8.430852f-5 0.00014713164 -1.2928104f-5 4.095135f-5 -0.00015691279 -0.00013806451 5.4008604f-5 -8.881955f-5 7.982801f-5 -6.0442053f-5 0.00010303316 1.1154679f-5 -6.417079f-5 -7.0635576f-7 1.0161936f-5 -3.7932576f-5 -9.273091f-5 6.587933f-5; 0.00015628956 -0.000100306606 0.0001417675 -1.8419723f-5 -0.00010336675 -3.745216f-6 1.3287905f-5 0.000110201225 -0.00012162571 0.00010194623 -0.00021031174 -0.00013408852 -0.00016633335 -0.00011588641 -0.00014500602 0.00023872862 -0.00019082129 0.00020116469 4.6417983f-5 4.0384627f-8 -4.1172363f-5 5.739644f-5 0.0002493159 1.0108878f-6 -6.098719f-6 1.7906605f-5 0.000208925 0.00012901663 5.7656558f-5 5.2600393f-5 -1.7499613f-5 -0.00023694857; 7.6452096f-5 -0.00013919755 5.0454488f-5 -0.00010899347 8.444795f-5 9.3733055f-5 4.9273887f-5 9.575293f-5 -0.00017926394 5.2648138f-5 -0.00015499145 6.0000508f-5 -0.00012347617 -1.0212578f-5 -3.7480448f-5 -2.8889195f-5 -2.5839225f-5 -0.00014959453 -4.838798f-5 -0.00010264639 -2.2373182f-5 0.000187367 -1.4591332f-5 7.587859f-5 6.715484f-5 -8.455441f-6 0.00010300705 9.579943f-5 6.2988795f-5 0.00013727028 5.8433718f-5 -7.581316f-5; 9.867598f-6 0.00013001259 -0.00015904017 0.00010726528 -7.944118f-5 -0.00010838457 -3.0477504f-5 -5.3470885f-5 -0.00017066523 5.8966172f-5 -4.431984f-5 0.00011518578 9.390664f-5 -3.4953282f-5 -2.5257921f-5 3.6932663f-5 5.5787146f-5 8.749019f-5 7.14501f-6 -4.0959952f-5 -6.223889f-5 -8.734941f-5 0.00012043471 -0.00015020274 5.2771924f-5 0.0002360519 -6.228447f-5 6.4803426f-5 0.00010642961 -5.2180483f-5 3.147583f-5 0.00012207228; 0.0001617969 -3.8236438f-5 7.3801225f-6 -6.36821f-5 5.370217f-5 -5.6450346f-5 2.389117f-5 1.3119574f-5 3.8583716f-5 3.039198f-5 0.00014524818 -9.020171f-5 -8.770791f-5 -7.468266f-5 0.00012475614 -1.5612637f-5 2.97265f-5 -5.011583f-5 -2.4989173f-5 6.159547f-5 3.4962607f-6 1.9588915f-6 -9.422292f-5 -9.7407676f-5 -9.5720214f-5 -1.895673f-7 -6.3542095f-5 -8.419821f-5 6.416182f-5 -0.00019523122 6.9585396f-5 -1.4270458f-5; -2.4227842f-5 8.622542f-5 0.00013249219 -0.00016822782 -9.379966f-5 -5.4564578f-5 -7.1605165f-5 -9.777959f-5 -1.5141534f-5 -2.2505346f-5 1.7262204f-5 0.00020515101 -0.00015698072 0.00014193162 -8.208326f-5 0.0001984161 2.1720332f-5 -7.3809155f-5 8.412848f-5 7.0492674f-6 0.00014390076 1.1922104f-5 -0.00014427994 -4.40556f-6 -9.2878225f-5 1.861306f-5 8.723424f-5 0.00021786611 3.6548522f-6 -4.011293f-5 2.7519702f-5 -1.2173145f-5; 2.1144622f-5 0.00014967582 -0.00022895339 0.00024803935 6.106838f-5 7.726797f-5 -9.1495414f-5 -0.00013571442 -1.2743402f-5 3.7127043f-5 5.741774f-6 5.7482393f-6 1.4281251f-5 1.6610835f-5 3.506546f-6 0.00017465782 -3.9547882f-5 -7.454391f-5 -7.768746f-5 -4.087552f-5 -9.2923765f-6 -7.646117f-5 -5.933055f-5 5.02955f-5 3.072864f-5 -5.4951495f-5 -9.214813f-5 4.7591155f-5 -1.3358998f-6 0.00012437423 -4.2005086f-5 -6.518599f-5; 5.2166448f-5 2.3049663f-6 -2.9499244f-5 6.810257f-5 4.4267512f-5 0.00014037451 -9.013224f-6 3.2751344f-5 -6.298081f-6 2.6862037f-5 -0.00011300167 -0.00017712849 0.00022498281 9.065984f-5 -1.6968424f-5 9.56821f-6 9.605599f-6 1.79725f-5 3.0613257f-5 -4.2208558f-5 4.7292133f-6 -4.070843f-6 -0.00016515724 4.4079825f-6 1.49199095f-5 -0.00012453321 1.52478315f-5 -4.9945927f-5 -0.00010899424 5.9470847f-5 4.765956f-5 -0.00019273692; -0.00014856191 4.997596f-5 0.00014711094 7.5645374f-5 -5.5391956f-5 -0.000105570536 8.8837296f-5 7.1964474f-5 -4.419326f-5 -7.311366f-5 4.8815167f-5 -0.0001266497 7.6390796f-5 -0.000102230515 0.00017258363 -2.6130465f-5 -4.5941284f-5 0.00018645074 0.00014535875 -1.6320302f-5 -0.00025578658 -2.491066f-5 -1.5766662f-5 4.3251843f-5 0.00015311797 -0.00011235082 -1.9891374f-6 -0.00019587998 -0.00021126727 8.496006f-5 7.781786f-5 6.961107f-5; 9.386831f-5 0.00015165155 5.2710024f-5 -9.78637f-5 6.203806f-5 0.00011791612 -8.990891f-5 -3.199821f-6 8.466118f-6 -7.408915f-5 -1.6125816f-5 -0.00015418157 0.00010376295 1.5213385f-5 0.000108165565 -5.207923f-5 1.2379622f-5 0.00018003177 -2.9956736f-5 -5.492753f-5 -8.3133375f-5 8.4283674f-5 -5.026297f-5 2.7448626f-5 6.964033f-5 -1.6386199f-5 -9.4f-6 -0.00011314431 0.00013795459 5.831961f-5 -2.2579597f-5 3.1594696f-5; 4.5288183f-5 0.000151896 0.00012069575 -0.0001195369 -7.6831355f-5 -2.757172f-5 -1.6826662f-5 -0.00013169323 6.2199084f-5 -7.3993283f-6 7.438026f-5 -0.00017041463 -7.112954f-5 -2.846346f-5 6.357693f-5 -6.5696972f-6 -4.935456f-5 -9.412396f-6 -5.3623813f-7 -1.5585098f-5 -2.8759412f-5 2.6243073f-5 0.00010650766 -8.971806f-6 -8.316659f-5 -8.4486215f-5 -0.0001286409 1.9484713f-5 -6.270333f-5 6.095372f-5 -0.00020690112 -2.4946847f-5; -9.633735f-5 2.9961959f-5 -8.814054f-5 0.00029132818 -8.873218f-5 -8.915458f-5 2.4145293f-5 -0.00016413987 -0.00013065261 -0.00016233047 -5.391122f-5 -1.16300425f-5 7.1261215f-6 -4.5942532f-5 0.00019776101 -9.504735f-5 -0.00012708372 -9.55583f-5 -4.6812784f-5 -0.00014748046 -4.5817545f-5 -0.00012706364 -3.134673f-5 -5.041207f-5 -9.92932f-5 0.00024251099 7.7417964f-5 -9.606522f-5 -3.9172697f-5 -7.114121f-5 0.00012291323 2.004699f-5; 2.6071615f-5 0.00023533189 4.6388315f-5 -4.4312666f-5 4.156144f-5 1.0031484f-5 0.00021525362 -5.070823f-6 8.614582f-5 -8.926076f-5 -7.542077f-5 7.453099f-6 -3.379711f-5 -0.0001935182 6.443749f-6 -6.77989f-5 0.00024368793 -0.00022429347 -8.256922f-5 0.00023213631 -6.02441f-5 3.6378533f-5 -5.4333955f-6 0.00017070145 -0.00035845037 0.000118600394 -0.00014000213 -2.6946222f-5 4.9569262f-5 7.846937f-5 -0.00016146098 0.00016950133; -7.044077f-5 7.863811f-5 -3.5487312f-5 -3.3633118f-5 0.00018373223 -0.00015216439 -3.533486f-5 3.2348104f-5 -7.659576f-5 0.00013982822 -6.3890446f-5 5.1099894f-5 -3.6681195f-5 9.415396f-5 0.00014207355 3.2957967f-5 -1.5705988f-5 -7.967105f-5 -2.180348f-5 -0.00013320909 -0.0001059336 8.3487f-5 2.282055f-5 5.110875f-5 -3.27335f-5 4.104298f-5 8.464971f-5 -1.7010881f-5 -1.4406375f-5 2.8203609f-5 5.213847f-5 0.0002157011; 0.00019730491 -0.00011324993 6.4773994f-5 0.000120778604 -5.4715092f-5 0.000103474944 -0.00018960478 4.083243f-5 -1.7024828f-5 -7.3320516f-5 -1.5217254f-5 -6.2785366f-5 6.3320535f-5 -4.9529168f-5 -1.9787989f-5 1.2121187f-5 7.928497f-5 -7.016558f-5 -0.00024311207 9.609716f-5 -2.9102543f-5 -3.4667796f-6 -2.930603f-5 5.678993f-5 8.58904f-5 -3.9016104f-5 0.00010542277 -4.1077605f-5 -9.56267f-5 -0.0001131797 -5.798505f-7 -6.970267f-5; -0.00013635393 5.3281066f-5 1.2604041f-5 7.2100185f-5 8.5174026f-5 -4.909403f-5 -2.3897335f-5 -1.1963698f-5 -7.487817f-5 0.00014533561 -5.2880594f-5 0.00010019395 -2.3352486f-5 0.00010543097 4.388293f-5 -5.3105534f-5 3.143798f-5 -6.4436004f-5 1.1926574f-5 -9.614401f-5 -0.00024020273 5.829732f-7 -0.0001988018 -3.7190214f-6 0.00012052626 2.0961197f-6 -6.0311537f-5 -6.3592706f-5 -0.00012088332 -5.7647132f-5 -1.2393728f-5 8.2915765f-5; 6.1345386f-6 -7.6362536f-5 5.3765398f-5 2.2559276f-5 -0.00020616876 -0.00017645433 0.000102468526 -7.835819f-5 -2.275776f-5 1.1462957f-5 -5.1330848f-5 -2.8339622f-5 3.2293367f-5 -5.2020674f-5 1.1928305f-5 -0.00014077095 -5.1057017f-5 -5.2451684f-5 4.570732f-5 -6.1287014f-6 -2.429942f-5 6.568433f-5 9.884206f-5 -0.00016328818 1.984889f-5 -7.699378f-5 2.0554815f-5 -0.00013284909 0.00014174149 4.0874344f-5 8.8765184f-5 0.00012290153; -7.284601f-5 0.00015113266 -8.370722f-5 -0.00010180767 -3.4999375f-5 -0.00018356094 0.00010763813 3.3040036f-5 6.449167f-5 -0.000103754785 0.00010147674 9.896282f-5 -8.177997f-5 1.5517104f-6 0.00017423871 -6.703495f-5 9.930151f-5 -0.00017044914 -4.4303477f-5 4.5368917f-5 -0.000130783 2.013616f-5 6.454598f-5 -0.00020466343 4.2728916f-5 -0.00015525491 9.641593f-6 -8.8821635f-6 0.00013840995 -7.628754f-5 5.2094652f-5 7.693837f-5; -1.936521f-6 -0.00012887 9.0301095f-5 0.00021892734 -5.0441475f-5 3.4005883f-5 0.0001176917 8.137417f-5 -6.481463f-5 3.448867f-5 8.3453546f-7 -0.00012861127 3.8067217f-5 0.00012599569 -0.000100411744 -8.965601f-5 4.6392575f-5 0.00017151685 2.4286144f-5 -4.66038f-5 5.022698f-6 0.00015643479 -4.898879f-5 -3.461263f-5 0.00012742943 6.658684f-5 -6.0342363f-6 -3.1366388f-5 -1.57381f-5 -7.21164f-5 0.00014184127 8.698141f-5; 0.00010477184 -0.00021518409 -0.00020540101 -0.00017861997 -8.4404535f-5 -3.70074f-5 3.9613344f-5 1.9685473f-5 -5.9209102f-5 -0.00015158781 6.52442f-5 6.160428f-5 -9.181076f-5 0.0001256964 0.00015453056 4.0800082f-6 2.2091914f-5 -3.218061f-5 -0.00012707147 -0.000215632 -1.9369683f-5 2.834695f-5 4.242915f-5 4.7958943f-6 -3.186571f-5 6.009155f-5 0.00021139008 4.160474f-5 -2.7288033f-6 4.625632f-5 -4.3435106f-5 7.146487f-5; 4.374089f-5 7.488287f-5 -9.8469856f-5 -0.00012004942 0.0001945894 2.170725f-5 1.7937835f-5 2.7179507f-5 -2.1116573f-5 -5.5853932f-5 3.3326544f-6 -9.3491435f-6 0.00010537235 6.931845f-5 -3.983922f-5 -4.7289675f-5 1.5653004f-5 -0.00022400402 5.7491754f-5 1.715078f-5 2.8108143f-6 5.3561467f-5 -5.169309f-5 -2.5652089f-5 -8.073286f-5 -2.4996185f-5 4.34875f-5 -0.00016427264 1.611109f-5 6.0825427f-5 3.671255f-5 -3.9133418f-5; 0.0002289462 9.411186f-5 -2.7011092f-5 -7.508443f-5 3.0149811f-5 -8.611271f-5 -1.721116f-5 0.00014790527 1.8963052f-5 0.0003108386 -0.00011547375 1.3582856f-5 -4.9441587f-5 -8.9482746f-5 8.650211f-5 -0.00014843841 3.330396f-5 7.7672965f-5 -2.6569027f-5 -5.260992f-5 -5.758783f-5 0.00012981516 8.676786f-5 -0.00012019002 0.00013454087 0.00012962654 -0.000145846 -2.2925251f-5 0.00014683808 0.000111876994 -6.814566f-5 -8.991099f-5; 0.00015247412 -5.2220308f-5 3.303562f-5 -4.0516537f-5 5.0913433f-5 4.4803448f-5 -9.616838f-5 1.784521f-5 7.0508744f-5 0.00018072716 -8.2390274f-5 -0.00016511144 6.84019f-5 -6.5895496f-5 0.00013130245 -0.00010432372 1.1481878f-5 -1.4197938f-5 -2.5276172f-6 1.1729372f-5 3.9101567f-5 5.2090098f-5 5.913563f-5 -2.8327304f-5 -9.641792f-5 0.00017448186 -5.872737f-5 1.536804f-5 7.327565f-5 0.00012235604 7.6064957f-6 1.5156055f-5; -0.000154704 1.4018213f-5 -4.0804447f-5 0.0001416553 4.0145318f-5 3.6607282f-6 8.820955f-5 3.786613f-5 -0.00020226216 4.6105897f-6 5.550528f-6 -0.00010234916 6.716201f-5 -6.043639f-5 -2.6334108f-5 -7.417747f-5 -7.517043f-5 1.1571982f-5 -5.7862057f-5 -3.0656676f-5 -3.4289274f-6 9.702859f-5 0.00014102783 -3.5850444f-5 1.851716f-5 8.838451f-5 -0.00016915894 -5.8839592f-5 0.00017786834 4.4703054f-5 -8.068646f-5 -0.00016312697; 4.442865f-5 -0.00012006202 -2.6558766f-5 5.672547f-5 5.6829463f-6 -6.2722575f-5 -5.5414985f-5 -6.616521f-5 -0.0002716475 -3.1431762f-5 2.5350297f-5 -1.0963497f-5 5.8565315f-6 1.1280099f-5 3.505499f-5 -2.5387439f-5 -4.471935f-5 1.5624903f-5 4.9636583f-5 0.00013870077 -0.00010204634 -1.8144423f-5 6.716244f-5 7.007615f-5 8.5767846f-5 2.7059134f-5 -0.0001835215 -5.475444f-5 -2.0878162f-5 -0.00017755636 -1.3773209f-5 0.00010980106; 5.2602372f-5 -6.791942f-6 -6.0015762f-5 -8.829484f-5 -7.536287f-5 -3.9357703f-5 -0.0002533797 -0.00019349939 -2.6312156f-5 -0.00014407777 5.654903f-6 -8.596633f-5 0.00010207755 -0.00014197214 -0.00021095501 -8.949662f-5 -8.78758f-5 8.230468f-5 5.0060156f-5 1.3623026f-5 8.192343f-5 9.648081f-5 2.901343f-5 -2.8943532f-5 0.00014445429 -0.00014949995 2.8125742f-5 9.118809f-6 3.9546387f-5 0.00011677827 0.00013162084 -9.251267f-5; 0.000111023226 8.540807f-5 -9.632401f-5 -2.0421841f-5 -0.00012313662 -5.2018618f-6 -5.1790317f-5 -0.00016129599 3.4691588f-5 0.00012465993 -5.4743723f-5 -7.375791f-5 -4.2607066f-5 5.131311f-5 0.000105108054 9.257429f-6 7.963778f-5 -0.00022747654 -6.015767f-5 6.143162f-5 0.0001239677 -7.84972f-5 -9.273924f-5 -9.1272304f-5 -5.0631555f-5 1.960298f-5 6.176991f-5 1.42621575f-5 -0.00014048498 -9.409552f-5 -6.69471f-5 -0.00012498979; -9.983786f-5 -9.002071f-5 6.0303675f-5 -4.6285055f-5 -4.9538403f-6 -2.9290824f-5 -5.955555f-5 -0.00017678842 5.0389506f-5 9.9634635f-6 -0.00013858659 7.5097923f-6 -7.146621f-5 -8.4323365f-5 -0.00018398082 4.5715612f-7 0.00019981673 7.600076f-5 -0.000113493494 6.6738976f-5 4.4671888f-5 5.1766674f-6 4.957016f-5 -5.165908f-5 -5.7047462f-5 -0.0001406546 2.3864255f-5 -3.3130655f-5 -3.6680107f-5 1.23412165f-5 0.000109933746 4.6290555f-5], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-9.012873f-5 7.417051f-5 0.00025161545 4.9274546f-5 0.00010122614 -5.0163922f-5 -0.00012842058 -0.0001206786 -2.989958f-5 -8.010971f-5 1.5434154f-5 -0.00016001904 -1.2780457f-5 -2.7766466f-5 7.45467f-5 2.1860842f-5 -0.00012804814 -7.033757f-5 -0.00016812379 -7.759783f-5 3.6243237f-6 9.2822374f-5 -2.3109782f-5 0.00027652478 -4.066539f-5 0.0001177831 2.4505709f-5 -0.00012254887 7.928667f-5 0.00022943318 1.2903607f-5 1.035586f-6; 5.3552827f-5 0.00012438484 0.00010002195 -4.2746266f-5 -8.1811064f-5 -6.5830077f-6 -9.607851f-5 -1.5115101f-5 1.7638582f-5 -0.00012033503 2.8257547f-5 -5.0352584f-5 -1.6083617f-5 0.00010446825 -0.0002548531 4.9123146f-6 -0.00016278515 1.3834254f-5 -0.00015796158 0.00020403395 -3.1378113f-5 -7.720094f-6 -7.166389f-5 -1.7671651f-5 -5.650439f-5 9.3580245f-5 -7.139312f-5 7.729014f-5 -0.00015755462 -2.9734461f-5 -6.472176f-5 -4.8089343f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
-
-const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
-    Chain(
-        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
-        layer_2 = Dense(1 => 32, cos),  # 64 parameters
-        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
-        layer_4 = Dense(32 => 2),       # 66 parameters
-    ),
-)         # Total: 1_186 parameters,
-          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",m,[(a(),i("svg",V,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",B,[(a(),i("svg",z,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=h("where, ")),A("mjx-container",T,[(a(),i("svg",q,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=h(", ")),A("mjx-container",K,[(a(),i("svg",D,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=h(", and ")),A("mjx-container",P,[(a(),i("svg",U,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=h(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
-    χ, ϕ = u
-    p, M, e = ode_model_params
-
-    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
-    # it, however, in general, we should use \`st\` to store the state of the neural network.
-    y = 1 .+ nn_model([first(u)], nn_params)
-
-    numer = (1 + e * cos(χ))^2
-    denom = M * (p^(3 / 2))
-
-    χ̇ = (numer / denom) * y[1]
-    ϕ̇ = (numer / denom) * y[2]
-
-    return [χ̇, ϕ̇]
-end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
-waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    axislegend(ax, [[l1, s1], [l2, s2]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
-
-    fig
-end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
-
-function loss(θ)
-    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
-    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
-    return mseloss(pred_waveform, waveform)
-end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0007080899236081684
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
-
-function callback(θ, l)
-    push!(losses, l)
-    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
-    return false
-end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
-optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
-optprob = Optimization.OptimizationProblem(optf, params)
-res = Optimization.solve(
-    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
-    callback, maxiters=1000)
    retcode: Success
-u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [3.189576818839681e-5; -6.391393253576819e-5; -3.340563489471322e-5; 4.434318725546035e-6; 0.00011549952614571395; -0.00010187309817400318; 0.0002378841309108547; 0.00013548151764520408; 6.836328975620413e-5; -7.016834570084262e-5; -1.1637022907966216e-5; 0.00010799095616670282; 0.00010711853246899926; -8.205675840138618e-5; -3.5767006920644584e-5; 0.00013818619481746985; -4.6840108552708576e-5; 0.0001642029965294685; -3.842115256698744e-5; 5.441859320852504e-5; 9.166129530056265e-6; 2.3945689463266466e-5; -2.244984534627751e-5; 4.243526291244579e-6; 0.00014469436428041194; -7.296805415528246e-5; 8.163950406014773e-5; -0.00010950736759688676; 0.00011171042569901284; -4.209781764071136e-5; -0.0002324687375221788; -6.524128548331127e-5;;], bias = [9.824179453448208e-18, -3.283579809077081e-17, -2.4121052656024447e-17, -2.820816984579966e-18, 1.7800006399111622e-16, -1.2252003989273822e-16, 1.3596170846996523e-16, 3.7697396587690263e-16, 3.469541374929302e-17, -1.2239306697871398e-16, 4.732456804452487e-18, 6.710255596708555e-17, 3.620084831552526e-17, -7.37712055415936e-17, -6.498955501547213e-18, 7.565232283015596e-17, -7.142283934572076e-18, 1.969496943298209e-16, -4.3998375610902485e-17, 1.4517763361935025e-17, 2.2528160954460315e-18, 2.6719434458280483e-17, 5.412172157702465e-18, 3.58740882046328e-18, 3.1671129438060196e-17, -8.253106300035819e-17, 2.757169253238071e-18, -8.791378034823515e-17, 1.3521537011914334e-16, -4.6599032319848886e-17, 4.036589722165776e-17, -4.872073037560773e-17]), layer_3 = (weight = [-4.072823511703495e-5 -5.09450001936559e-5 -0.0001259864585244711 -2.9315197107257145e-5 0.00016674744975718865 7.63891147562606e-5 -5.600401354057221e-5 -0.00014100888082791929 -2.158594733371077e-5 -1.2108931596208436e-5 -9.704584962489625e-5 0.0001742295643544126 0.0002702743585450876 8.124682054587932e-5 2.3121163312703365e-5 -1.3257828950352319e-5 -6.28844719267135e-5 -5.0237235054718365e-5 -6.0434169508857686e-5 -5.904574581106778e-5 5.3575589763591236e-5 8.389868881752165e-5 6.905497602952288e-6 -9.497072391919446e-7 -3.320390905171548e-5 -0.0003212005468321654 0.0001560697491277318 6.604789078667043e-5 -4.8091948056004725e-5 0.00012601838526281895 -2.4301196472136584e-5 7.288929910830549e-5; 5.081267089944231e-5 0.00017587639239744914 -0.0001901418340891797 -0.00016309728881215715 -1.4870646289379445e-5 0.00010889856866223033 -9.05132087680134e-5 -9.43336467641714e-5 0.000142089973898955 -4.825303904700446e-5 -0.0001510564520755974 3.31889191259365e-5 -0.00010077403340593758 0.00012599683622492008 1.5012522901353627e-5 8.972270233451054e-5 -1.3044723807714801e-5 -0.00015159335953988042 0.00011351757938636337 0.0001516661527532875 0.0001435068065732527 -0.00010061746934998852 -0.00016779499629383977 0.00016463631845207847 6.636214902053133e-5 0.00010201420308684551 0.0001155409067835426 -0.0001398492509067325 9.045491105306053e-5 3.674142362096196e-5 4.865882373656356e-5 0.00012175496021181036; 0.00022024982202853847 -5.519529771869915e-6 -9.600964373387555e-5 0.00013558740496225476 -5.821833807569876e-5 -7.169227288980036e-5 8.991714306465219e-5 -0.00010023864857796439 0.00017261645583491674 -3.408923263489361e-5 -0.00019824269808307467 0.00018713126367945243 -8.108157604876883e-5 3.3114480480411414e-5 -0.00010184901444884173 -5.5567568473656066e-5 -6.268739495137848e-5 -0.00011883301309719392 0.00013338797028697872 -2.658692361731209e-5 4.584856775156783e-6 -8.509010482523759e-5 -2.3480024232437663e-5 9.172469923189455e-6 7.118406968945824e-5 0.000114721893929503 -1.7309833914647513e-5 -8.865269743548675e-5 5.1576735437242696e-5 -9.788579306043833e-5 5.233503937777888e-5 -7.615570908822224e-5; 0.00020963441425483624 -1.864013388039402e-5 0.00011834211550667836 -6.72948844519828e-5 -0.00015167990563421714 -7.08284969230501e-5 3.436825141329051e-5 -0.0001698451632771909 9.74804461795482e-5 6.599940757515103e-5 -6.250048676956555e-5 -0.00012259212593998138 -0.0001928729288418158 -0.00012372035047955503 -0.0002444638630047816 -3.005956927589448e-5 0.0001629407816428321 9.313154082994632e-5 -0.0001398982530597972 -1.6169224131586253e-5 -9.899676974598491e-6 -3.678356728850944e-5 3.872238090043803e-5 0.00010280328211156535 -6.28813831506549e-5 3.451919842992198e-5 3.668987124475024e-5 1.649340816148732e-5 -1.0300211165293852e-5 -3.4339540841721874e-6 6.583667350718321e-5 -4.138688332017845e-6; 3.143527677819532e-5 0.00010077263899800934 -0.0001992857793629967 -8.428917371946206e-6 2.0227933728156163e-5 -1.8326939597203966e-7 -4.314327421523914e-5 -6.050110792785467e-5 0.00013706251806068834 -8.744973425565604e-5 -8.169817707313807e-5 1.0383734970663711e-5 -1.1153668001507944e-5 2.0162168166267354e-5 -8.43102022008836e-5 0.000147129955021222 -1.2929784734312002e-5 4.0949668200481244e-5 -0.00015691447302744298 -0.00013806619213939137 5.400692347391857e-5 -8.882123043805916e-5 7.982632845594918e-5 -6.044373336409806e-5 0.00010303148186721462 1.115299832234561e-5 -6.417246882848665e-5 -7.080362002041241e-7 1.0160255751533918e-5 -3.7934256088207734e-5 -9.273258880055931e-5 6.58776483214611e-5; 0.000156290270588567 -0.0001003058989041292 0.00014176820133833353 -1.841901592307562e-5 -0.00010336604298537744 -3.74450853961715e-6 1.328861237231886e-5 0.00011020193216643964 -0.00012162500041050782 0.00010194693624761005 -0.00021031103658934895 -0.00013408781358486153 -0.00016633264583980773 -0.00011588570321656757 -0.00014500531522536515 0.0002387293296309346 -0.0001908205796822708 0.00020116539401022254 4.6418690381312165e-5 4.109205071819074e-8 -4.1171655702431014e-5 5.739714635673128e-5 0.0002493166046591676 1.0115952094448078e-6 -6.098011617376218e-6 1.7907312049270067e-5 0.00020892571212278687 0.00012901733300203338 5.765726547941438e-5 5.2601100669914204e-5 -1.7498905599156158e-5 -0.00023694786630550335; 7.645258701715071e-5 -0.00013919706143595875 5.0454979281233715e-5 -0.00010899297522072281 8.444844242959467e-5 9.373354659957113e-5 4.927437876076855e-5 9.575341791688568e-5 -0.00017926344626274568 5.26486295702061e-5 -0.00015499095352523564 6.000099929125259e-5 -0.0001234756815402654 -1.0212086273065653e-5 -3.7479956155117537e-5 -2.8888703674137614e-5 -2.5838733200622807e-5 -0.00014959404106874985 -4.838748973369222e-5 -0.00010264589545939696 -2.2372690728922117e-5 0.00018736748861457612 -1.4590840329298094e-5 7.58790814860857e-5 6.715533199795584e-5 -8.454949785197743e-6 0.00010300754038227083 9.579991856199895e-5 6.29892860151009e-5 0.00013727077421109564 5.843420913052272e-5 -7.581266581936409e-5; 9.868717612555944e-6 0.00013001370713595114 -0.00015903904680613504 0.0001072664006335285 -7.944006323101522e-5 -0.00010838345154639748 -3.0476384594093793e-5 -5.3469765802004945e-5 -0.0001706641062207913 5.896729159416418e-5 -4.431872115106359e-5 0.00011518689895296008 9.390775700579875e-5 -3.4952162635599276e-5 -2.5256801776556165e-5 3.693378199953781e-5 5.578826564146093e-5 8.749130826457307e-5 7.1461293066412356e-6 -4.0958833081965815e-5 -6.223777130231299e-5 -8.734828700995126e-5 0.000120435832983497 -0.00015020162503220176 5.2773043322517866e-5 0.00023605301863994222 -6.228334790081066e-5 6.480454500546205e-5 0.00010643072779770449 -5.217936380518698e-5 3.147694839410321e-5 0.0001220733963839318; 0.00016179558928054307 -3.8237744104818825e-5 7.378816223378737e-6 -6.36834049720832e-5 5.370086340629576e-5 -5.645165243838028e-5 2.38898644391641e-5 1.3118267576678817e-5 3.85824093762756e-5 3.0390674270919503e-5 0.00014524687479645457 -9.020301901106715e-5 -8.770921364263631e-5 -7.468396894470992e-5 0.0001247548386680384 -1.5613943344910157e-5 2.9725193359652275e-5 -5.011713461786774e-5 -2.4990479017262914e-5 6.159416419616535e-5 3.494954386793892e-6 1.957585242805049e-6 -9.42242297816603e-5 -9.740898191300713e-5 -9.572151999524389e-5 -1.9087357604430672e-7 -6.354340099566823e-5 -8.419951724424006e-5 6.416051086201375e-5 -0.00019523252218566324 6.95840897031777e-5 -1.4271764724227435e-5; -2.422680986988012e-5 8.622644858782885e-5 0.00013249322259448345 -0.0001682267913164207 -9.379862709088791e-5 -5.4563545887372177e-5 -7.160413329608878e-5 -9.77785613539327e-5 -1.5140502142324223e-5 -2.2504314216214893e-5 1.726323611742261e-5 0.00020515204127527292 -0.000156979688795996 0.00014193265710141196 -8.208222530434039e-5 0.00019841713842119285 2.172136450476319e-5 -7.380812272082148e-5 8.412951032746447e-5 7.050299558748259e-6 0.00014390179316628636 1.1923136156973306e-5 -0.00014427890815191073 -4.404527913585472e-6 -9.287719254267265e-5 1.8614092227059548e-5 8.723527466294975e-5 0.00021786714100393905 3.655884342282818e-6 -4.0111896992904666e-5 2.7520733823738473e-5 -1.2172112434066336e-5; 2.1144479453016605e-5 0.00014967567678637593 -0.00022895353047546222 0.00024803920225336914 6.106823550168255e-5 7.726782771156918e-5 -9.149555715169725e-5 -0.00013571456381954395 -1.2743545065074921e-5 3.7126900699024794e-5 5.7416314319765904e-6 5.748096575064684e-6 1.4281108653324745e-5 1.661069202042451e-5 3.50640313796474e-6 0.00017465767725467546 -3.95480246680904e-5 -7.454405293004695e-5 -7.768760131342562e-5 -4.0875661377992795e-5 -9.29251926039576e-6 -7.646131141713119e-5 -5.933069130891232e-5 5.0295358070672964e-5 3.072849633268878e-5 -5.4951638028921765e-5 -9.214827603331037e-5 4.75910119547398e-5 -1.3360425811123943e-6 0.0001243740871558316 -4.200522833174072e-5 -6.518613097626927e-5; 5.216586965387029e-5 2.3043880969255088e-6 -2.9499822396721737e-5 6.810199216258969e-5 4.4266933634826405e-5 0.00014037393362512332 -9.01380257574673e-6 3.275076620182626e-5 -6.2986593934707995e-6 2.6861458770761945e-5 -0.000113002246273502 -0.00017712906605313057 0.00022498223211867276 9.065926520404789e-5 -1.6969002665539284e-5 9.567631657302037e-6 9.605020984488702e-6 1.7971921131432287e-5 3.0612678763170755e-5 -4.2209135920094944e-5 4.7286350831903895e-6 -4.0714210104147525e-6 -0.00016515781657519486 4.407404283008284e-6 1.4919331313177951e-5 -0.0001245337878049409 1.5247253265878265e-5 -4.9946505662337705e-5 -0.00010899481616663255 5.947026898636771e-5 4.765898143658767e-5 -0.00019273749398282397; -0.00014856220460460773 4.997566847050341e-5 0.0001471106521323205 7.564508322429582e-5 -5.5392246768680794e-5 -0.00010557082660925735 8.883700555925537e-5 7.19641835432398e-5 -4.4193551194260185e-5 -7.311395112156149e-5 4.881487674465144e-5 -0.0001266499888613928 7.639050508187e-5 -0.00010223080554242442 0.00017258334318211984 -2.6130755405394208e-5 -4.5941574545216334e-5 0.00018645044527984222 0.00014535845601967287 -1.632059266812866e-5 -0.0002557868714673563 -2.4910950206325143e-5 -1.5766952320339846e-5 4.325155222358853e-5 0.00015311768273859223 -0.00011235110961097501 -1.9894279474126975e-6 -0.000195880274450976 -0.00021126755692494242 8.495977143793123e-5 7.781756775257433e-5 6.961078232888846e-5; 9.387014516933704e-5 0.000151653384923252 5.2711861021368445e-5 -9.786186627506336e-5 6.203989697923951e-5 0.00011791795633571468 -8.99070690914685e-5 -3.1979838107002867e-6 8.467955117070908e-6 -7.408731097284129e-5 -1.6123978535044658e-5 -0.0001541797354414483 0.00010376478541247726 1.5215222551617784e-5 0.0001081674018812073 -5.207739367079272e-5 1.2381458977603318e-5 0.00018003360684332612 -2.995489911453944e-5 -5.492569097013449e-5 -8.313153729003915e-5 8.428551165225825e-5 -5.026113275134092e-5 2.745046378068568e-5 6.964216674049922e-5 -1.6384361411197174e-5 -9.398162442536506e-6 -0.00011314247593445556 0.00013795642381837713 5.8321446080941926e-5 -2.257776021481562e-5 3.1596533621909674e-5; 4.528550362491412e-5 0.00015189332571507132 0.00012069307359096032 -0.00011953957636680179 -7.683403436631394e-5 -2.7574398877464584e-5 -1.6829341070353303e-5 -0.00013169591165451328 6.219640436510303e-5 -7.402007845222216e-6 7.437757977212133e-5 -0.00017041730661249363 -7.11322229060031e-5 -2.846614024266207e-5 6.357425245849798e-5 -6.572376782694359e-6 -4.935723899935839e-5 -9.415075687402303e-6 -5.389176814917797e-7 -1.558777712008384e-5 -2.876209163761416e-5 2.6240393531484048e-5 0.00010650497807117498 -8.974485531067046e-6 -8.316926886863775e-5 -8.448889474922055e-5 -0.0001286435746757973 1.948203299006832e-5 -6.270600915259254e-5 6.095104054592046e-5 -0.00020690380387774357 -2.4949526227566494e-5; -9.634147099579145e-5 2.9957840891336297e-5 -8.814466090999923e-5 0.00029132406089327705 -8.873629812742245e-5 -8.915869657076024e-5 2.4141174802176025e-5 -0.00016414398814676247 -0.00013065672792155194 -0.0001623345884553702 -5.3915336681637446e-5 -1.1634160361519357e-5 7.1220037199041655e-6 -4.594664969491395e-5 0.00019775689034130065 -9.505146553864519e-5 -0.0001270878416079679 -9.55624196620906e-5 -4.681690151930187e-5 -0.00014748457724355295 -4.5821663295026476e-5 -0.00012706775996495492 -3.135084952820556e-5 -5.041618678380332e-5 -9.929731422461133e-5 0.00024250686729200373 7.741384652316938e-5 -9.606933562909097e-5 -3.917681496832297e-5 -7.1145324542991e-5 0.00012290911436426698 2.004287162261593e-5; 2.6072455727442996e-5 0.00023533272563963474 4.638915582927304e-5 -4.43118258435799e-5 4.156228191004157e-5 1.0032324314120804e-5 0.00021525445866880696 -5.069982546962557e-6 8.614665806544275e-5 -8.925991620617663e-5 -7.541993160248347e-5 7.45393957640346e-6 -3.3796269929557296e-5 -0.0001935173529664127 6.444589632336342e-6 -6.779806041141176e-5 0.00024368877005781378 -0.00022429263095819746 -8.25683780594787e-5 0.00023213715415931616 -6.024325899761714e-5 3.637937355688216e-5 -5.432554883556168e-6 0.0001707022859018242 -0.00035844953141623366 0.00011860123435053287 -0.00014000128773023168 -2.6945381035152222e-5 4.957010263478429e-5 7.847020991630263e-5 -0.00016146013855451884 0.00016950217490102439; -7.043909705477776e-5 7.863978838779784e-5 -3.54856367828419e-5 -3.363144265837922e-5 0.00018373390934195612 -0.00015216271118495206 -3.53331836429825e-5 3.234977861049552e-5 -7.659408425913405e-5 0.00013982989343114906 -6.388877069345636e-5 5.1101569386649514e-5 -3.6679519563928514e-5 9.415563703280549e-5 0.0001420752248470576 3.295964210175835e-5 -1.5704312556200466e-5 -7.966937688414516e-5 -2.1801805143487045e-5 -0.0001332074155123078 -0.00010593192406465257 8.348867571678961e-5 2.2822224980909573e-5 5.1110424227072386e-5 -3.2731825069079087e-5 4.104465346083729e-5 8.465138829544799e-5 -1.7009206441595775e-5 -1.4404700106549434e-5 2.8205284014101566e-5 5.2140146852432286e-5 0.0002157027790765166; 0.00019730367283343157 -0.00011325116567307211 6.477275972726577e-5 0.0001207773698145222 -5.471632677356395e-5 0.00010347370924434819 -0.0001896060107507815 4.083119573195054e-5 -1.7026062338780957e-5 -7.332175074837196e-5 -1.5218488912733589e-5 -6.278660068048847e-5 6.331929988234545e-5 -4.953040253597263e-5 -1.9789223658090315e-5 1.2119952301216232e-5 7.928373314213804e-5 -7.016681549131931e-5 -0.00024311330118210217 9.609592460335126e-5 -2.910377726651017e-5 -3.468014271430115e-6 -2.930726397310587e-5 5.678869500413314e-5 8.588916760643641e-5 -3.9017338637430295e-5 0.00010542153402936693 -4.107883934623298e-5 -9.562793776851796e-5 -0.00011318093813018094 -5.810851614175347e-7 -6.970390451599896e-5; -0.00013635585733932896 5.3279143085132605e-5 1.2602118058605339e-5 7.209826193506313e-5 8.517210275390521e-5 -4.90959525086064e-5 -2.389925853996703e-5 -1.1965621215445414e-5 -7.48800909240845e-5 0.00014533368570486814 -5.28825172841227e-5 0.00010019202940388812 -2.3354409367994097e-5 0.00010542904541584576 4.388100691361851e-5 -5.31074571517319e-5 3.143605621520986e-5 -6.44379275944988e-5 1.1924650916664847e-5 -9.614593242072826e-5 -0.0002402046544920797 5.810499283526083e-7 -0.0001988037280716143 -3.7209446508843805e-6 0.00012052433643480654 2.094196443108954e-6 -6.0313460071441674e-5 -6.359462955510926e-5 -0.00012088524332279989 -5.764905532483414e-5 -1.2395651228799042e-5 8.291384196131248e-5; 6.13265146020744e-6 -7.63644228849465e-5 5.3763510631684424e-5 2.2557388628770116e-5 -0.00020617064265970482 -0.00017645621383999068 0.00010246663904548159 -7.836007981531355e-5 -2.27596473545436e-5 1.1461070325456786e-5 -5.13327349021029e-5 -2.834150919661371e-5 3.229147941019859e-5 -5.202256116754194e-5 1.192641783709016e-5 -0.00014077283677615214 -5.1058904237291846e-5 -5.2453570706699625e-5 4.5705431229722175e-5 -6.1305885632492694e-6 -2.430130635767008e-5 6.568244597437593e-5 9.884017080330087e-5 -0.00016329006294691238 1.9847003468029552e-5 -7.699567041563707e-5 2.0552927789685223e-5 -0.00013285097785264679 0.0001417396025222873 4.0872456420034774e-5 8.876329692578255e-5 0.000122899645329742; -7.284701142105474e-5 0.00015113165126769704 -8.370822625345749e-5 -0.00010180867694196524 -3.5000379381177776e-5 -0.00018356194787536639 0.0001076371262514288 3.3039032035799645e-5 6.449066849547358e-5 -0.00010375578868313764 0.00010147573620435553 9.896181281312519e-5 -8.178097060060917e-5 1.5507062187873954e-6 0.00017423770613624525 -6.703595145715941e-5 9.930050864006232e-5 -0.0001704501483856543 -4.4304481076488265e-5 4.536791302036357e-5 -0.00013078399943531718 2.013515576783818e-5 6.454497624310447e-5 -0.00020466443684761645 4.2727911835596774e-5 -0.00015525591161911432 9.640589172541938e-6 -8.883167635009892e-6 0.00013840894088736912 -7.628854664852085e-5 5.2093648167863977e-5 7.693736542371006e-5; -1.9334187523694303e-6 -0.0001288668983303012 9.030419719275371e-5 0.0002189304385414387 -5.043837259072836e-5 3.400898535837695e-5 0.00011769480156576498 8.137727386419124e-5 -6.481152845444233e-5 3.449177061193522e-5 8.376376062284453e-7 -0.00012860816527754854 3.807031939722761e-5 0.00012599879378322225 -0.0001004086417647983 -8.965290648994648e-5 4.6395677464177725e-5 0.0001715199494814266 2.428924640335857e-5 -4.660069785094722e-5 5.025800283338583e-6 0.0001564378934294019 -4.8985687635276715e-5 -3.4609526801058776e-5 0.00012743253578299793 6.658994502723074e-5 -6.0311341709135985e-6 -3.1363285551926876e-5 -1.5734997895927483e-5 -7.211330116623792e-5 0.00014184437427534283 8.698451242744888e-5; 0.00010477013827213785 -0.00021518579044466895 -0.00020540271245962234 -0.00017862167326355638 -8.44062367160201e-5 -3.700910267623787e-5 3.961164227252867e-5 1.968377116456888e-5 -5.9210803307071616e-5 -0.00015158951166639118 6.524250172689278e-5 6.160258023912445e-5 -9.181246307532888e-5 0.00012569470097689044 0.00015452885534060902 4.078306823363043e-6 2.209021229092797e-5 -3.2182312430513194e-5 -0.00012707317248540205 -0.00021563369839539874 -1.9371384702104172e-5 2.834524936944457e-5 4.2427448783123144e-5 4.7941928404855354e-6 -3.1867412622953456e-5 6.008984680549249e-5 0.00021138838317157782 4.1603039129696805e-5 -2.730504753415649e-6 4.625461704009955e-5 -4.343680732386287e-5 7.146316900063252e-5; 4.374030813774188e-5 7.488228711731884e-5 -9.847043711492695e-5 -0.0001200500033519782 0.00019458882288741277 2.170666939487488e-5 1.7937254051470556e-5 2.7178926211450773e-5 -2.111715421055745e-5 -5.5854513084383856e-5 3.332073583762591e-6 -9.349724314275018e-6 0.0001053717727699719 6.931786756463552e-5 -3.983979931949539e-5 -4.729025622513638e-5 1.5652423280655655e-5 -0.00022400460020439108 5.749117317459649e-5 1.7150199164418895e-5 2.8102334908473548e-6 5.35608864083336e-5 -5.1693669411222534e-5 -2.5652669683084067e-5 -8.073344233711706e-5 -2.4996765751026667e-5 4.348691927585022e-5 -0.00016427322279927948 1.6110508477099578e-5 6.082484596841146e-5 3.671196955107326e-5 -3.913399868910976e-5; 0.0002289486765587484 9.411434079108195e-5 -2.7008614742707937e-5 -7.508194899831204e-5 3.0152288709595648e-5 -8.611023436946012e-5 -1.720868200189139e-5 0.00014790775097958878 1.8965529343647317e-5 0.00031084107835364813 -0.00011547126992784686 1.3585333399770132e-5 -4.943910967971433e-5 -8.948026876144895e-5 8.650458406943489e-5 -0.0001484359344974525 3.3306438163226055e-5 7.767544233524609e-5 -2.6566549385943286e-5 -5.2607443612697635e-5 -5.758535186007862e-5 0.00012981763942686558 8.677033842129356e-5 -0.00012018754565336047 0.00013454334479343072 0.00012962901750167164 -0.0001458435253514241 -2.2922773459631004e-5 0.00014684055717249084 0.00011187947169698394 -6.814318108788655e-5 -8.99085097987449e-5; 0.0001524763183963476 -5.221811348996438e-5 3.3037814352846633e-5 -4.0514342765836275e-5 5.091562801430886e-5 4.480564260780046e-5 -9.616618896214645e-5 1.7847404739992356e-5 7.051093874657169e-5 0.00018072935657733683 -8.238807910196516e-5 -0.00016510924884892 6.840409790788863e-5 -6.589330129167569e-5 0.00013130464961910515 -0.00010432152608951083 1.148407230835695e-5 -1.4195743426863102e-5 -2.5254225536087655e-6 1.1731566735086957e-5 3.910376202589805e-5 5.209229224170496e-5 5.913782391622606e-5 -2.8325109235756322e-5 -9.641572520445694e-5 0.00017448405291111741 -5.872517700731323e-5 1.5370234029661434e-5 7.327784718404539e-5 0.0001223582340001114 7.608690372961008e-6 1.515824991407245e-5; -0.00015470543068577935 1.4016783171330845e-5 -4.0805876705333804e-5 0.00014165386899630094 4.014388840086942e-5 3.659298573344575e-6 8.820811861453015e-5 3.786470195418644e-5 -0.0002022635925814998 4.609160060584738e-6 5.549098278507849e-6 -0.00010235058645694412 6.716057767416446e-5 -6.0437818246196376e-5 -2.6335537694747692e-5 -7.417890075803277e-5 -7.5171857969606e-5 1.1570552561265787e-5 -5.7863486216872975e-5 -3.065810589678892e-5 -3.4303570092641735e-6 9.702716131950309e-5 0.00014102640496356923 -3.585187361669241e-5 1.851572968089565e-5 8.838308356725419e-5 -0.0001691603719510617 -5.884102203626996e-5 0.0001778669066706221 4.470162464794341e-5 -8.068789240413099e-5 -0.0001631283993620916; 4.4426369299592315e-5 -0.00012006429676623617 -2.6561045501900908e-5 5.672318877693364e-5 5.680666731479961e-6 -6.272485404699174e-5 -5.5417264207046406e-5 -6.616748794405194e-5 -0.0002716497787107297 -3.143404171644247e-5 2.5348017174812176e-5 -1.0965776141546363e-5 5.8542519807642565e-6 1.127781945322305e-5 3.505270944949432e-5 -2.538971818941064e-5 -4.4721628342109925e-5 1.5622622981749016e-5 4.963430330896957e-5 0.0001386984855569372 -0.00010204861825989456 -1.8146702473339157e-5 6.716015707276895e-5 7.007387250702925e-5 8.576556649567345e-5 2.7056854037051348e-5 -0.0001835237729894497 -5.475672003305463e-5 -2.0880441639035963e-5 -0.00017755863734712595 -1.3775488128143772e-5 0.00010979878209107037; 5.259896126273798e-5 -6.795352858422536e-6 -6.001917289057025e-5 -8.829824808588582e-5 -7.536628116455649e-5 -3.93611139613029e-5 -0.0002533831052302163 -0.00019350280352469694 -2.6315567522647815e-5 -0.00014408118157250598 5.651491862698763e-6 -8.596973792252105e-5 0.00010207414153895346 -0.00014197554854278972 -0.0002109584224696663 -8.950003255691461e-5 -8.787921113496737e-5 8.230126833709747e-5 5.005674531032161e-5 1.3619614507954043e-5 8.192001543404211e-5 9.647740218253537e-5 2.9010019277228988e-5 -2.894694305967406e-5 0.0001444508757388977 -0.00014950335615383206 2.812233062043259e-5 9.115398386105128e-6 3.9542975643902326e-5 0.00011677485737647723 0.00013161742528371577 -9.25160843350352e-5; 0.0001110199960913336 8.540484179145182e-5 -9.63272390271709e-5 -2.0425071290386567e-5 -0.00012313984860331249 -5.205091731042327e-6 -5.17935471924539e-5 -0.00016129921949133171 3.4688357680417196e-5 0.00012465669771524689 -5.474695297555959e-5 -7.376114024911682e-5 -4.2610295515187747e-5 5.130987903437382e-5 0.0001051048244499431 9.254199192663128e-6 7.963454901807268e-5 -0.00022747977289500564 -6.0160898448823084e-5 6.142839321734721e-5 0.00012396446310782835 -7.85004298285759e-5 -9.274246929811238e-5 -9.12755343796869e-5 -5.063478545881434e-5 1.959974962853483e-5 6.1766681590665e-5 1.4258927529299515e-5 -0.00014048820538111552 -9.40987514531405e-5 -6.695032732920575e-5 -0.00012499302045573937; -9.984058251908748e-5 -9.002343122452919e-5 6.030095264244136e-5 -4.628777733493452e-5 -4.956562773853718e-6 -2.9293546871173684e-5 -5.955827273611378e-5 -0.00017679114662646396 5.0386783729061495e-5 9.960741041448202e-6 -0.00013858931324981928 7.507069855223133e-6 -7.146893167705971e-5 -8.432608720218951e-5 -0.00018398354714342134 4.5443363442410884e-7 0.00019981400564241558 7.59980381156852e-5 -0.00011349621618470833 6.673625338984129e-5 4.466916525082814e-5 5.173944915744557e-6 4.9567438142124586e-5 -5.1661803266749204e-5 -5.705018466291802e-5 -0.0001406573295786636 2.3861532641912734e-5 -3.3133377113386027e-5 -3.668282934418554e-5 1.2338493976435376e-5 0.00010993102350438319 4.628783298934402e-5], bias = [4.437515830585584e-10, 1.9594993252322064e-9, -5.308393406822397e-10, -1.9317165401249315e-9, -1.6804369970380884e-9, 7.074235209701761e-10, 4.914899489757273e-10, 1.119375377567678e-9, -1.3062737465727374e-9, 1.0321347406851244e-9, -1.4277078442324252e-10, -5.782044560431486e-10, -2.9050243347228516e-10, 1.8372836237685824e-9, -2.679548484324747e-9, -4.117825409116929e-9, 8.405886536388658e-10, 1.6750511624909356e-9, -1.2346542742130683e-9, -1.923272028122853e-9, -1.887120020134511e-9, -1.0041787219358996e-9, 3.102150377145509e-9, -1.7014154634250893e-9, -5.808033600365818e-10, 2.4775001941826805e-9, 2.1946446036671373e-9, -1.4296521011660889e-9, -2.2795349908595105e-9, -3.4110441278516157e-9, -3.22996305276747e-9, -2.722484775317188e-9]), layer_4 = (weight = [-0.0007867009066537449 -0.0006224015980308057 -0.000444956728781826 -0.0006472975633747372 -0.0005953459865030687 -0.0007467360952999541 -0.0008249927572664053 -0.0008172507531907208 -0.0007264717282625864 -0.0007766818734608853 -0.0006811380285880877 -0.0008565912204912905 -0.0007093526384052144 -0.0007243385789572986 -0.0006220253485736737 -0.000674710999483002 -0.0008246203059352311 -0.0007669096947228696 -0.0008646959428964223 -0.0007741699374504568 -0.0006929477876594768 -0.000603749788953262 -0.0007196817701465205 -0.0004200473523943576 -0.0007372375652553526 -0.0005787889621067207 -0.0006720663781192687 -0.0008191210063249355 -0.0006172854121024927 -0.0004671387862449773 -0.0006836683670708925 -0.000695536447742295; 0.0003091439472017924 0.0003799759317775172 0.00035561306879658736 0.000212844829094219 0.00017378003779430537 0.00024900811024501776 0.00015951260887843164 0.00024047601132138021 0.0002732296908907871 0.00013525608072101174 0.00028384866901007055 0.00020523853473720257 0.00023950750399452675 0.0003600593464662572 7.379666400123996e-7 0.0002605033113944275 9.280597112235319e-5 0.000269425354240891 9.762953174736238e-5 0.00045962503873739547 0.00022421298252437347 0.0002478710205721238 0.00018392716073959746 0.00023791945092090395 0.00019908673084726726 0.00034917132288773224 0.0001841979643931747 0.0003328812460768363 9.803646552512888e-5 0.00022585658063969594 0.00019086928500806669 0.00020750172390734595], bias = [-0.0006965721831938202, 0.00025559112170524283]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
-
-    lines!(ax, losses; linewidth=4, alpha=0.75)
-    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
-
-    fig
-end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
-waveform_nn_trained = first(compute_waveform(
-    dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
-    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
-        alpha=0.5, strokewidth=2, markersize=12)
-
-    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
-        position=:lb)
-
-    fig
-end
    Appendix ​
    julia
    using InteractiveUtils
-InteractiveUtils.versioninfo()
-
-if @isdefined(MLDataDevices)
-    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
-        println()
-        CUDA.versioninfo()
-    end
-
-    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
-        println()
-        AMDGPU.versioninfo()
-    end
-end
    Julia Version 1.10.6
-Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
-Build Info:
-  Official https://julialang.org/ release
-Platform Info:
-  OS: Linux (x86_64-linux-gnu)
-  CPU: 128 × AMD EPYC 7502 32-Core Processor
-  WORD_SIZE: 64
-  LIBM: libopenlibm
-  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
-Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
-Environment:
-  JULIA_CPU_THREADS = 16
-  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
-  JULIA_PKG_SERVER = 
-  JULIA_NUM_THREADS = 16
-  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
-  JULIA_PKG_PRECOMPILE_AUTO = 0
-  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const M=t(e,[["render",Z]]);export{j as __pageData,M as default};
diff --git a/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.BfLFXCWx.lean.js b/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.BfLFXCWx.lean.js
deleted file mode 100644
index 0af67503bc..0000000000
--- a/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.BfLFXCWx.lean.js
+++ /dev/null
@@ -1,286 +0,0 @@
-import{_ as t,c as i,a2 as n,j as A,a as h,o as a}from"./chunks/framework.CCjWn1F9.js";const j=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),e={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},m={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},B={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},K={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},D={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function Z(X,s,R,x,w,O){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
-      OptimizationOptimJL, Printf, Random, SciMLSensitivity
-using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=h("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=h(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=h(", ")),A("mjx-container",r,[(a(),i("svg",d,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=h(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
-    M = m₁ + m₂
-    r₁ = m₂ / M .* path
-    r₂ = -m₁ / M .* path
-    return r₁, r₂
-end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=h("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",C,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
-    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
-
-    if size(soln, 1) == 2
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-
-        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
-        p, M, e = model_params
-    else
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-        p = soln[3, :]
-        e = soln[4, :]
-    end
-
-    r = p ./ (1 .+ e .* cos.(χ))
-    x = r .* cos.(ϕ)
-    y = r .* sin.(ϕ)
-
-    orbit = vcat(x', y')
-    return orbit
-end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
-    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
-    b = (v[3:end] .- v[1:(end - 2)]) / 2
-    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
-    return [a; b; c] / dt
-end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
-    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
-    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
-    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
-    return [a; b; c] / (dt^2)
-end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
-    x = orbit[1, :]
-    y = orbit[2, :]
-
-    Ixx = x .^ 2
-    Iyy = y .^ 2
-    Ixy = x .* y
-    trace = Ixx .+ Iyy
-
-    if component[1] == 1 && component[2] == 1
-        tmp = Ixx .- trace ./ 3
-    elseif component[1] == 2 && component[2] == 2
-        tmp = Iyy .- trace ./ 3
-    else
-        tmp = Ixy
-    end
-
-    return mass .* tmp
-end
-
-function h_22_quadrupole_components(dt, orbit, component, mass=1)
-    mtensor = orbit2tensor(orbit, component, mass)
-    mtensor_ddot = d2_dt2(mtensor, dt)
-    return 2 * mtensor_ddot
-end
-
-function h_22_quadrupole(dt, orbit, mass=1)
-    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
-    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
-    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
-    return h11, h12, h22
-end
-
-function h_22_strain_one_body(dt::T, orbit) where {T}
-    h11, h12, h22 = h_22_quadrupole(dt, orbit)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
-    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
-    h11 = h11_1 + h11_2
-    h12 = h12_1 + h12_2
-    h22 = h22_1 + h22_2
-    return h11, h12, h22
-end
-
-function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
-    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
-
-    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
-
-    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
-    @assert mass_ratio≤1 "mass_ratio must be <= 1"
-    @assert mass_ratio≥0 "mass_ratio must be non-negative"
-
-    orbit = soln2orbit(soln, model_params)
-    if mass_ratio > 0
-        m₂ = inv(T(1) + mass_ratio)
-        m₁ = mass_ratio * m₂
-
-        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
-        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
-    else
-        waveform = h_22_strain_one_body(dt, orbit)
-    end
-    return waveform
-end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",g,[(a(),i("svg",o,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=h("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=h(", ")),A("mjx-container",u,[(a(),i("svg",F,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=h(", and ")),A("mjx-container",c,[(a(),i("svg",b,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=h(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
-    χ, ϕ = u
-
-    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
-    denom = sqrt((p - 2)^2 - 4 * e^2)
-
-    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
-    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
-
-    return [χ̇, ϕ̇]
-end
-
-mass_ratio = 0.0         # test particle
-u0 = Float64[π, 0.0]     # initial conditions
-datasize = 250
-tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
-tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
-dt_data = tsteps[2] - tsteps[1]
-dt = 100.0
-const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
-soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
-waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
-
-    axislegend(ax, [[l, s]], ["Waveform Data"])
-
-    fig
-end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
-    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
-ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[3.189577f-5; -6.391393f-5; -3.3405635f-5; 4.4343187f-6; 0.000115499526; -0.0001018731; 0.00023788413; 0.00013548152; 6.836329f-5; -7.0168346f-5; -1.1637023f-5; 0.000107990956; 0.00010711853; -8.205676f-5; -3.5767007f-5; 0.0001381862; -4.684011f-5; 0.000164203; -3.8421153f-5; 5.4418593f-5; 9.1661295f-6; 2.394569f-5; -2.2449845f-5; 4.2435263f-6; 0.00014469436; -7.2968054f-5; 8.1639504f-5; -0.00010950737; 0.000111710426; -4.2097818f-5; -0.00023246874; -6.5241285f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-4.072868f-5 -5.0945444f-5 -0.0001259869 -2.931564f-5 0.000166747 7.638867f-5 -5.6004457f-5 -0.00014100932 -2.1586391f-5 -1.2109375f-5 -9.704629f-5 0.00017422912 0.00027027391 8.124638f-5 2.312072f-5 -1.3258273f-5 -6.2884916f-5 -5.023768f-5 -6.0434613f-5 -5.904619f-5 5.3575146f-5 8.3898245f-5 6.905054f-6 -9.50151f-7 -3.3204353f-5 -0.000321201 0.0001560693 6.604745f-5 -4.809239f-5 0.00012601794 -2.430164f-5 7.2888855f-5; 5.081071f-5 0.00017587443 -0.0001901438 -0.00016309925 -1.4872606f-5 0.00010889661 -9.051517f-5 -9.4335606f-5 0.00014208801 -4.8255f-5 -0.00015105841 3.318696f-5 -0.00010077599 0.00012599488 1.5010563f-5 8.972074f-5 -1.3046683f-5 -0.00015159532 0.00011351562 0.0001516642 0.00014350485 -0.00010061943 -0.00016779696 0.00016463436 6.636019f-5 0.00010201224 0.00011553895 -0.00013985121 9.045295f-5 3.6739464f-5 4.8656864f-5 0.000121753; 0.00022025035 -5.518999f-6 -9.600911f-5 0.00013558794 -5.8217807f-5 -7.169174f-5 8.9917674f-5 -0.00010023812 0.00017261699 -3.40887f-5 -0.00019824217 0.0001871318 -8.1081045f-5 3.311501f-5 -0.00010184848 -5.5567038f-5 -6.2686864f-5 -0.00011883248 0.0001333885 -2.6586393f-5 4.5853876f-6 -8.5089574f-5 -2.3479493f-5 9.173001f-6 7.11846f-5 0.000114722425 -1.7309303f-5 -8.865217f-5 5.1577266f-5 -9.788526f-5 5.233557f-5 -7.615518f-5; 0.00020963635 -1.8638202f-5 0.00011834405 -6.729295f-5 -0.00015167797 -7.0826565f-5 3.4370183f-5 -0.00016984323 9.748238f-5 6.600134f-5 -6.2498555f-5 -0.0001225902 -0.000192871 -0.00012371842 -0.00024446193 -3.0057638f-5 0.00016294271 9.313347f-5 -0.00013989632 -1.6167292f-5 -9.897745f-6 -3.6781636f-5 3.8724313f-5 0.000102805214 -6.287945f-5 3.452113f-5 3.6691803f-5 1.649534f-5 -1.0298279f-5 -3.4320224f-6 6.5838605f-5 -4.1367566f-6; 3.1436957f-5 0.00010077432 -0.0001992841 -8.427237f-6 2.0229614f-5 -1.8158896f-7 -4.3141594f-5 -6.0499427f-5 0.0001370642 -8.7448054f-5 -8.16965f-5 1.0385415f-5 -1.1151988f-5 2.0163849f-5 -8.430852f-5 0.00014713164 -1.2928104f-5 4.095135f-5 -0.00015691279 -0.00013806451 5.4008604f-5 -8.881955f-5 7.982801f-5 -6.0442053f-5 0.00010303316 1.1154679f-5 -6.417079f-5 -7.0635576f-7 1.0161936f-5 -3.7932576f-5 -9.273091f-5 6.587933f-5; 0.00015628956 -0.000100306606 0.0001417675 -1.8419723f-5 -0.00010336675 -3.745216f-6 1.3287905f-5 0.000110201225 -0.00012162571 0.00010194623 -0.00021031174 -0.00013408852 -0.00016633335 -0.00011588641 -0.00014500602 0.00023872862 -0.00019082129 0.00020116469 4.6417983f-5 4.0384627f-8 -4.1172363f-5 5.739644f-5 0.0002493159 1.0108878f-6 -6.098719f-6 1.7906605f-5 0.000208925 0.00012901663 5.7656558f-5 5.2600393f-5 -1.7499613f-5 -0.00023694857; 7.6452096f-5 -0.00013919755 5.0454488f-5 -0.00010899347 8.444795f-5 9.3733055f-5 4.9273887f-5 9.575293f-5 -0.00017926394 5.2648138f-5 -0.00015499145 6.0000508f-5 -0.00012347617 -1.0212578f-5 -3.7480448f-5 -2.8889195f-5 -2.5839225f-5 -0.00014959453 -4.838798f-5 -0.00010264639 -2.2373182f-5 0.000187367 -1.4591332f-5 7.587859f-5 6.715484f-5 -8.455441f-6 0.00010300705 9.579943f-5 6.2988795f-5 0.00013727028 5.8433718f-5 -7.581316f-5; 9.867598f-6 0.00013001259 -0.00015904017 0.00010726528 -7.944118f-5 -0.00010838457 -3.0477504f-5 -5.3470885f-5 -0.00017066523 5.8966172f-5 -4.431984f-5 0.00011518578 9.390664f-5 -3.4953282f-5 -2.5257921f-5 3.6932663f-5 5.5787146f-5 8.749019f-5 7.14501f-6 -4.0959952f-5 -6.223889f-5 -8.734941f-5 0.00012043471 -0.00015020274 5.2771924f-5 0.0002360519 -6.228447f-5 6.4803426f-5 0.00010642961 -5.2180483f-5 3.147583f-5 0.00012207228; 0.0001617969 -3.8236438f-5 7.3801225f-6 -6.36821f-5 5.370217f-5 -5.6450346f-5 2.389117f-5 1.3119574f-5 3.8583716f-5 3.039198f-5 0.00014524818 -9.020171f-5 -8.770791f-5 -7.468266f-5 0.00012475614 -1.5612637f-5 2.97265f-5 -5.011583f-5 -2.4989173f-5 6.159547f-5 3.4962607f-6 1.9588915f-6 -9.422292f-5 -9.7407676f-5 -9.5720214f-5 -1.895673f-7 -6.3542095f-5 -8.419821f-5 6.416182f-5 -0.00019523122 6.9585396f-5 -1.4270458f-5; -2.4227842f-5 8.622542f-5 0.00013249219 -0.00016822782 -9.379966f-5 -5.4564578f-5 -7.1605165f-5 -9.777959f-5 -1.5141534f-5 -2.2505346f-5 1.7262204f-5 0.00020515101 -0.00015698072 0.00014193162 -8.208326f-5 0.0001984161 2.1720332f-5 -7.3809155f-5 8.412848f-5 7.0492674f-6 0.00014390076 1.1922104f-5 -0.00014427994 -4.40556f-6 -9.2878225f-5 1.861306f-5 8.723424f-5 0.00021786611 3.6548522f-6 -4.011293f-5 2.7519702f-5 -1.2173145f-5; 2.1144622f-5 0.00014967582 -0.00022895339 0.00024803935 6.106838f-5 7.726797f-5 -9.1495414f-5 -0.00013571442 -1.2743402f-5 3.7127043f-5 5.741774f-6 5.7482393f-6 1.4281251f-5 1.6610835f-5 3.506546f-6 0.00017465782 -3.9547882f-5 -7.454391f-5 -7.768746f-5 -4.087552f-5 -9.2923765f-6 -7.646117f-5 -5.933055f-5 5.02955f-5 3.072864f-5 -5.4951495f-5 -9.214813f-5 4.7591155f-5 -1.3358998f-6 0.00012437423 -4.2005086f-5 -6.518599f-5; 5.2166448f-5 2.3049663f-6 -2.9499244f-5 6.810257f-5 4.4267512f-5 0.00014037451 -9.013224f-6 3.2751344f-5 -6.298081f-6 2.6862037f-5 -0.00011300167 -0.00017712849 0.00022498281 9.065984f-5 -1.6968424f-5 9.56821f-6 9.605599f-6 1.79725f-5 3.0613257f-5 -4.2208558f-5 4.7292133f-6 -4.070843f-6 -0.00016515724 4.4079825f-6 1.49199095f-5 -0.00012453321 1.52478315f-5 -4.9945927f-5 -0.00010899424 5.9470847f-5 4.765956f-5 -0.00019273692; -0.00014856191 4.997596f-5 0.00014711094 7.5645374f-5 -5.5391956f-5 -0.000105570536 8.8837296f-5 7.1964474f-5 -4.419326f-5 -7.311366f-5 4.8815167f-5 -0.0001266497 7.6390796f-5 -0.000102230515 0.00017258363 -2.6130465f-5 -4.5941284f-5 0.00018645074 0.00014535875 -1.6320302f-5 -0.00025578658 -2.491066f-5 -1.5766662f-5 4.3251843f-5 0.00015311797 -0.00011235082 -1.9891374f-6 -0.00019587998 -0.00021126727 8.496006f-5 7.781786f-5 6.961107f-5; 9.386831f-5 0.00015165155 5.2710024f-5 -9.78637f-5 6.203806f-5 0.00011791612 -8.990891f-5 -3.199821f-6 8.466118f-6 -7.408915f-5 -1.6125816f-5 -0.00015418157 0.00010376295 1.5213385f-5 0.000108165565 -5.207923f-5 1.2379622f-5 0.00018003177 -2.9956736f-5 -5.492753f-5 -8.3133375f-5 8.4283674f-5 -5.026297f-5 2.7448626f-5 6.964033f-5 -1.6386199f-5 -9.4f-6 -0.00011314431 0.00013795459 5.831961f-5 -2.2579597f-5 3.1594696f-5; 4.5288183f-5 0.000151896 0.00012069575 -0.0001195369 -7.6831355f-5 -2.757172f-5 -1.6826662f-5 -0.00013169323 6.2199084f-5 -7.3993283f-6 7.438026f-5 -0.00017041463 -7.112954f-5 -2.846346f-5 6.357693f-5 -6.5696972f-6 -4.935456f-5 -9.412396f-6 -5.3623813f-7 -1.5585098f-5 -2.8759412f-5 2.6243073f-5 0.00010650766 -8.971806f-6 -8.316659f-5 -8.4486215f-5 -0.0001286409 1.9484713f-5 -6.270333f-5 6.095372f-5 -0.00020690112 -2.4946847f-5; -9.633735f-5 2.9961959f-5 -8.814054f-5 0.00029132818 -8.873218f-5 -8.915458f-5 2.4145293f-5 -0.00016413987 -0.00013065261 -0.00016233047 -5.391122f-5 -1.16300425f-5 7.1261215f-6 -4.5942532f-5 0.00019776101 -9.504735f-5 -0.00012708372 -9.55583f-5 -4.6812784f-5 -0.00014748046 -4.5817545f-5 -0.00012706364 -3.134673f-5 -5.041207f-5 -9.92932f-5 0.00024251099 7.7417964f-5 -9.606522f-5 -3.9172697f-5 -7.114121f-5 0.00012291323 2.004699f-5; 2.6071615f-5 0.00023533189 4.6388315f-5 -4.4312666f-5 4.156144f-5 1.0031484f-5 0.00021525362 -5.070823f-6 8.614582f-5 -8.926076f-5 -7.542077f-5 7.453099f-6 -3.379711f-5 -0.0001935182 6.443749f-6 -6.77989f-5 0.00024368793 -0.00022429347 -8.256922f-5 0.00023213631 -6.02441f-5 3.6378533f-5 -5.4333955f-6 0.00017070145 -0.00035845037 0.000118600394 -0.00014000213 -2.6946222f-5 4.9569262f-5 7.846937f-5 -0.00016146098 0.00016950133; -7.044077f-5 7.863811f-5 -3.5487312f-5 -3.3633118f-5 0.00018373223 -0.00015216439 -3.533486f-5 3.2348104f-5 -7.659576f-5 0.00013982822 -6.3890446f-5 5.1099894f-5 -3.6681195f-5 9.415396f-5 0.00014207355 3.2957967f-5 -1.5705988f-5 -7.967105f-5 -2.180348f-5 -0.00013320909 -0.0001059336 8.3487f-5 2.282055f-5 5.110875f-5 -3.27335f-5 4.104298f-5 8.464971f-5 -1.7010881f-5 -1.4406375f-5 2.8203609f-5 5.213847f-5 0.0002157011; 0.00019730491 -0.00011324993 6.4773994f-5 0.000120778604 -5.4715092f-5 0.000103474944 -0.00018960478 4.083243f-5 -1.7024828f-5 -7.3320516f-5 -1.5217254f-5 -6.2785366f-5 6.3320535f-5 -4.9529168f-5 -1.9787989f-5 1.2121187f-5 7.928497f-5 -7.016558f-5 -0.00024311207 9.609716f-5 -2.9102543f-5 -3.4667796f-6 -2.930603f-5 5.678993f-5 8.58904f-5 -3.9016104f-5 0.00010542277 -4.1077605f-5 -9.56267f-5 -0.0001131797 -5.798505f-7 -6.970267f-5; -0.00013635393 5.3281066f-5 1.2604041f-5 7.2100185f-5 8.5174026f-5 -4.909403f-5 -2.3897335f-5 -1.1963698f-5 -7.487817f-5 0.00014533561 -5.2880594f-5 0.00010019395 -2.3352486f-5 0.00010543097 4.388293f-5 -5.3105534f-5 3.143798f-5 -6.4436004f-5 1.1926574f-5 -9.614401f-5 -0.00024020273 5.829732f-7 -0.0001988018 -3.7190214f-6 0.00012052626 2.0961197f-6 -6.0311537f-5 -6.3592706f-5 -0.00012088332 -5.7647132f-5 -1.2393728f-5 8.2915765f-5; 6.1345386f-6 -7.6362536f-5 5.3765398f-5 2.2559276f-5 -0.00020616876 -0.00017645433 0.000102468526 -7.835819f-5 -2.275776f-5 1.1462957f-5 -5.1330848f-5 -2.8339622f-5 3.2293367f-5 -5.2020674f-5 1.1928305f-5 -0.00014077095 -5.1057017f-5 -5.2451684f-5 4.570732f-5 -6.1287014f-6 -2.429942f-5 6.568433f-5 9.884206f-5 -0.00016328818 1.984889f-5 -7.699378f-5 2.0554815f-5 -0.00013284909 0.00014174149 4.0874344f-5 8.8765184f-5 0.00012290153; -7.284601f-5 0.00015113266 -8.370722f-5 -0.00010180767 -3.4999375f-5 -0.00018356094 0.00010763813 3.3040036f-5 6.449167f-5 -0.000103754785 0.00010147674 9.896282f-5 -8.177997f-5 1.5517104f-6 0.00017423871 -6.703495f-5 9.930151f-5 -0.00017044914 -4.4303477f-5 4.5368917f-5 -0.000130783 2.013616f-5 6.454598f-5 -0.00020466343 4.2728916f-5 -0.00015525491 9.641593f-6 -8.8821635f-6 0.00013840995 -7.628754f-5 5.2094652f-5 7.693837f-5; -1.936521f-6 -0.00012887 9.0301095f-5 0.00021892734 -5.0441475f-5 3.4005883f-5 0.0001176917 8.137417f-5 -6.481463f-5 3.448867f-5 8.3453546f-7 -0.00012861127 3.8067217f-5 0.00012599569 -0.000100411744 -8.965601f-5 4.6392575f-5 0.00017151685 2.4286144f-5 -4.66038f-5 5.022698f-6 0.00015643479 -4.898879f-5 -3.461263f-5 0.00012742943 6.658684f-5 -6.0342363f-6 -3.1366388f-5 -1.57381f-5 -7.21164f-5 0.00014184127 8.698141f-5; 0.00010477184 -0.00021518409 -0.00020540101 -0.00017861997 -8.4404535f-5 -3.70074f-5 3.9613344f-5 1.9685473f-5 -5.9209102f-5 -0.00015158781 6.52442f-5 6.160428f-5 -9.181076f-5 0.0001256964 0.00015453056 4.0800082f-6 2.2091914f-5 -3.218061f-5 -0.00012707147 -0.000215632 -1.9369683f-5 2.834695f-5 4.242915f-5 4.7958943f-6 -3.186571f-5 6.009155f-5 0.00021139008 4.160474f-5 -2.7288033f-6 4.625632f-5 -4.3435106f-5 7.146487f-5; 4.374089f-5 7.488287f-5 -9.8469856f-5 -0.00012004942 0.0001945894 2.170725f-5 1.7937835f-5 2.7179507f-5 -2.1116573f-5 -5.5853932f-5 3.3326544f-6 -9.3491435f-6 0.00010537235 6.931845f-5 -3.983922f-5 -4.7289675f-5 1.5653004f-5 -0.00022400402 5.7491754f-5 1.715078f-5 2.8108143f-6 5.3561467f-5 -5.169309f-5 -2.5652089f-5 -8.073286f-5 -2.4996185f-5 4.34875f-5 -0.00016427264 1.611109f-5 6.0825427f-5 3.671255f-5 -3.9133418f-5; 0.0002289462 9.411186f-5 -2.7011092f-5 -7.508443f-5 3.0149811f-5 -8.611271f-5 -1.721116f-5 0.00014790527 1.8963052f-5 0.0003108386 -0.00011547375 1.3582856f-5 -4.9441587f-5 -8.9482746f-5 8.650211f-5 -0.00014843841 3.330396f-5 7.7672965f-5 -2.6569027f-5 -5.260992f-5 -5.758783f-5 0.00012981516 8.676786f-5 -0.00012019002 0.00013454087 0.00012962654 -0.000145846 -2.2925251f-5 0.00014683808 0.000111876994 -6.814566f-5 -8.991099f-5; 0.00015247412 -5.2220308f-5 3.303562f-5 -4.0516537f-5 5.0913433f-5 4.4803448f-5 -9.616838f-5 1.784521f-5 7.0508744f-5 0.00018072716 -8.2390274f-5 -0.00016511144 6.84019f-5 -6.5895496f-5 0.00013130245 -0.00010432372 1.1481878f-5 -1.4197938f-5 -2.5276172f-6 1.1729372f-5 3.9101567f-5 5.2090098f-5 5.913563f-5 -2.8327304f-5 -9.641792f-5 0.00017448186 -5.872737f-5 1.536804f-5 7.327565f-5 0.00012235604 7.6064957f-6 1.5156055f-5; -0.000154704 1.4018213f-5 -4.0804447f-5 0.0001416553 4.0145318f-5 3.6607282f-6 8.820955f-5 3.786613f-5 -0.00020226216 4.6105897f-6 5.550528f-6 -0.00010234916 6.716201f-5 -6.043639f-5 -2.6334108f-5 -7.417747f-5 -7.517043f-5 1.1571982f-5 -5.7862057f-5 -3.0656676f-5 -3.4289274f-6 9.702859f-5 0.00014102783 -3.5850444f-5 1.851716f-5 8.838451f-5 -0.00016915894 -5.8839592f-5 0.00017786834 4.4703054f-5 -8.068646f-5 -0.00016312697; 4.442865f-5 -0.00012006202 -2.6558766f-5 5.672547f-5 5.6829463f-6 -6.2722575f-5 -5.5414985f-5 -6.616521f-5 -0.0002716475 -3.1431762f-5 2.5350297f-5 -1.0963497f-5 5.8565315f-6 1.1280099f-5 3.505499f-5 -2.5387439f-5 -4.471935f-5 1.5624903f-5 4.9636583f-5 0.00013870077 -0.00010204634 -1.8144423f-5 6.716244f-5 7.007615f-5 8.5767846f-5 2.7059134f-5 -0.0001835215 -5.475444f-5 -2.0878162f-5 -0.00017755636 -1.3773209f-5 0.00010980106; 5.2602372f-5 -6.791942f-6 -6.0015762f-5 -8.829484f-5 -7.536287f-5 -3.9357703f-5 -0.0002533797 -0.00019349939 -2.6312156f-5 -0.00014407777 5.654903f-6 -8.596633f-5 0.00010207755 -0.00014197214 -0.00021095501 -8.949662f-5 -8.78758f-5 8.230468f-5 5.0060156f-5 1.3623026f-5 8.192343f-5 9.648081f-5 2.901343f-5 -2.8943532f-5 0.00014445429 -0.00014949995 2.8125742f-5 9.118809f-6 3.9546387f-5 0.00011677827 0.00013162084 -9.251267f-5; 0.000111023226 8.540807f-5 -9.632401f-5 -2.0421841f-5 -0.00012313662 -5.2018618f-6 -5.1790317f-5 -0.00016129599 3.4691588f-5 0.00012465993 -5.4743723f-5 -7.375791f-5 -4.2607066f-5 5.131311f-5 0.000105108054 9.257429f-6 7.963778f-5 -0.00022747654 -6.015767f-5 6.143162f-5 0.0001239677 -7.84972f-5 -9.273924f-5 -9.1272304f-5 -5.0631555f-5 1.960298f-5 6.176991f-5 1.42621575f-5 -0.00014048498 -9.409552f-5 -6.69471f-5 -0.00012498979; -9.983786f-5 -9.002071f-5 6.0303675f-5 -4.6285055f-5 -4.9538403f-6 -2.9290824f-5 -5.955555f-5 -0.00017678842 5.0389506f-5 9.9634635f-6 -0.00013858659 7.5097923f-6 -7.146621f-5 -8.4323365f-5 -0.00018398082 4.5715612f-7 0.00019981673 7.600076f-5 -0.000113493494 6.6738976f-5 4.4671888f-5 5.1766674f-6 4.957016f-5 -5.165908f-5 -5.7047462f-5 -0.0001406546 2.3864255f-5 -3.3130655f-5 -3.6680107f-5 1.23412165f-5 0.000109933746 4.6290555f-5], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-9.012873f-5 7.417051f-5 0.00025161545 4.9274546f-5 0.00010122614 -5.0163922f-5 -0.00012842058 -0.0001206786 -2.989958f-5 -8.010971f-5 1.5434154f-5 -0.00016001904 -1.2780457f-5 -2.7766466f-5 7.45467f-5 2.1860842f-5 -0.00012804814 -7.033757f-5 -0.00016812379 -7.759783f-5 3.6243237f-6 9.2822374f-5 -2.3109782f-5 0.00027652478 -4.066539f-5 0.0001177831 2.4505709f-5 -0.00012254887 7.928667f-5 0.00022943318 1.2903607f-5 1.035586f-6; 5.3552827f-5 0.00012438484 0.00010002195 -4.2746266f-5 -8.1811064f-5 -6.5830077f-6 -9.607851f-5 -1.5115101f-5 1.7638582f-5 -0.00012033503 2.8257547f-5 -5.0352584f-5 -1.6083617f-5 0.00010446825 -0.0002548531 4.9123146f-6 -0.00016278515 1.3834254f-5 -0.00015796158 0.00020403395 -3.1378113f-5 -7.720094f-6 -7.166389f-5 -1.7671651f-5 -5.650439f-5 9.3580245f-5 -7.139312f-5 7.729014f-5 -0.00015755462 -2.9734461f-5 -6.472176f-5 -4.8089343f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
-
-const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
-    Chain(
-        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
-        layer_2 = Dense(1 => 32, cos),  # 64 parameters
-        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
-        layer_4 = Dense(32 => 2),       # 66 parameters
-    ),
-)         # Total: 1_186 parameters,
-          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",m,[(a(),i("svg",V,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",B,[(a(),i("svg",z,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=h("where, ")),A("mjx-container",T,[(a(),i("svg",q,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=h(", ")),A("mjx-container",K,[(a(),i("svg",D,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=h(", and ")),A("mjx-container",P,[(a(),i("svg",U,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=h(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
-    χ, ϕ = u
-    p, M, e = ode_model_params
-
-    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
-    # it, however, in general, we should use \`st\` to store the state of the neural network.
-    y = 1 .+ nn_model([first(u)], nn_params)
-
-    numer = (1 + e * cos(χ))^2
-    denom = M * (p^(3 / 2))
-
-    χ̇ = (numer / denom) * y[1]
-    ϕ̇ = (numer / denom) * y[2]
-
-    return [χ̇, ϕ̇]
-end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
-waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    axislegend(ax, [[l1, s1], [l2, s2]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
-
-    fig
-end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
-
-function loss(θ)
-    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
-    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
-    return mseloss(pred_waveform, waveform)
-end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0007080899236081684
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
-
-function callback(θ, l)
-    push!(losses, l)
-    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
-    return false
-end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
-optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
-optprob = Optimization.OptimizationProblem(optf, params)
-res = Optimization.solve(
-    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
-    callback, maxiters=1000)
    retcode: Success
-u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [3.189576818839681e-5; -6.391393253576819e-5; -3.340563489471322e-5; 4.434318725546035e-6; 0.00011549952614571395; -0.00010187309817400318; 0.0002378841309108547; 0.00013548151764520408; 6.836328975620413e-5; -7.016834570084262e-5; -1.1637022907966216e-5; 0.00010799095616670282; 0.00010711853246899926; -8.205675840138618e-5; -3.5767006920644584e-5; 0.00013818619481746985; -4.6840108552708576e-5; 0.0001642029965294685; -3.842115256698744e-5; 5.441859320852504e-5; 9.166129530056265e-6; 2.3945689463266466e-5; -2.244984534627751e-5; 4.243526291244579e-6; 0.00014469436428041194; -7.296805415528246e-5; 8.163950406014773e-5; -0.00010950736759688676; 0.00011171042569901284; -4.209781764071136e-5; -0.0002324687375221788; -6.524128548331127e-5;;], bias = [9.824179453448208e-18, -3.283579809077081e-17, -2.4121052656024447e-17, -2.820816984579966e-18, 1.7800006399111622e-16, -1.2252003989273822e-16, 1.3596170846996523e-16, 3.7697396587690263e-16, 3.469541374929302e-17, -1.2239306697871398e-16, 4.732456804452487e-18, 6.710255596708555e-17, 3.620084831552526e-17, -7.37712055415936e-17, -6.498955501547213e-18, 7.565232283015596e-17, -7.142283934572076e-18, 1.969496943298209e-16, -4.3998375610902485e-17, 1.4517763361935025e-17, 2.2528160954460315e-18, 2.6719434458280483e-17, 5.412172157702465e-18, 3.58740882046328e-18, 3.1671129438060196e-17, -8.253106300035819e-17, 2.757169253238071e-18, -8.791378034823515e-17, 1.3521537011914334e-16, -4.6599032319848886e-17, 4.036589722165776e-17, -4.872073037560773e-17]), layer_3 = (weight = [-4.072823511703495e-5 -5.09450001936559e-5 -0.0001259864585244711 -2.9315197107257145e-5 0.00016674744975718865 7.63891147562606e-5 -5.600401354057221e-5 -0.00014100888082791929 -2.158594733371077e-5 -1.2108931596208436e-5 -9.704584962489625e-5 0.0001742295643544126 0.0002702743585450876 8.124682054587932e-5 2.3121163312703365e-5 -1.3257828950352319e-5 -6.28844719267135e-5 -5.0237235054718365e-5 -6.0434169508857686e-5 -5.904574581106778e-5 5.3575589763591236e-5 8.389868881752165e-5 6.905497602952288e-6 -9.497072391919446e-7 -3.320390905171548e-5 -0.0003212005468321654 0.0001560697491277318 6.604789078667043e-5 -4.8091948056004725e-5 0.00012601838526281895 -2.4301196472136584e-5 7.288929910830549e-5; 5.081267089944231e-5 0.00017587639239744914 -0.0001901418340891797 -0.00016309728881215715 -1.4870646289379445e-5 0.00010889856866223033 -9.05132087680134e-5 -9.43336467641714e-5 0.000142089973898955 -4.825303904700446e-5 -0.0001510564520755974 3.31889191259365e-5 -0.00010077403340593758 0.00012599683622492008 1.5012522901353627e-5 8.972270233451054e-5 -1.3044723807714801e-5 -0.00015159335953988042 0.00011351757938636337 0.0001516661527532875 0.0001435068065732527 -0.00010061746934998852 -0.00016779499629383977 0.00016463631845207847 6.636214902053133e-5 0.00010201420308684551 0.0001155409067835426 -0.0001398492509067325 9.045491105306053e-5 3.674142362096196e-5 4.865882373656356e-5 0.00012175496021181036; 0.00022024982202853847 -5.519529771869915e-6 -9.600964373387555e-5 0.00013558740496225476 -5.821833807569876e-5 -7.169227288980036e-5 8.991714306465219e-5 -0.00010023864857796439 0.00017261645583491674 -3.408923263489361e-5 -0.00019824269808307467 0.00018713126367945243 -8.108157604876883e-5 3.3114480480411414e-5 -0.00010184901444884173 -5.5567568473656066e-5 -6.268739495137848e-5 -0.00011883301309719392 0.00013338797028697872 -2.658692361731209e-5 4.584856775156783e-6 -8.509010482523759e-5 -2.3480024232437663e-5 9.172469923189455e-6 7.118406968945824e-5 0.000114721893929503 -1.7309833914647513e-5 -8.865269743548675e-5 5.1576735437242696e-5 -9.788579306043833e-5 5.233503937777888e-5 -7.615570908822224e-5; 0.00020963441425483624 -1.864013388039402e-5 0.00011834211550667836 -6.72948844519828e-5 -0.00015167990563421714 -7.08284969230501e-5 3.436825141329051e-5 -0.0001698451632771909 9.74804461795482e-5 6.599940757515103e-5 -6.250048676956555e-5 -0.00012259212593998138 -0.0001928729288418158 -0.00012372035047955503 -0.0002444638630047816 -3.005956927589448e-5 0.0001629407816428321 9.313154082994632e-5 -0.0001398982530597972 -1.6169224131586253e-5 -9.899676974598491e-6 -3.678356728850944e-5 3.872238090043803e-5 0.00010280328211156535 -6.28813831506549e-5 3.451919842992198e-5 3.668987124475024e-5 1.649340816148732e-5 -1.0300211165293852e-5 -3.4339540841721874e-6 6.583667350718321e-5 -4.138688332017845e-6; 3.143527677819532e-5 0.00010077263899800934 -0.0001992857793629967 -8.428917371946206e-6 2.0227933728156163e-5 -1.8326939597203966e-7 -4.314327421523914e-5 -6.050110792785467e-5 0.00013706251806068834 -8.744973425565604e-5 -8.169817707313807e-5 1.0383734970663711e-5 -1.1153668001507944e-5 2.0162168166267354e-5 -8.43102022008836e-5 0.000147129955021222 -1.2929784734312002e-5 4.0949668200481244e-5 -0.00015691447302744298 -0.00013806619213939137 5.400692347391857e-5 -8.882123043805916e-5 7.982632845594918e-5 -6.044373336409806e-5 0.00010303148186721462 1.115299832234561e-5 -6.417246882848665e-5 -7.080362002041241e-7 1.0160255751533918e-5 -3.7934256088207734e-5 -9.273258880055931e-5 6.58776483214611e-5; 0.000156290270588567 -0.0001003058989041292 0.00014176820133833353 -1.841901592307562e-5 -0.00010336604298537744 -3.74450853961715e-6 1.328861237231886e-5 0.00011020193216643964 -0.00012162500041050782 0.00010194693624761005 -0.00021031103658934895 -0.00013408781358486153 -0.00016633264583980773 -0.00011588570321656757 -0.00014500531522536515 0.0002387293296309346 -0.0001908205796822708 0.00020116539401022254 4.6418690381312165e-5 4.109205071819074e-8 -4.1171655702431014e-5 5.739714635673128e-5 0.0002493166046591676 1.0115952094448078e-6 -6.098011617376218e-6 1.7907312049270067e-5 0.00020892571212278687 0.00012901733300203338 5.765726547941438e-5 5.2601100669914204e-5 -1.7498905599156158e-5 -0.00023694786630550335; 7.645258701715071e-5 -0.00013919706143595875 5.0454979281233715e-5 -0.00010899297522072281 8.444844242959467e-5 9.373354659957113e-5 4.927437876076855e-5 9.575341791688568e-5 -0.00017926344626274568 5.26486295702061e-5 -0.00015499095352523564 6.000099929125259e-5 -0.0001234756815402654 -1.0212086273065653e-5 -3.7479956155117537e-5 -2.8888703674137614e-5 -2.5838733200622807e-5 -0.00014959404106874985 -4.838748973369222e-5 -0.00010264589545939696 -2.2372690728922117e-5 0.00018736748861457612 -1.4590840329298094e-5 7.58790814860857e-5 6.715533199795584e-5 -8.454949785197743e-6 0.00010300754038227083 9.579991856199895e-5 6.29892860151009e-5 0.00013727077421109564 5.843420913052272e-5 -7.581266581936409e-5; 9.868717612555944e-6 0.00013001370713595114 -0.00015903904680613504 0.0001072664006335285 -7.944006323101522e-5 -0.00010838345154639748 -3.0476384594093793e-5 -5.3469765802004945e-5 -0.0001706641062207913 5.896729159416418e-5 -4.431872115106359e-5 0.00011518689895296008 9.390775700579875e-5 -3.4952162635599276e-5 -2.5256801776556165e-5 3.693378199953781e-5 5.578826564146093e-5 8.749130826457307e-5 7.1461293066412356e-6 -4.0958833081965815e-5 -6.223777130231299e-5 -8.734828700995126e-5 0.000120435832983497 -0.00015020162503220176 5.2773043322517866e-5 0.00023605301863994222 -6.228334790081066e-5 6.480454500546205e-5 0.00010643072779770449 -5.217936380518698e-5 3.147694839410321e-5 0.0001220733963839318; 0.00016179558928054307 -3.8237744104818825e-5 7.378816223378737e-6 -6.36834049720832e-5 5.370086340629576e-5 -5.645165243838028e-5 2.38898644391641e-5 1.3118267576678817e-5 3.85824093762756e-5 3.0390674270919503e-5 0.00014524687479645457 -9.020301901106715e-5 -8.770921364263631e-5 -7.468396894470992e-5 0.0001247548386680384 -1.5613943344910157e-5 2.9725193359652275e-5 -5.011713461786774e-5 -2.4990479017262914e-5 6.159416419616535e-5 3.494954386793892e-6 1.957585242805049e-6 -9.42242297816603e-5 -9.740898191300713e-5 -9.572151999524389e-5 -1.9087357604430672e-7 -6.354340099566823e-5 -8.419951724424006e-5 6.416051086201375e-5 -0.00019523252218566324 6.95840897031777e-5 -1.4271764724227435e-5; -2.422680986988012e-5 8.622644858782885e-5 0.00013249322259448345 -0.0001682267913164207 -9.379862709088791e-5 -5.4563545887372177e-5 -7.160413329608878e-5 -9.77785613539327e-5 -1.5140502142324223e-5 -2.2504314216214893e-5 1.726323611742261e-5 0.00020515204127527292 -0.000156979688795996 0.00014193265710141196 -8.208222530434039e-5 0.00019841713842119285 2.172136450476319e-5 -7.380812272082148e-5 8.412951032746447e-5 7.050299558748259e-6 0.00014390179316628636 1.1923136156973306e-5 -0.00014427890815191073 -4.404527913585472e-6 -9.287719254267265e-5 1.8614092227059548e-5 8.723527466294975e-5 0.00021786714100393905 3.655884342282818e-6 -4.0111896992904666e-5 2.7520733823738473e-5 -1.2172112434066336e-5; 2.1144479453016605e-5 0.00014967567678637593 -0.00022895353047546222 0.00024803920225336914 6.106823550168255e-5 7.726782771156918e-5 -9.149555715169725e-5 -0.00013571456381954395 -1.2743545065074921e-5 3.7126900699024794e-5 5.7416314319765904e-6 5.748096575064684e-6 1.4281108653324745e-5 1.661069202042451e-5 3.50640313796474e-6 0.00017465767725467546 -3.95480246680904e-5 -7.454405293004695e-5 -7.768760131342562e-5 -4.0875661377992795e-5 -9.29251926039576e-6 -7.646131141713119e-5 -5.933069130891232e-5 5.0295358070672964e-5 3.072849633268878e-5 -5.4951638028921765e-5 -9.214827603331037e-5 4.75910119547398e-5 -1.3360425811123943e-6 0.0001243740871558316 -4.200522833174072e-5 -6.518613097626927e-5; 5.216586965387029e-5 2.3043880969255088e-6 -2.9499822396721737e-5 6.810199216258969e-5 4.4266933634826405e-5 0.00014037393362512332 -9.01380257574673e-6 3.275076620182626e-5 -6.2986593934707995e-6 2.6861458770761945e-5 -0.000113002246273502 -0.00017712906605313057 0.00022498223211867276 9.065926520404789e-5 -1.6969002665539284e-5 9.567631657302037e-6 9.605020984488702e-6 1.7971921131432287e-5 3.0612678763170755e-5 -4.2209135920094944e-5 4.7286350831903895e-6 -4.0714210104147525e-6 -0.00016515781657519486 4.407404283008284e-6 1.4919331313177951e-5 -0.0001245337878049409 1.5247253265878265e-5 -4.9946505662337705e-5 -0.00010899481616663255 5.947026898636771e-5 4.765898143658767e-5 -0.00019273749398282397; -0.00014856220460460773 4.997566847050341e-5 0.0001471106521323205 7.564508322429582e-5 -5.5392246768680794e-5 -0.00010557082660925735 8.883700555925537e-5 7.19641835432398e-5 -4.4193551194260185e-5 -7.311395112156149e-5 4.881487674465144e-5 -0.0001266499888613928 7.639050508187e-5 -0.00010223080554242442 0.00017258334318211984 -2.6130755405394208e-5 -4.5941574545216334e-5 0.00018645044527984222 0.00014535845601967287 -1.632059266812866e-5 -0.0002557868714673563 -2.4910950206325143e-5 -1.5766952320339846e-5 4.325155222358853e-5 0.00015311768273859223 -0.00011235110961097501 -1.9894279474126975e-6 -0.000195880274450976 -0.00021126755692494242 8.495977143793123e-5 7.781756775257433e-5 6.961078232888846e-5; 9.387014516933704e-5 0.000151653384923252 5.2711861021368445e-5 -9.786186627506336e-5 6.203989697923951e-5 0.00011791795633571468 -8.99070690914685e-5 -3.1979838107002867e-6 8.467955117070908e-6 -7.408731097284129e-5 -1.6123978535044658e-5 -0.0001541797354414483 0.00010376478541247726 1.5215222551617784e-5 0.0001081674018812073 -5.207739367079272e-5 1.2381458977603318e-5 0.00018003360684332612 -2.995489911453944e-5 -5.492569097013449e-5 -8.313153729003915e-5 8.428551165225825e-5 -5.026113275134092e-5 2.745046378068568e-5 6.964216674049922e-5 -1.6384361411197174e-5 -9.398162442536506e-6 -0.00011314247593445556 0.00013795642381837713 5.8321446080941926e-5 -2.257776021481562e-5 3.1596533621909674e-5; 4.528550362491412e-5 0.00015189332571507132 0.00012069307359096032 -0.00011953957636680179 -7.683403436631394e-5 -2.7574398877464584e-5 -1.6829341070353303e-5 -0.00013169591165451328 6.219640436510303e-5 -7.402007845222216e-6 7.437757977212133e-5 -0.00017041730661249363 -7.11322229060031e-5 -2.846614024266207e-5 6.357425245849798e-5 -6.572376782694359e-6 -4.935723899935839e-5 -9.415075687402303e-6 -5.389176814917797e-7 -1.558777712008384e-5 -2.876209163761416e-5 2.6240393531484048e-5 0.00010650497807117498 -8.974485531067046e-6 -8.316926886863775e-5 -8.448889474922055e-5 -0.0001286435746757973 1.948203299006832e-5 -6.270600915259254e-5 6.095104054592046e-5 -0.00020690380387774357 -2.4949526227566494e-5; -9.634147099579145e-5 2.9957840891336297e-5 -8.814466090999923e-5 0.00029132406089327705 -8.873629812742245e-5 -8.915869657076024e-5 2.4141174802176025e-5 -0.00016414398814676247 -0.00013065672792155194 -0.0001623345884553702 -5.3915336681637446e-5 -1.1634160361519357e-5 7.1220037199041655e-6 -4.594664969491395e-5 0.00019775689034130065 -9.505146553864519e-5 -0.0001270878416079679 -9.55624196620906e-5 -4.681690151930187e-5 -0.00014748457724355295 -4.5821663295026476e-5 -0.00012706775996495492 -3.135084952820556e-5 -5.041618678380332e-5 -9.929731422461133e-5 0.00024250686729200373 7.741384652316938e-5 -9.606933562909097e-5 -3.917681496832297e-5 -7.1145324542991e-5 0.00012290911436426698 2.004287162261593e-5; 2.6072455727442996e-5 0.00023533272563963474 4.638915582927304e-5 -4.43118258435799e-5 4.156228191004157e-5 1.0032324314120804e-5 0.00021525445866880696 -5.069982546962557e-6 8.614665806544275e-5 -8.925991620617663e-5 -7.541993160248347e-5 7.45393957640346e-6 -3.3796269929557296e-5 -0.0001935173529664127 6.444589632336342e-6 -6.779806041141176e-5 0.00024368877005781378 -0.00022429263095819746 -8.25683780594787e-5 0.00023213715415931616 -6.024325899761714e-5 3.637937355688216e-5 -5.432554883556168e-6 0.0001707022859018242 -0.00035844953141623366 0.00011860123435053287 -0.00014000128773023168 -2.6945381035152222e-5 4.957010263478429e-5 7.847020991630263e-5 -0.00016146013855451884 0.00016950217490102439; -7.043909705477776e-5 7.863978838779784e-5 -3.54856367828419e-5 -3.363144265837922e-5 0.00018373390934195612 -0.00015216271118495206 -3.53331836429825e-5 3.234977861049552e-5 -7.659408425913405e-5 0.00013982989343114906 -6.388877069345636e-5 5.1101569386649514e-5 -3.6679519563928514e-5 9.415563703280549e-5 0.0001420752248470576 3.295964210175835e-5 -1.5704312556200466e-5 -7.966937688414516e-5 -2.1801805143487045e-5 -0.0001332074155123078 -0.00010593192406465257 8.348867571678961e-5 2.2822224980909573e-5 5.1110424227072386e-5 -3.2731825069079087e-5 4.104465346083729e-5 8.465138829544799e-5 -1.7009206441595775e-5 -1.4404700106549434e-5 2.8205284014101566e-5 5.2140146852432286e-5 0.0002157027790765166; 0.00019730367283343157 -0.00011325116567307211 6.477275972726577e-5 0.0001207773698145222 -5.471632677356395e-5 0.00010347370924434819 -0.0001896060107507815 4.083119573195054e-5 -1.7026062338780957e-5 -7.332175074837196e-5 -1.5218488912733589e-5 -6.278660068048847e-5 6.331929988234545e-5 -4.953040253597263e-5 -1.9789223658090315e-5 1.2119952301216232e-5 7.928373314213804e-5 -7.016681549131931e-5 -0.00024311330118210217 9.609592460335126e-5 -2.910377726651017e-5 -3.468014271430115e-6 -2.930726397310587e-5 5.678869500413314e-5 8.588916760643641e-5 -3.9017338637430295e-5 0.00010542153402936693 -4.107883934623298e-5 -9.562793776851796e-5 -0.00011318093813018094 -5.810851614175347e-7 -6.970390451599896e-5; -0.00013635585733932896 5.3279143085132605e-5 1.2602118058605339e-5 7.209826193506313e-5 8.517210275390521e-5 -4.90959525086064e-5 -2.389925853996703e-5 -1.1965621215445414e-5 -7.48800909240845e-5 0.00014533368570486814 -5.28825172841227e-5 0.00010019202940388812 -2.3354409367994097e-5 0.00010542904541584576 4.388100691361851e-5 -5.31074571517319e-5 3.143605621520986e-5 -6.44379275944988e-5 1.1924650916664847e-5 -9.614593242072826e-5 -0.0002402046544920797 5.810499283526083e-7 -0.0001988037280716143 -3.7209446508843805e-6 0.00012052433643480654 2.094196443108954e-6 -6.0313460071441674e-5 -6.359462955510926e-5 -0.00012088524332279989 -5.764905532483414e-5 -1.2395651228799042e-5 8.291384196131248e-5; 6.13265146020744e-6 -7.63644228849465e-5 5.3763510631684424e-5 2.2557388628770116e-5 -0.00020617064265970482 -0.00017645621383999068 0.00010246663904548159 -7.836007981531355e-5 -2.27596473545436e-5 1.1461070325456786e-5 -5.13327349021029e-5 -2.834150919661371e-5 3.229147941019859e-5 -5.202256116754194e-5 1.192641783709016e-5 -0.00014077283677615214 -5.1058904237291846e-5 -5.2453570706699625e-5 4.5705431229722175e-5 -6.1305885632492694e-6 -2.430130635767008e-5 6.568244597437593e-5 9.884017080330087e-5 -0.00016329006294691238 1.9847003468029552e-5 -7.699567041563707e-5 2.0552927789685223e-5 -0.00013285097785264679 0.0001417396025222873 4.0872456420034774e-5 8.876329692578255e-5 0.000122899645329742; -7.284701142105474e-5 0.00015113165126769704 -8.370822625345749e-5 -0.00010180867694196524 -3.5000379381177776e-5 -0.00018356194787536639 0.0001076371262514288 3.3039032035799645e-5 6.449066849547358e-5 -0.00010375578868313764 0.00010147573620435553 9.896181281312519e-5 -8.178097060060917e-5 1.5507062187873954e-6 0.00017423770613624525 -6.703595145715941e-5 9.930050864006232e-5 -0.0001704501483856543 -4.4304481076488265e-5 4.536791302036357e-5 -0.00013078399943531718 2.013515576783818e-5 6.454497624310447e-5 -0.00020466443684761645 4.2727911835596774e-5 -0.00015525591161911432 9.640589172541938e-6 -8.883167635009892e-6 0.00013840894088736912 -7.628854664852085e-5 5.2093648167863977e-5 7.693736542371006e-5; -1.9334187523694303e-6 -0.0001288668983303012 9.030419719275371e-5 0.0002189304385414387 -5.043837259072836e-5 3.400898535837695e-5 0.00011769480156576498 8.137727386419124e-5 -6.481152845444233e-5 3.449177061193522e-5 8.376376062284453e-7 -0.00012860816527754854 3.807031939722761e-5 0.00012599879378322225 -0.0001004086417647983 -8.965290648994648e-5 4.6395677464177725e-5 0.0001715199494814266 2.428924640335857e-5 -4.660069785094722e-5 5.025800283338583e-6 0.0001564378934294019 -4.8985687635276715e-5 -3.4609526801058776e-5 0.00012743253578299793 6.658994502723074e-5 -6.0311341709135985e-6 -3.1363285551926876e-5 -1.5734997895927483e-5 -7.211330116623792e-5 0.00014184437427534283 8.698451242744888e-5; 0.00010477013827213785 -0.00021518579044466895 -0.00020540271245962234 -0.00017862167326355638 -8.44062367160201e-5 -3.700910267623787e-5 3.961164227252867e-5 1.968377116456888e-5 -5.9210803307071616e-5 -0.00015158951166639118 6.524250172689278e-5 6.160258023912445e-5 -9.181246307532888e-5 0.00012569470097689044 0.00015452885534060902 4.078306823363043e-6 2.209021229092797e-5 -3.2182312430513194e-5 -0.00012707317248540205 -0.00021563369839539874 -1.9371384702104172e-5 2.834524936944457e-5 4.2427448783123144e-5 4.7941928404855354e-6 -3.1867412622953456e-5 6.008984680549249e-5 0.00021138838317157782 4.1603039129696805e-5 -2.730504753415649e-6 4.625461704009955e-5 -4.343680732386287e-5 7.146316900063252e-5; 4.374030813774188e-5 7.488228711731884e-5 -9.847043711492695e-5 -0.0001200500033519782 0.00019458882288741277 2.170666939487488e-5 1.7937254051470556e-5 2.7178926211450773e-5 -2.111715421055745e-5 -5.5854513084383856e-5 3.332073583762591e-6 -9.349724314275018e-6 0.0001053717727699719 6.931786756463552e-5 -3.983979931949539e-5 -4.729025622513638e-5 1.5652423280655655e-5 -0.00022400460020439108 5.749117317459649e-5 1.7150199164418895e-5 2.8102334908473548e-6 5.35608864083336e-5 -5.1693669411222534e-5 -2.5652669683084067e-5 -8.073344233711706e-5 -2.4996765751026667e-5 4.348691927585022e-5 -0.00016427322279927948 1.6110508477099578e-5 6.082484596841146e-5 3.671196955107326e-5 -3.913399868910976e-5; 0.0002289486765587484 9.411434079108195e-5 -2.7008614742707937e-5 -7.508194899831204e-5 3.0152288709595648e-5 -8.611023436946012e-5 -1.720868200189139e-5 0.00014790775097958878 1.8965529343647317e-5 0.00031084107835364813 -0.00011547126992784686 1.3585333399770132e-5 -4.943910967971433e-5 -8.948026876144895e-5 8.650458406943489e-5 -0.0001484359344974525 3.3306438163226055e-5 7.767544233524609e-5 -2.6566549385943286e-5 -5.2607443612697635e-5 -5.758535186007862e-5 0.00012981763942686558 8.677033842129356e-5 -0.00012018754565336047 0.00013454334479343072 0.00012962901750167164 -0.0001458435253514241 -2.2922773459631004e-5 0.00014684055717249084 0.00011187947169698394 -6.814318108788655e-5 -8.99085097987449e-5; 0.0001524763183963476 -5.221811348996438e-5 3.3037814352846633e-5 -4.0514342765836275e-5 5.091562801430886e-5 4.480564260780046e-5 -9.616618896214645e-5 1.7847404739992356e-5 7.051093874657169e-5 0.00018072935657733683 -8.238807910196516e-5 -0.00016510924884892 6.840409790788863e-5 -6.589330129167569e-5 0.00013130464961910515 -0.00010432152608951083 1.148407230835695e-5 -1.4195743426863102e-5 -2.5254225536087655e-6 1.1731566735086957e-5 3.910376202589805e-5 5.209229224170496e-5 5.913782391622606e-5 -2.8325109235756322e-5 -9.641572520445694e-5 0.00017448405291111741 -5.872517700731323e-5 1.5370234029661434e-5 7.327784718404539e-5 0.0001223582340001114 7.608690372961008e-6 1.515824991407245e-5; -0.00015470543068577935 1.4016783171330845e-5 -4.0805876705333804e-5 0.00014165386899630094 4.014388840086942e-5 3.659298573344575e-6 8.820811861453015e-5 3.786470195418644e-5 -0.0002022635925814998 4.609160060584738e-6 5.549098278507849e-6 -0.00010235058645694412 6.716057767416446e-5 -6.0437818246196376e-5 -2.6335537694747692e-5 -7.417890075803277e-5 -7.5171857969606e-5 1.1570552561265787e-5 -5.7863486216872975e-5 -3.065810589678892e-5 -3.4303570092641735e-6 9.702716131950309e-5 0.00014102640496356923 -3.585187361669241e-5 1.851572968089565e-5 8.838308356725419e-5 -0.0001691603719510617 -5.884102203626996e-5 0.0001778669066706221 4.470162464794341e-5 -8.068789240413099e-5 -0.0001631283993620916; 4.4426369299592315e-5 -0.00012006429676623617 -2.6561045501900908e-5 5.672318877693364e-5 5.680666731479961e-6 -6.272485404699174e-5 -5.5417264207046406e-5 -6.616748794405194e-5 -0.0002716497787107297 -3.143404171644247e-5 2.5348017174812176e-5 -1.0965776141546363e-5 5.8542519807642565e-6 1.127781945322305e-5 3.505270944949432e-5 -2.538971818941064e-5 -4.4721628342109925e-5 1.5622622981749016e-5 4.963430330896957e-5 0.0001386984855569372 -0.00010204861825989456 -1.8146702473339157e-5 6.716015707276895e-5 7.007387250702925e-5 8.576556649567345e-5 2.7056854037051348e-5 -0.0001835237729894497 -5.475672003305463e-5 -2.0880441639035963e-5 -0.00017755863734712595 -1.3775488128143772e-5 0.00010979878209107037; 5.259896126273798e-5 -6.795352858422536e-6 -6.001917289057025e-5 -8.829824808588582e-5 -7.536628116455649e-5 -3.93611139613029e-5 -0.0002533831052302163 -0.00019350280352469694 -2.6315567522647815e-5 -0.00014408118157250598 5.651491862698763e-6 -8.596973792252105e-5 0.00010207414153895346 -0.00014197554854278972 -0.0002109584224696663 -8.950003255691461e-5 -8.787921113496737e-5 8.230126833709747e-5 5.005674531032161e-5 1.3619614507954043e-5 8.192001543404211e-5 9.647740218253537e-5 2.9010019277228988e-5 -2.894694305967406e-5 0.0001444508757388977 -0.00014950335615383206 2.812233062043259e-5 9.115398386105128e-6 3.9542975643902326e-5 0.00011677485737647723 0.00013161742528371577 -9.25160843350352e-5; 0.0001110199960913336 8.540484179145182e-5 -9.63272390271709e-5 -2.0425071290386567e-5 -0.00012313984860331249 -5.205091731042327e-6 -5.17935471924539e-5 -0.00016129921949133171 3.4688357680417196e-5 0.00012465669771524689 -5.474695297555959e-5 -7.376114024911682e-5 -4.2610295515187747e-5 5.130987903437382e-5 0.0001051048244499431 9.254199192663128e-6 7.963454901807268e-5 -0.00022747977289500564 -6.0160898448823084e-5 6.142839321734721e-5 0.00012396446310782835 -7.85004298285759e-5 -9.274246929811238e-5 -9.12755343796869e-5 -5.063478545881434e-5 1.959974962853483e-5 6.1766681590665e-5 1.4258927529299515e-5 -0.00014048820538111552 -9.40987514531405e-5 -6.695032732920575e-5 -0.00012499302045573937; -9.984058251908748e-5 -9.002343122452919e-5 6.030095264244136e-5 -4.628777733493452e-5 -4.956562773853718e-6 -2.9293546871173684e-5 -5.955827273611378e-5 -0.00017679114662646396 5.0386783729061495e-5 9.960741041448202e-6 -0.00013858931324981928 7.507069855223133e-6 -7.146893167705971e-5 -8.432608720218951e-5 -0.00018398354714342134 4.5443363442410884e-7 0.00019981400564241558 7.59980381156852e-5 -0.00011349621618470833 6.673625338984129e-5 4.466916525082814e-5 5.173944915744557e-6 4.9567438142124586e-5 -5.1661803266749204e-5 -5.705018466291802e-5 -0.0001406573295786636 2.3861532641912734e-5 -3.3133377113386027e-5 -3.668282934418554e-5 1.2338493976435376e-5 0.00010993102350438319 4.628783298934402e-5], bias = [4.437515830585584e-10, 1.9594993252322064e-9, -5.308393406822397e-10, -1.9317165401249315e-9, -1.6804369970380884e-9, 7.074235209701761e-10, 4.914899489757273e-10, 1.119375377567678e-9, -1.3062737465727374e-9, 1.0321347406851244e-9, -1.4277078442324252e-10, -5.782044560431486e-10, -2.9050243347228516e-10, 1.8372836237685824e-9, -2.679548484324747e-9, -4.117825409116929e-9, 8.405886536388658e-10, 1.6750511624909356e-9, -1.2346542742130683e-9, -1.923272028122853e-9, -1.887120020134511e-9, -1.0041787219358996e-9, 3.102150377145509e-9, -1.7014154634250893e-9, -5.808033600365818e-10, 2.4775001941826805e-9, 2.1946446036671373e-9, -1.4296521011660889e-9, -2.2795349908595105e-9, -3.4110441278516157e-9, -3.22996305276747e-9, -2.722484775317188e-9]), layer_4 = (weight = [-0.0007867009066537449 -0.0006224015980308057 -0.000444956728781826 -0.0006472975633747372 -0.0005953459865030687 -0.0007467360952999541 -0.0008249927572664053 -0.0008172507531907208 -0.0007264717282625864 -0.0007766818734608853 -0.0006811380285880877 -0.0008565912204912905 -0.0007093526384052144 -0.0007243385789572986 -0.0006220253485736737 -0.000674710999483002 -0.0008246203059352311 -0.0007669096947228696 -0.0008646959428964223 -0.0007741699374504568 -0.0006929477876594768 -0.000603749788953262 -0.0007196817701465205 -0.0004200473523943576 -0.0007372375652553526 -0.0005787889621067207 -0.0006720663781192687 -0.0008191210063249355 -0.0006172854121024927 -0.0004671387862449773 -0.0006836683670708925 -0.000695536447742295; 0.0003091439472017924 0.0003799759317775172 0.00035561306879658736 0.000212844829094219 0.00017378003779430537 0.00024900811024501776 0.00015951260887843164 0.00024047601132138021 0.0002732296908907871 0.00013525608072101174 0.00028384866901007055 0.00020523853473720257 0.00023950750399452675 0.0003600593464662572 7.379666400123996e-7 0.0002605033113944275 9.280597112235319e-5 0.000269425354240891 9.762953174736238e-5 0.00045962503873739547 0.00022421298252437347 0.0002478710205721238 0.00018392716073959746 0.00023791945092090395 0.00019908673084726726 0.00034917132288773224 0.0001841979643931747 0.0003328812460768363 9.803646552512888e-5 0.00022585658063969594 0.00019086928500806669 0.00020750172390734595], bias = [-0.0006965721831938202, 0.00025559112170524283]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
-
-    lines!(ax, losses; linewidth=4, alpha=0.75)
-    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
-
-    fig
-end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
-waveform_nn_trained = first(compute_waveform(
-    dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
-    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
-        alpha=0.5, strokewidth=2, markersize=12)
-
-    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
-        position=:lb)
-
-    fig
-end
    Appendix ​
    julia
    using InteractiveUtils
-InteractiveUtils.versioninfo()
-
-if @isdefined(MLDataDevices)
-    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
-        println()
-        CUDA.versioninfo()
-    end
-
-    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
-        println()
-        AMDGPU.versioninfo()
-    end
-end
    Julia Version 1.10.6
-Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
-Build Info:
-  Official https://julialang.org/ release
-Platform Info:
-  OS: Linux (x86_64-linux-gnu)
-  CPU: 128 × AMD EPYC 7502 32-Core Processor
-  WORD_SIZE: 64
-  LIBM: libopenlibm
-  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
-Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
-Environment:
-  JULIA_CPU_THREADS = 16
-  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
-  JULIA_PKG_SERVER = 
-  JULIA_NUM_THREADS = 16
-  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
-  JULIA_PKG_PRECOMPILE_AUTO = 0
-  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const M=t(e,[["render",Z]]);export{j as __pageData,M as default};
diff --git a/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.D_HX1t4_.js b/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.D_HX1t4_.js
new file mode 100644
index 0000000000..e8c06c33be
--- /dev/null
+++ b/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.D_HX1t4_.js
@@ -0,0 +1,286 @@
+import{_ as h,c as i,a2 as n,j as A,a as t,o as a}from"./chunks/framework.CCjWn1F9.js";const L=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),e={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},m={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},c={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},X={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},D={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function R(N,s,U,Z,j,w){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
+      OptimizationOptimJL, Printf, Random, SciMLSensitivity
+using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=t("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=t(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=t(", ")),A("mjx-container",r,[(a(),i("svg",d,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=t(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
+    M = m₁ + m₂
+    r₁ = m₂ / M .* path
+    r₂ = -m₁ / M .* path
+    return r₁, r₂
+end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=t("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",C,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
+    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
+
+    if size(soln, 1) == 2
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+
+        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
+        p, M, e = model_params
+    else
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+        p = soln[3, :]
+        e = soln[4, :]
+    end
+
+    r = p ./ (1 .+ e .* cos.(χ))
+    x = r .* cos.(ϕ)
+    y = r .* sin.(ϕ)
+
+    orbit = vcat(x', y')
+    return orbit
+end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
+    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
+    b = (v[3:end] .- v[1:(end - 2)]) / 2
+    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
+    return [a; b; c] / dt
+end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
+    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
+    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
+    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
+    return [a; b; c] / (dt^2)
+end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
+    x = orbit[1, :]
+    y = orbit[2, :]
+
+    Ixx = x .^ 2
+    Iyy = y .^ 2
+    Ixy = x .* y
+    trace = Ixx .+ Iyy
+
+    if component[1] == 1 && component[2] == 1
+        tmp = Ixx .- trace ./ 3
+    elseif component[1] == 2 && component[2] == 2
+        tmp = Iyy .- trace ./ 3
+    else
+        tmp = Ixy
+    end
+
+    return mass .* tmp
+end
+
+function h_22_quadrupole_components(dt, orbit, component, mass=1)
+    mtensor = orbit2tensor(orbit, component, mass)
+    mtensor_ddot = d2_dt2(mtensor, dt)
+    return 2 * mtensor_ddot
+end
+
+function h_22_quadrupole(dt, orbit, mass=1)
+    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
+    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
+    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
+    return h11, h12, h22
+end
+
+function h_22_strain_one_body(dt::T, orbit) where {T}
+    h11, h12, h22 = h_22_quadrupole(dt, orbit)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
+    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
+    h11 = h11_1 + h11_2
+    h12 = h12_1 + h12_2
+    h22 = h22_1 + h22_2
+    return h11, h12, h22
+end
+
+function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
+    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
+
+    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
+
+    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
+    @assert mass_ratio≤1 "mass_ratio must be <= 1"
+    @assert mass_ratio≥0 "mass_ratio must be non-negative"
+
+    orbit = soln2orbit(soln, model_params)
+    if mass_ratio > 0
+        m₂ = inv(T(1) + mass_ratio)
+        m₁ = mass_ratio * m₂
+
+        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
+        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
+    else
+        waveform = h_22_strain_one_body(dt, orbit)
+    end
+    return waveform
+end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",g,[(a(),i("svg",o,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=t("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=t(", ")),A("mjx-container",F,[(a(),i("svg",u,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=t(", and ")),A("mjx-container",T,[(a(),i("svg",m,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=t(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
+    χ, ϕ = u
+
+    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
+    denom = sqrt((p - 2)^2 - 4 * e^2)
+
+    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
+    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
+
+    return [χ̇, ϕ̇]
+end
+
+mass_ratio = 0.0         # test particle
+u0 = Float64[π, 0.0]     # initial conditions
+datasize = 250
+tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
+tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
+dt_data = tsteps[2] - tsteps[1]
+dt = 100.0
+const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
+soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
+waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
+
+    axislegend(ax, [[l, s]], ["Waveform Data"])
+
+    fig
+end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
+    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
+ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[0.000101433325; -0.00014970472; -3.5847308f-5; -3.0095046f-6; -5.837021f-5; -5.530635f-5; -0.00014913769; -8.2657076f-5; 1.8703817f-6; 2.8848355f-5; 1.4417893f-5; 4.5086595f-5; 4.1238396f-5; 5.863067f-5; 0.000118168115; -2.7185877f-5; -2.2295013f-5; 0.00021102077; -2.7826984f-5; -7.046538f-5; 3.6819965f-5; -0.000122086; 4.216995f-5; 3.7783706f-5; -0.00029299365; 0.00012795332; 2.003964f-5; -7.480524f-5; -1.2783274f-5; 8.586675f-5; -0.00016691082; -0.00014107962;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-5.25561f-5 5.0319617f-5 0.000121188066 -0.0001252668 -0.00012370043 1.3147543f-5 6.7192734f-5 0.00028501067 -4.615136f-5 5.010297f-5 0.0001109413 0.00012366676 0.00012497939 -5.900952f-6 1.5949308f-5 -0.00010800126 -0.00019976492 8.1667444f-5 -0.00020031813 9.338467f-5 -6.742277f-5 4.060099f-5 -8.8431494f-5 -9.7043405f-5 4.6010473f-6 -0.0001405853 5.4320717f-5 8.4588464f-5 -7.09648f-5 -8.0628546f-5 6.95526f-5 -9.106946f-5; 5.6481567f-5 -3.276788f-5 -0.0001785962 -9.946028f-5 -0.00015278214 -6.958734f-5 -8.3435276f-5 6.671795f-5 0.00021108559 3.278153f-5 0.000119060285 5.4152362f-5 6.264283f-5 7.926161f-5 -0.00019341416 -6.553496f-6 7.6160606f-5 -9.66632f-5 -9.5061914f-5 -4.2958072f-5 0.00013052954 -0.00019412015 8.9826804f-5 7.1718045f-5 0.00019083949 6.192604f-6 -3.821931f-6 4.3647802f-7 -0.00014268934 5.3210428f-5 8.25371f-5 -1.2173807f-6; 4.5715973f-5 -5.495184f-5 -0.00011186839 -5.385712f-5 2.0592182f-5 -0.00011375335 -0.00012681524 -5.3366508f-5 8.4260675f-5 -0.000104376966 -9.465132f-5 9.176428f-5 -0.00019846226 4.180067f-5 2.441713f-5 -2.2916829f-5 -3.6965794f-5 8.515369f-5 -4.7670128f-5 -2.4597295f-5 -0.000115552284 5.7741276f-5 -2.149032f-5 0.0001659628 -3.8053244f-5 9.7449265f-5 1.5938956f-5 0.0001071606 0.00015616491 -0.0002005834 -2.687522f-5 1.2993707f-5; 5.1912615f-5 -9.1409915f-5 4.7006324f-5 0.0001274501 0.00010122003 3.093884f-5 4.6041696f-5 -2.819437f-5 7.968095f-6 5.1355753f-5 -9.012398f-5 -0.00016678765 4.6333556f-5 -1.850147f-5 -0.00011283306 2.4248398f-5 -0.0001531043 -0.00020931265 0.00019833563 5.7953574f-5 -8.870693f-5 -9.257044f-5 2.1976448f-5 -3.2718603f-5 -5.585929f-5 7.810437f-5 0.00022372045 3.585174f-5 -2.6321854f-5 9.45166f-5 0.00018568635 -0.000118498785; -8.421658f-5 0.00017808175 0.00015194518 -0.000113057125 -0.00012066419 -0.00018935993 -1.0009757f-5 -6.276838f-5 -2.1114492f-5 -1.8010991f-5 -0.00012293975 0.00010749552 4.4360055f-5 1.2918086f-5 -0.00012972708 1.2805929f-5 0.00014863386 -0.00019670448 -0.00028618306 -8.474048f-5 4.8318067f-5 -0.00013781365 -2.8484217f-5 1.2321886f-5 3.064775f-5 -6.4827924f-5 3.7555023f-6 1.5734184f-5 2.1832733f-5 0.0001064967 -3.4004093f-5 0.00016090772; -4.9504306f-5 1.5704822f-5 9.3366885f-5 6.956209f-5 -5.8176258f-5 -3.641004f-5 1.0974424f-5 -3.3011274f-5 6.968066f-5 -0.00012221691 -2.241614f-5 -0.00021536481 9.290237f-5 -4.6644625f-5 -1.2897161f-5 0.00013195348 -0.00010431149 -0.00011168598 9.9419194f-5 -0.00010479866 -2.6149884f-5 4.048816f-5 -5.4505414f-5 9.3042545f-5 -3.5675401f-7 3.167108f-5 0.00010144765 -2.0471824f-5 7.499174f-5 0.00012745548 -1.9343881f-5 -4.0860472f-5; 3.480437f-5 7.530175f-5 -0.00013266527 0.00024938013 0.00016262576 -8.980506f-5 7.534006f-6 7.878023f-5 4.098693f-5 7.753387f-7 6.413197f-5 -4.2840365f-5 -0.00011310589 4.09235f-5 -4.548707f-5 -4.3824835f-5 -3.7166952f-5 -0.0001460987 -8.8912086f-5 -4.724518f-5 -6.30762f-5 0.0001248795 -2.1304417f-5 0.000109929126 9.500971f-5 -6.1527586f-5 3.875689f-5 -1.0796941f-5 -8.256911f-5 3.3877268f-6 2.0451178f-5 -9.746762f-5; -8.7214146f-5 0.00010691131 -7.3489726f-5 -0.00017095437 1.7527753f-5 8.161695f-5 -8.896779f-5 1.6436641f-5 0.00014055088 3.1515363f-5 -8.762248f-5 -3.935946f-5 -0.00022448925 0.00015336409 1.1825823f-5 0.00012907668 -0.00011167856 0.00010605673 -1.8883887f-6 6.4180625f-5 -0.00012380801 -2.5067176f-5 4.1593237f-5 5.507989f-5 0.00016835274 0.00010800524 -5.2248088f-5 -2.9934452f-5 5.9419897f-5 7.210349f-5 -9.05596f-5 -4.981542f-5; -8.3190884f-5 -0.00013716334 9.333262f-5 3.0901905f-5 0.00018078604 3.6113823f-5 -2.474314f-5 -1.1500239f-5 -2.5257072f-5 -7.240517f-5 -7.065769f-5 -0.0002254977 1.833079f-5 -7.38984f-5 8.5560125f-5 0.0001946904 8.675337f-5 6.741229f-5 -6.8029505f-5 -5.6107692f-5 -0.00012546312 -9.770373f-6 -3.158323f-5 -2.5108455f-5 0.00012979835 5.185795f-5 -3.5403886f-5 0.00020563042 -0.00017649106 9.987908f-5 8.9251225f-5 -6.09851f-5; 0.00015459415 -1.9436993f-5 3.1855652f-5 -2.8804572f-5 0.00010122694 4.723484f-5 1.5064943f-6 5.0922765f-5 1.0278612f-6 -4.2696785f-5 -5.4365966f-5 -0.00020424122 5.340165f-6 -5.1149924f-5 5.5771772f-5 -0.00011463864 -1.7955612f-5 -3.0032486f-5 -0.00012965355 1.8046007f-5 -0.00013679774 2.9808103f-5 4.8935217f-5 -3.587022f-5 -5.076042f-5 -1.201444f-5 -4.7412987f-6 0.00022637536 6.3304215f-5 -1.0715298f-5 -1.32741f-5 -3.9808005f-5; -0.00017152919 8.569154f-5 -0.00015427994 0.0001518926 -7.394991f-5 6.4593505f-6 0.00010508767 0.0001652748 -0.00011657923 8.97161f-6 3.1926393f-5 -0.00014740185 3.5467187f-5 0.00016369366 0.00010648671 0.000103018654 0.00016443468 1.6788868f-5 6.344208f-5 2.0847823f-5 9.915987f-6 0.00013888412 -0.00020764141 0.00014734587 3.127726f-5 0.0001764748 6.793151f-5 -2.079785f-5 -7.642907f-5 0.00010638243 4.9633407f-5 -0.00011511093; 4.7663376f-5 5.5237564f-5 0.00010843301 -5.452355f-5 3.5808926f-6 -3.5294324f-5 2.4117173f-6 0.00012329305 -0.0001269051 -7.7190236f-7 -0.000119915334 -6.727885f-5 0.0002076928 -6.7405424f-5 -4.3333755f-5 5.2977874f-5 -0.00011457315 0.00019208078 -5.5476045f-5 -5.8278398f-5 9.9641446f-5 7.6247874f-5 -3.2882388f-5 2.0014795f-5 5.4410673f-5 2.1790565f-5 6.1696584f-5 8.6672895f-5 0.00012334628 1.748379f-5 0.0001684268 -0.00013439129; 6.154388f-6 -0.000103090264 1.8575416f-5 4.248712f-5 -2.0510362f-5 2.7966196f-5 -4.1805106f-6 0.000100216464 -3.9296287f-5 0.00015753518 -0.00010629292 0.00015053408 5.062221f-6 0.00017994971 2.6507929f-5 6.469896f-5 3.0912197f-5 -3.787202f-5 -9.180992f-5 9.380597f-6 1.7937424f-5 0.0001484272 0.00014581943 -0.00015441813 2.5858413f-5 0.00018289514 5.500523f-5 -1.8279512f-5 0.00018384538 -2.7853945f-5 1.1434443f-5 9.653837f-5; 1.1493156f-5 0.00012901722 -0.00022245743 0.00010968725 -5.9707843f-5 -5.9917904f-5 -3.61098f-5 8.152264f-5 -7.820346f-5 -3.3505003f-5 -3.7833874f-5 0.00011963542 -0.000101400816 -2.5519417f-5 -3.9913408f-5 -1.4419071f-5 1.0952946f-5 3.0841573f-5 -3.0217423f-5 -8.11942f-5 0.00012752488 0.00019579084 -7.06113f-6 0.00011654196 0.00023443296 0.000111647285 0.00014851333 4.3228825f-5 0.00012218115 0.00017196768 -2.9187097f-5 0.00010714686; 2.3869788f-5 -2.8650662f-5 0.00021611001 -1.7511142f-5 2.731428f-5 7.015406f-5 0.00011499756 3.0493577f-5 -1.3791175f-5 -9.098295f-5 -8.075956f-5 6.571922f-5 -4.6558005f-5 3.8361726f-5 -6.263231f-5 9.8436394f-5 -5.444673f-5 -0.0001525929 -2.8079692f-5 0.00020451599 -0.00011710578 8.0096115f-6 1.7739687f-5 -0.00017146657 -1.1216441f-5 5.582888f-5 2.102432f-5 -3.2097083f-5 -5.654088f-5 -0.0002029514 0.00012005623 -4.6535497f-5; 4.0600895f-5 -0.00016371673 -0.00017366491 2.4281542f-5 3.0944266f-5 9.9873614f-5 -2.5847257f-5 0.00020838533 -8.538478f-5 -4.1817697f-5 0.00021643919 1.0913612f-5 -0.00013417867 -0.00011062858 -5.303284f-5 -1.5524723f-6 -2.2601067f-5 3.064764f-5 -0.00012356581 9.590809f-5 0.00014010115 5.2191863f-6 -0.0001268768 -5.8050922f-5 -8.722839f-5 -7.693819f-5 -9.255185f-5 0.00025757347 3.3973716f-5 -0.00011420436 -3.4002467f-5 0.00013258631; 3.1190964f-5 0.00014198378 -0.000109832414 3.0770072f-5 -9.37026f-5 -6.4940985f-5 -6.932041f-6 0.00013815345 0.00014333393 -6.7661975f-5 -4.511986f-5 -2.4029983f-5 -5.3136577f-5 0.00026887178 -0.00015120419 6.6838904f-5 0.00013478157 0.00014922114 2.1898917f-5 -9.330819f-5 2.4437604f-5 -3.936817f-5 -0.00010707723 -1.4017039f-5 6.4641405f-5 4.3191016f-5 7.486379f-5 -9.5469855f-5 -1.4938067f-5 0.00012933458 9.605733f-5 -0.00014525898; 0.00016662048 9.602477f-6 -5.0595074f-5 -0.00010192549 -0.00010132676 -7.178766f-5 -4.139689f-5 6.0371243f-5 -1.2646691f-5 -0.00015618675 -4.537753f-6 0.00019719187 1.20949235f-5 -0.00019831482 -0.00012459549 -3.3654065f-5 1.3041183f-6 8.898287f-5 -4.7517875f-5 -9.4529514f-7 -2.2393571f-5 -7.230482f-5 -5.2415697f-5 0.00015502863 -1.2729995f-5 -3.9127535f-5 -6.104951f-5 -2.6234753f-5 -0.00013506036 -5.3432243f-5 -0.000111437635 8.188779f-5; 9.936778f-5 9.609397f-5 1.8602841f-5 8.174792f-5 3.0406422f-5 5.6035467f-5 2.6022355f-5 8.32676f-5 -0.00011590395 0.0001480582 3.0665713f-5 -8.2545f-5 -3.1364634f-5 3.6824866f-5 6.449255f-5 5.598043f-5 0.00011874976 0.00021320413 -9.1623195f-5 6.0359876f-6 -1.1393765f-5 -0.00022938936 -9.551152f-5 6.278853f-5 0.00015201893 4.6772777f-5 3.4416792f-6 -5.5460823f-5 -3.434725f-5 -0.0001919227 -0.00016547991 -8.913108f-6; -1.0548262f-5 -0.00022290568 6.221811f-5 -3.8161495f-5 -3.203011f-5 5.9161786f-5 2.3230212f-5 3.1728236f-5 8.554471f-5 1.3226349f-5 0.00014533396 7.015475f-5 0.00011469073 8.4968866f-5 0.000119700926 3.688433f-5 0.000105032515 -0.0001410187 4.3452623f-5 -0.00012144251 4.901844f-6 4.2582407f-5 0.0001310712 2.195234f-5 8.650805f-5 4.625665f-5 -7.943035f-5 -7.426819f-5 -9.102839f-5 -4.7004334f-5 6.47525f-5 6.314009f-5; -4.3685573f-5 -7.0061484f-5 -8.564832f-5 -2.5506959f-5 -3.3139113f-5 -0.00012296431 5.0306004f-5 -2.598046f-5 3.1052587f-5 -2.203405f-5 -2.4192013f-5 -1.2408228f-7 0.00018310527 8.597894f-5 -0.00016243967 -8.256807f-5 4.5036827f-6 -4.9110735f-5 -9.529562f-5 0.00011435464 1.8392873f-5 1.6941232f-7 -5.0154013f-5 7.089302f-5 2.9975381f-5 -5.9508693f-5 -2.4033567f-5 0.00012779109 -0.00018340634 -6.473091f-5 -0.00015620314 -8.254381f-5; -2.5668716f-5 4.8003552f-5 -3.2946562f-5 4.2170883f-5 8.5557236f-5 0.0001284341 -5.9376423f-5 -2.011942f-5 -3.5316505f-5 4.182381f-5 -2.0610467f-5 -8.933989f-6 2.7255755f-5 -0.00017342126 2.8045699f-5 2.8669716f-5 -7.505687f-5 -0.00023754647 -0.0002875935 -0.00011733108 -7.4658514f-5 8.5557935f-5 5.5539273f-5 0.00014043871 7.694104f-5 0.0001427624 -7.485815f-5 3.3148644f-5 -9.5938376f-5 -7.424506f-5 8.785261f-5 -4.980164f-5; -9.200019f-5 -0.00024000471 4.1095227f-5 -4.7544443f-5 -8.818522f-6 0.00020778744 7.85882f-6 6.168399f-5 0.0001310395 -2.998988f-5 -2.7495777f-5 0.00013481667 9.5430165f-5 -0.000120056735 -5.0351315f-5 1.976981f-5 -3.72669f-5 4.0843f-6 3.3779917f-5 1.3540122f-5 0.00011324852 3.395571f-5 3.1503678f-5 -0.00010043496 7.797303f-5 0.00017457195 -1.5625114f-5 5.3623786f-5 -0.00012425627 -1.9527579f-5 4.4190932f-5 6.129173f-5; -0.00019794701 0.0002352912 -4.594587f-5 3.4971752f-5 -6.8977897f-6 4.942128f-5 5.3490952f-5 5.9408554f-5 -0.00011715945 2.3265313f-6 0.00010682193 -8.7325025f-5 0.00018497492 -7.528666f-5 6.642244f-5 -8.35431f-6 5.2583666f-5 -1.5171533f-5 -0.00012801825 6.694276f-5 -0.00013729051 -2.2952234f-5 -2.2788714f-5 -6.407778f-5 -0.00012958882 -1.7841387f-5 5.423201f-5 3.9521266f-5 -8.18932f-5 -7.0958435f-5 -1.9372379f-5 5.161007f-5; -7.9233185f-5 -7.5692906f-6 9.44239f-5 -5.664948f-5 -6.476559f-5 -6.323061f-5 -0.00011693359 7.1488954f-5 -4.89746f-5 0.00010430032 3.8002876f-5 -0.00014357506 -3.477876f-5 -1.2791332f-5 3.5682064f-5 0.00010522572 0.00013691728 4.3447893f-5 -0.0001987566 -7.380969f-5 1.4467216f-5 -1.857528f-5 5.0088503f-5 -2.678466f-5 -3.583864f-5 0.000121977566 -0.00011532569 8.403785f-5 -0.00012407296 0.000100155325 -0.00011512785 0.00010114831; 0.00012335247 2.6316393f-6 7.941742f-5 0.00013124838 -1.791447f-5 0.000108683664 1.9190291f-5 3.1741965f-5 4.0499726f-5 2.5934998f-5 0.00022457454 2.7964175f-5 -7.64013f-5 -6.803024f-6 6.500241f-5 1.1759023f-5 -4.710968f-5 -4.3526998f-5 2.3740988f-5 -4.677318f-5 5.2313593f-5 4.8671518f-5 -1.3229032f-5 -5.2332278f-5 -3.818454f-5 -2.7911294f-5 3.0976364f-5 0.000104312705 -1.6617805f-5 1.4596436f-5 -0.00011833158 3.6496947f-6; -2.8291538f-6 1.3292323f-5 0.00013545212 5.787567f-5 -0.00013302402 1.1834459f-5 -0.00031529382 0.00012097406 -1.0305068f-5 -9.146762f-5 -3.9248484f-5 7.469065f-5 6.0470862f-5 -0.0001779905 1.3491451f-5 -7.1607115f-5 -6.8425274f-5 0.00010432296 -4.0927673f-5 9.4371106f-5 -1.8413417f-5 9.450085f-5 6.1815495f-5 0.00010917549 0.00013303825 -0.0001648625 0.00019893328 9.623225f-5 0.000281235 4.8627415f-5 0.00021276702 1.8762299f-5; 0.00020731823 5.1270174f-5 4.631842f-5 -5.7199144f-5 -1.864361f-5 -9.590181f-5 -3.1887441f-6 -6.806665f-5 -5.867507f-5 5.4535365f-5 0.00013211838 2.669618f-5 0.00025670705 0.0001305444 -0.0001418291 4.950635f-5 -6.779437f-5 0.000101174875 1.6520089f-5 -3.5010715f-5 8.299325f-5 -7.4818556f-5 -3.9792998f-5 8.352244f-5 0.00018409271 -5.325691f-5 -2.133156f-5 -0.00020594428 -0.0001694123 -8.223999f-5 -8.586627f-5 7.546048f-5; 3.154289f-5 3.5365934f-5 2.9560859f-5 5.9459628f-5 -0.00012007089 -2.0401096f-5 3.01665f-6 0.00011454203 -2.1529537f-5 9.529565f-5 5.5518474f-5 -6.0801143f-5 0.00011015713 -3.3875676f-5 1.8247714f-5 6.0837523f-5 2.8100952f-5 -6.3356914f-5 -1.7917748f-5 -5.492708f-6 -0.00024452427 0.00015829744 -0.00016907782 -0.000117363714 4.680081f-5 8.013908f-5 -9.65113f-5 -0.00019136128 -0.00011032749 3.5442154f-5 -9.167989f-5 -7.021357f-5; -0.00011151572 -0.00011673683 -7.134072f-5 0.000118420925 -9.90588f-5 -3.7308397f-5 -0.00015447076 1.6448188f-7 -5.2847383f-5 -8.245325f-5 2.159159f-5 5.021692f-5 2.6484282f-5 0.0001992939 -0.000114767594 -1.20055f-5 3.6866324f-5 -2.5806152f-5 8.276754f-6 1.4364273f-5 0.00019493535 3.649333f-5 0.00010192474 9.985735f-5 4.2894528f-5 -0.00013973896 -3.5243203f-5 3.8965212f-5 -5.715798f-5 -4.9403927f-5 0.00021121927 3.575224f-5; 0.00014715974 -2.641252f-5 0.00017523114 -4.8744627f-5 0.00014663808 6.443972f-5 5.1060815f-5 -0.00017406323 7.325399f-5 -3.0334577f-5 -3.290726f-5 -2.8855047f-5 -0.00010346927 0.000109804154 6.330436f-5 0.000105802435 0.00013098773 -9.474001f-6 -8.608316f-6 -7.789192f-5 -3.9351817f-5 -3.7995953f-5 -2.9535186f-5 -4.6726905f-6 -2.3550838f-5 -3.8522256f-5 -0.00021247026 4.007786f-6 0.00018809095 -9.240431f-5 0.00011848084 -8.009428f-5; -0.00012021001 0.00016060712 -0.00013262503 8.1844984f-5 0.000104574036 0.00010485861 -0.00019663687 3.4952027f-5 -0.00017723769 -2.6319058f-5 3.1220454f-5 -9.912223f-5 -0.00012967248 9.97342f-5 -4.808475f-6 -0.0001229508 2.4496549f-5 3.486451f-5 -0.00023266925 8.925509f-5 7.762511f-5 5.531554f-6 3.353919f-5 5.509532f-5 -5.4878186f-5 0.00011217324 -1.2349672f-6 0.00019266526 4.035966f-5 7.287375f-5 9.475787f-5 -6.576237f-5], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-9.368759f-5 1.6582828f-5 -3.5511866f-6 3.0926352f-5 7.026318f-5 -0.00013142922 3.7274265f-5 -6.480119f-5 9.593723f-5 -2.6824922f-5 -6.485373f-5 2.9562175f-6 -9.454155f-5 -6.707946f-5 2.5691726f-5 -0.00011177807 3.0246403f-5 7.450661f-5 -0.00012384634 6.464391f-5 7.969684f-5 -4.669152f-5 7.165037f-6 0.00011992512 4.711118f-5 7.2177645f-5 4.6727633f-5 -4.2947895f-6 -4.7917452f-5 -8.243532f-5 -0.00013366244 -5.2003056f-6; 8.115342f-5 7.3465126f-5 1.5182278f-5 -4.776809f-6 0.00017834424 3.881097f-5 0.00015579326 -4.561653f-5 2.4572842f-5 4.7196598f-5 -2.3822335f-5 -8.144347f-5 -3.6691436f-5 -0.00021805007 -7.189355f-5 9.418967f-5 -4.648693f-5 3.2287953f-5 -3.575809f-5 0.00014702333 -0.00015603313 -5.8873688f-5 -8.486313f-5 7.7675686f-5 2.935245f-5 0.000111844696 -0.00010060676 -1.0563847f-6 -8.139529f-5 -1.7000513f-5 -0.00029249687 -8.839286f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
+
+const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
+    Chain(
+        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
+        layer_2 = Dense(1 => 32, cos),  # 64 parameters
+        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
+        layer_4 = Dense(32 => 2),       # 66 parameters
+    ),
+)         # Total: 1_186 parameters,
+          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",c,[(a(),i("svg",B,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",z,[(a(),i("svg",V,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=t("where, ")),A("mjx-container",q,[(a(),i("svg",b,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=t(", ")),A("mjx-container",X,[(a(),i("svg",D,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=t(", and ")),A("mjx-container",P,[(a(),i("svg",K,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=t(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
+    χ, ϕ = u
+    p, M, e = ode_model_params
+
+    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
+    # it, however, in general, we should use \`st\` to store the state of the neural network.
+    y = 1 .+ nn_model([first(u)], nn_params)
+
+    numer = (1 + e * cos(χ))^2
+    denom = M * (p^(3 / 2))
+
+    χ̇ = (numer / denom) * y[1]
+    ϕ̇ = (numer / denom) * y[2]
+
+    return [χ̇, ϕ̇]
+end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
+waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    axislegend(ax, [[l1, s1], [l2, s2]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
+
+    fig
+end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
+
+function loss(θ)
+    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
+    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
+    return mseloss(pred_waveform, waveform)
+end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0006994921614618403
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
+
+function callback(θ, l)
+    push!(losses, l)
+    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
+    return false
+end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
+optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
+optprob = Optimization.OptimizationProblem(optf, params)
+res = Optimization.solve(
+    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
+    callback, maxiters=1000)
    retcode: Success
+u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [0.00010143332474399266; -0.00014970471966083986; -3.584730802686131e-5; -3.0095045531156137e-6; -5.837020944455422e-5; -5.530635098688162e-5; -0.00014913768973205503; -8.265707583622401e-5; 1.870381652225233e-6; 2.8848355213978774e-5; 1.4417892998606726e-5; 4.508659549173389e-5; 4.123839607928261e-5; 5.863067053716329e-5; 0.00011816811456800735; -2.7185877115678053e-5; -2.229501296823788e-5; 0.0002110207715302368; -2.7826983568923675e-5; -7.046537939462137e-5; 3.681996531663573e-5; -0.00012208599946444672; 4.2169951484519065e-5; 3.778370592044982e-5; -0.00029299364541612773; 0.00012795331713263893; 2.003964073079977e-5; -7.480524072890363e-5; -1.278327363249883e-5; 8.586674812241179e-5; -0.00016691081691508447; -0.0001410796248819983;;], bias = [7.6713176236549e-19, 3.696847498061422e-17, 2.4055689200670786e-18, -4.74888813316211e-18, 8.733758721019089e-18, -6.464424972716262e-17, 4.685057209353382e-17, -1.141839913481628e-16, -5.842869798050569e-19, 1.8510923759576366e-17, 1.3679823189400309e-18, 6.669020875018921e-18, -4.063228054300107e-18, 8.877328713490479e-17, 1.1229820843349717e-16, -2.356901393761471e-17, -6.315824425403808e-18, 2.5453541817767217e-16, -3.7526064434901786e-17, 3.5326060372288575e-17, 5.257347495428192e-17, -1.9470226826847666e-16, 2.2141221075579763e-17, -3.109513202642436e-18, -5.151280825108199e-16, 1.6121381886129855e-16, 3.3708327975928686e-17, 1.2404157686012208e-17, -2.664045442200898e-17, 1.0050086139984667e-16, -2.8553122257375194e-16, 3.2154322768126957e-17]), layer_3 = (weight = [-5.25564992604587e-5 5.03192185969298e-5 0.00012118766785344962 -0.000125267199157021 -0.00012370083100183914 1.3147144225444096e-5 6.719233528922508e-5 0.00028501027004306206 -4.615175753504133e-5 5.010257332098586e-5 0.00011094090036560689 0.00012366636105791602 0.0001249789874672603 -5.901350552518186e-6 1.5948909956864962e-5 -0.00010800165732878346 -0.00019976532171501866 8.166704533135075e-5 -0.0002003185273243402 9.338426930109939e-5 -6.742316877723749e-5 4.060059104442697e-5 -8.84318924837101e-5 -9.704380322734901e-5 4.6006488882410714e-6 -0.00014058569472770083 5.4320318620669685e-5 8.458806582705095e-5 -7.096519953127111e-5 -8.062894453548825e-5 6.955220482989485e-5 -9.106985640034847e-5; 5.648153107789875e-5 -3.276791524081877e-5 -0.00017859623023947177 -9.946031771074462e-5 -0.0001527821803622454 -6.958737487844602e-5 -8.343531210380995e-5 6.671791213624706e-5 0.000211085549540022 3.278149249557925e-5 0.0001190602489939384 5.4152326060594614e-5 6.264279379569514e-5 7.926157575160787e-5 -0.00019341419813395872 -6.553532171218581e-6 7.616056989240049e-5 -9.666323587703881e-5 -9.506195040126744e-5 -4.2958108520232484e-5 0.0001305295083282009 -0.0001941201843012629 8.982676824847756e-5 7.171800841637475e-5 0.000190839455330881 6.192567684357778e-6 -3.821967152597545e-6 4.3644179731121464e-7 -0.0001426893794134766 5.32103915017989e-5 8.253706635036125e-5 -1.2174169344313458e-6; 4.57142954495314e-5 -5.4953518517909694e-5 -0.00011187006439073063 -5.385879612518597e-5 2.0590504757222437e-5 -0.0001137550249019633 -0.00012681691859838386 -5.336818557921545e-5 8.42589974148187e-5 -0.00010437864384371862 -9.465299499211943e-5 9.176260519083579e-5 -0.0001984639426133598 4.179899351797133e-5 2.4415452028142198e-5 -2.291850669067434e-5 -3.696747134896089e-5 8.515201207261988e-5 -4.76718056212653e-5 -2.4598972775180253e-5 -0.0001155539617663254 5.773959801658752e-5 -2.1491996992752195e-5 0.00016596112564215394 -3.80549214220148e-5 9.74475873611945e-5 1.5937278842412807e-5 0.00010715892440416103 0.00015616323380740252 -0.00020058507343278912 -2.6876896843207e-5 1.2992029407862942e-5; 5.19131751478338e-5 -9.140935461817854e-5 4.700688415124854e-5 0.00012745065733708302 0.00010122059003134992 3.093940150376217e-5 4.6042255880650304e-5 -2.8193810330943946e-5 7.968654890480487e-6 5.1356313369812806e-5 -9.012341642135895e-5 -0.00016678708952002404 4.633411636843179e-5 -1.8500910564565225e-5 -0.00011283250231843832 2.4248958388686553e-5 -0.00015310373445589243 -0.0002093120931842273 0.00019833619418006238 5.7954133544459536e-5 -8.870637274429363e-5 -9.256987713359704e-5 2.1977007881848314e-5 -3.271804261219849e-5 -5.58587310390065e-5 7.810493089874772e-5 0.00022372100621379767 3.5852299069862294e-5 -2.6321293526218726e-5 9.451716296991064e-5 0.0001856869127639691 -0.0001184982250289855; -8.421906372894266e-5 0.0001780792737459982 0.00015194269521963324 -0.0001130596063834579 -0.00012066667100078717 -0.00018936240653637186 -1.0012237771997909e-5 -6.27708576306609e-5 -2.1116973448797237e-5 -1.8013471936127532e-5 -0.00012294222674462564 0.00010749304069541923 4.4357573778699496e-5 1.291560524770234e-5 -0.00012972956014925983 1.2803448180066917e-5 0.00014863137780558722 -0.00019670696187890814 -0.0002861855390168684 -8.474296178099866e-5 4.831558558145079e-5 -0.00013781613220123864 -2.84866981477426e-5 1.2319405095342203e-5 3.064526767928963e-5 -6.483040474483915e-5 3.7530213827364957e-6 1.5731702796634066e-5 2.1830252216760097e-5 0.00010649421906202128 -3.400657428789124e-5 0.0001609052343607146; -4.950440444129138e-5 1.5704723128396553e-5 9.336678675997424e-5 6.956199257790459e-5 -5.817635638716272e-5 -3.641014000943504e-5 1.0974325678648426e-5 -3.3011372860824714e-5 6.968056158318532e-5 -0.0001222170070007688 -2.2416237979383453e-5 -0.00021536490777118708 9.290227507396954e-5 -4.6644723906252673e-5 -1.289725906583554e-5 0.00013195338005123078 -0.00010431158835610957 -0.00011168607698830286 9.941909565047711e-5 -0.0001047987573741247 -2.6149982940593018e-5 4.0488062668986844e-5 -5.450551207952779e-5 9.304244639740679e-5 -3.5685252152966485e-7 3.167098083456447e-5 0.00010144755259777768 -2.0471922024922518e-5 7.499163956770091e-5 0.0001274553830541428 -1.934397942870884e-5 -4.0860570433619975e-5; 3.480445915249443e-5 7.530184260451466e-5 -0.00013266518041143833 0.00024938021991749594 0.00016262584862181272 -8.980496610399982e-5 7.534096345463743e-6 7.878031786878868e-5 4.098702223620674e-5 7.754292474808159e-7 6.413206168565263e-5 -4.28402749670361e-5 -0.00011310579835788056 4.0923590437726185e-5 -4.548698094675063e-5 -4.382474477404435e-5 -3.7166861569291764e-5 -0.00014609860633152372 -8.891199510196862e-5 -4.724508877495954e-5 -6.307611007594211e-5 0.00012487959205520594 -2.1304326276892678e-5 0.00010992921628405631 9.500979971255219e-5 -6.152749525733976e-5 3.875698124522638e-5 -1.0796850313797592e-5 -8.256901898073669e-5 3.3878173204562125e-6 2.0451268516464432e-5 -9.746753172601335e-5; -8.72137428008044e-5 0.00010691171672865347 -7.348932223189744e-5 -0.00017095396861927688 1.7528156321214866e-5 8.161735212402272e-5 -8.896738682901885e-5 1.6437044622444454e-5 0.0001405512845746231 3.151576646692004e-5 -8.762207681806867e-5 -3.9359056528472726e-5 -0.00022448884948184845 0.00015336449331575854 1.1826226626589792e-5 0.00012907708486514048 -0.00011167815347230175 0.00010605713367899186 -1.8879852053473542e-6 6.418102901866489e-5 -0.00012380761137260248 -2.5066772933633293e-5 4.159364078864327e-5 5.508029397665399e-5 0.00016835314062394827 0.00010800564240403152 -5.224768420825822e-5 -2.9934048426502315e-5 5.942030083010798e-5 7.210389202432504e-5 -9.055919900797779e-5 -4.981501507368307e-5; -8.31906623362331e-5 -0.00013716311928972227 9.333284418874019e-5 3.090212652653063e-5 0.0001807862603881634 3.611404412270109e-5 -2.474291846836073e-5 -1.1500018045876406e-5 -2.525685027746876e-5 -7.240495011599202e-5 -7.06574688238783e-5 -0.00022549747933323905 1.8331010723091334e-5 -7.389818034523583e-5 8.556034612313352e-5 0.0001946906153888681 8.675358861994533e-5 6.741251439864033e-5 -6.802928379427108e-5 -5.610747081034315e-5 -0.00012546289923291126 -9.770151847147264e-6 -3.1583008022675615e-5 -2.5108233386230962e-5 0.00012979857587098995 5.185817229987542e-5 -3.540366429758654e-5 0.0002056306433941474 -0.00017649083434760035 9.987929974060993e-5 8.925144669676538e-5 -6.098487989318162e-5; 0.00015459353811902082 -1.9437603677126076e-5 3.18550416894993e-5 -2.880518264565269e-5 0.00010122633169966698 4.723422966062034e-5 1.5058837933777822e-6 5.092215427875536e-5 1.027250724194729e-6 -4.269739589208838e-5 -5.436657671371589e-5 -0.0002042418339070592 5.339554482719717e-6 -5.115053432664885e-5 5.577116166104724e-5 -0.00011463925262351623 -1.7956222344861515e-5 -3.003309689947565e-5 -0.0001296541588627464 1.804539601006992e-5 -0.00013679834888453683 2.98074926294218e-5 4.893460640313953e-5 -3.587083052398726e-5 -5.076103048767231e-5 -1.2015050068933292e-5 -4.7419091598444446e-6 0.00022637474534430537 6.330360405708549e-5 -1.0715908737533902e-5 -1.3274710235328225e-5 -3.9808615164567725e-5; -0.0001715258825111625 8.569484790716378e-5 -0.00015427663882797108 0.00015189590495942777 -7.394660341156399e-5 6.462655359833105e-6 0.00010509097202377471 0.00016527810813790064 -0.00011657592248007039 8.974915285260097e-6 3.19296975380428e-5 -0.00014739854512804722 3.547049137928025e-5 0.00016369696978876612 0.00010649001498171168 0.00010302195888875587 0.0001644379824160282 1.6792173223071718e-5 6.344538425259823e-5 2.0851128023826387e-5 9.919291837329557e-6 0.00013888742406782632 -0.00020763810451676645 0.0001473491735982382 3.128056570344334e-5 0.00017647810610872474 6.793481744757494e-5 -2.07945452398759e-5 -7.642576227731405e-5 0.00010638573595719233 4.96367117324698e-5 -0.00011510762695759204; 4.766572836640417e-5 5.523991656924728e-5 0.00010843535945721786 -5.452119641151299e-5 3.58324503875292e-6 -3.529197184863753e-5 2.414069818099648e-6 0.0001232954042145835 -0.0001269027465375235 -7.69549878640041e-7 -0.00011991298144013656 -6.727649944397793e-5 0.00020769515921099894 -6.740307200263535e-5 -4.3331402971243175e-5 5.2980226051005725e-5 -0.00011457079873682055 0.0001920831369582128 -5.547369202278632e-5 -5.8276045287786293e-5 9.964379804412713e-5 7.625022643448559e-5 -3.288003555528041e-5 2.0017147640174854e-5 5.441302581421965e-5 2.179291785548102e-5 6.169893693848465e-5 8.667524760792516e-5 0.00012334863512049402 1.748614211944747e-5 0.00016842915882558278 -0.0001343889356710264; 6.158470438758418e-6 -0.0001030861814281599 1.8579498698038935e-5 4.249120398230365e-5 -2.0506279855235563e-5 2.7970278876400933e-5 -4.17642799519663e-6 0.00010022054708018434 -3.929220396753885e-5 0.0001575392605961893 -0.00010628883969116538 0.0001505381595568376 5.066303726145864e-6 0.0001799537939011677 2.6512011442468293e-5 6.470304002208984e-5 3.091627955358855e-5 -3.786793890260071e-5 -9.180583505736319e-5 9.384679157608353e-6 1.794150635463917e-5 0.0001484312896143118 0.0001458235136458298 -0.00015441404606759293 2.5862495801101184e-5 0.00018289921795869367 5.500931079654415e-5 -1.8275429413973564e-5 0.00018384945802312768 -2.7849862038479912e-5 1.1438525558973062e-5 9.654245591874819e-5; 1.1497044230215683e-5 0.00012902111012968843 -0.0002224535393759746 0.00010969113901973412 -5.970395539753857e-5 -5.991401593183841e-5 -3.610591118907857e-5 8.152652555161868e-5 -7.81995742580044e-5 -3.350111472519508e-5 -3.7829985725519594e-5 0.00011963930848304569 -0.00010139692784269844 -2.5515529326924823e-5 -3.990951989528911e-5 -1.4415182871570897e-5 1.095683348120538e-5 3.0845460802201694e-5 -3.021353512286594e-5 -8.119031297957508e-5 0.0001275287675673003 0.000195794726838513 -7.057242040728919e-6 0.00011654584779188501 0.00023443684296223387 0.0001116511728616103 0.00014851721317990345 4.32327131567796e-5 0.00012218504236097142 0.00017197156813943473 -2.918320859302846e-5 0.0001071507456946745; 2.38693978535884e-5 -2.865105262750728e-5 0.00021610962220441316 -1.7511532416364836e-5 2.7313888938834663e-5 7.015367235729255e-5 0.00011499716719120773 3.049318592096048e-5 -1.3791565390392404e-5 -9.098333749713619e-5 -8.075994766110806e-5 6.571883067229534e-5 -4.655839576295809e-5 3.8361335544351264e-5 -6.263269962070584e-5 9.843600354363898e-5 -5.444711997549296e-5 -0.0001525932871226935 -2.8080082767673397e-5 0.00020451560202494075 -0.00011710617173955082 8.009220880951566e-6 1.7739296243262974e-5 -0.00017146696110282325 -1.1216831364397432e-5 5.582849052010059e-5 2.1023929082758117e-5 -3.209747366051371e-5 -5.654127154409205e-5 -0.00020295179048113686 0.00012005584220608799 -4.65358875880763e-5; 4.060015640986507e-5 -0.00016371746818351842 -0.0001736656521987529 2.4280803756635753e-5 3.0943527292192e-5 9.98728756368737e-5 -2.5847995800928283e-5 0.0002083845939179537 -8.538551986733346e-5 -4.181843536374034e-5 0.00021643845326449829 1.0912873278510902e-5 -0.0001341794085763753 -0.00011062931656654735 -5.303357824034544e-5 -1.55321077984955e-6 -2.2601805483492784e-5 3.0646901052186956e-5 -0.0001235665512811753 9.590735504586477e-5 0.00014010041564885663 5.218447824674669e-6 -0.00012687753400117012 -5.805166068769852e-5 -8.722913111142489e-5 -7.693892615664247e-5 -9.255258549986294e-5 0.0002575727303903201 3.397297774624188e-5 -0.00011420510048795209 -3.400320559905991e-5 0.0001325855737942889; 3.119263521317027e-5 0.00014198544987980186 -0.00010983074287845628 3.0771742893068196e-5 -9.37009271020907e-5 -6.493931403578232e-5 -6.93036998395775e-6 0.00013815512385723008 0.00014333560567853233 -6.766030390475728e-5 -4.5118190320989156e-5 -2.4028312402278282e-5 -5.313490590391579e-5 0.00026887345040255226 -0.00015120252181272583 6.684057472544877e-5 0.00013478324137200767 0.0001492228158147876 2.1900587960646143e-5 -9.330651926769951e-5 2.443927507361983e-5 -3.9366498533260655e-5 -0.00010707555600869373 -1.4015367816808866e-5 6.46430754548516e-5 4.319268656932053e-5 7.486546120877551e-5 -9.54681844470009e-5 -1.4936395820894294e-5 0.00012933625577521304 9.60590015145911e-5 -0.00014525730774234935; 0.00016661754547165592 9.5995452626037e-6 -5.059800547535577e-5 -0.00010192842397594061 -0.00010132969269982383 -7.179059263073716e-5 -4.1399819895018836e-5 6.03683116765555e-5 -1.264962280622851e-5 -0.0001561896855150079 -4.540684460542714e-6 0.00019718893657916247 1.2091991845930406e-5 -0.0001983177565875168 -0.00012459842342870078 -3.365699682256304e-5 1.301186682732619e-6 8.897994179061968e-5 -4.75208065494625e-5 -9.482267727648984e-7 -2.2396502902144953e-5 -7.230775314602677e-5 -5.241862833138554e-5 0.00015502569352754357 -1.2732927063860792e-5 -3.9130466905054665e-5 -6.105244334542321e-5 -2.6237684653481535e-5 -0.00013506328774604167 -5.343517419753061e-5 -0.00011144056712728296 8.188485720674376e-5; 9.93689685051731e-5 9.609515865262418e-5 1.8604027274294636e-5 8.174910936900174e-5 3.0407607765980987e-5 5.603665368794104e-5 2.602354135118717e-5 8.32687877041703e-5 -0.00011590276670945117 0.00014805938582098773 3.066689924848949e-5 -8.254381072079152e-5 -3.136344793206388e-5 3.682605193857661e-5 6.449373932950347e-5 5.598161470656944e-5 0.00011875094816018567 0.00021320531260383206 -9.162200848409247e-5 6.037173817773901e-6 -1.1392578643109326e-5 -0.00022938817682048194 -9.551033116481576e-5 6.278971733222418e-5 0.00015202011155589635 4.6773963105390396e-5 3.442865419446369e-6 -5.545963694062543e-5 -3.434606304695145e-5 -0.0001919215207530704 -0.00016547872627458 -8.91192183958915e-6; -1.0545969271800749e-5 -0.00022290339151415838 6.222040352394661e-5 -3.815920248644475e-5 -3.202781848859344e-5 5.916407932044661e-5 2.3232504790464034e-5 3.173052860755846e-5 8.55469999437402e-5 1.3228642419102902e-5 0.00014533625759167532 7.015703991737016e-5 0.00011469302367867727 8.497115883027829e-5 0.00011970321898796433 3.6886621418938875e-5 0.00010503480840224687 -0.00014101640303173694 4.3454915757587986e-5 -0.00012144021351606074 4.904137003444741e-6 4.258470031298464e-5 0.00013107348891889498 2.1954633811668815e-5 8.651034400775136e-5 4.625894249282434e-5 -7.94280557242975e-5 -7.426589476427531e-5 -9.102609950833082e-5 -4.70020412108269e-5 6.475479418392951e-5 6.314238376897153e-5; -4.368849663732603e-5 -7.006440687544458e-5 -8.565123976295764e-5 -2.550988215516858e-5 -3.3142036093340545e-5 -0.00012296723257604767 5.030308052229863e-5 -2.598338328679915e-5 3.104966340033266e-5 -2.2036972998469065e-5 -2.4194936534334495e-5 -1.2700544089169337e-7 0.00018310234185919255 8.597602022491649e-5 -0.0001624425913658241 -8.257099221082406e-5 4.500759539220936e-6 -4.911365799922083e-5 -9.529854110296813e-5 0.00011435171649936983 1.8389950080016263e-5 1.664891530819169e-7 -5.015693573368616e-5 7.089009336341235e-5 2.9972457875633954e-5 -5.951161624833396e-5 -2.4036489824359646e-5 0.0001277881692178221 -0.00018340926731325743 -6.473383660049485e-5 -0.00015620606250447454 -8.254673416812025e-5; -2.56702805278856e-5 4.80019880990992e-5 -3.29481262530745e-5 4.216931854108423e-5 8.555567189552139e-5 0.00012843253359398103 -5.9377987716903e-5 -2.0120984053104614e-5 -3.5318069357061005e-5 4.1822244448951675e-5 -2.0612031156531617e-5 -8.935553458910985e-6 2.725419114659701e-5 -0.0001734228207430405 2.8044134588817553e-5 2.8668151636623035e-5 -7.505843671305004e-5 -0.0002375480325360175 -0.0002875950521490153 -0.00011733264539424001 -7.466007803367306e-5 8.555637038745822e-5 5.5537708314034874e-5 0.00014043715061353668 7.69394756128898e-5 0.00014276082954104035 -7.485971575869152e-5 3.314708016779633e-5 -9.59399403006389e-5 -7.424662174224403e-5 8.785104734791074e-5 -4.980320584643474e-5; -9.199854510901844e-5 -0.00024000306265029188 4.109687316864863e-5 -4.7542796970296486e-5 -8.816875457193028e-6 0.0002077890839374628 7.860466134072651e-6 6.168563587758528e-5 0.00013104114797366984 -2.9988234096740654e-5 -2.749413041404722e-5 0.0001348183158499198 9.543181105451772e-5 -0.00012005508877867815 -5.034966860497914e-5 1.977145613106131e-5 -3.726525218174617e-5 4.085946251633898e-6 3.378156353770916e-5 1.3541768488869374e-5 0.00011325016559036582 3.395735431163909e-5 3.150532385385703e-5 -0.00010043331538857763 7.797467871033851e-5 0.00017457359528103169 -1.5623467412146784e-5 5.362543189600014e-5 -0.00012425462599443082 -1.9525932735486006e-5 4.419257849271984e-5 6.129337445068244e-5; -0.00019794775219478724 0.00023529045644837644 -4.594661298169121e-5 3.497101058174528e-5 -6.898531164818111e-6 4.9420538534566074e-5 5.349021082084834e-5 5.9407812632078533e-5 -0.00011716019001168527 2.3257898826220205e-6 0.00010682118660925566 -8.732576609899568e-5 0.00018497418274792557 -7.52874031880243e-5 6.642170116693722e-5 -8.35505145941473e-6 5.258292437220608e-5 -1.5172274497928558e-5 -0.00012801898922615992 6.694201944784054e-5 -0.0001372912513309467 -2.2952975296256756e-5 -2.278945542483876e-5 -6.40785212729435e-5 -0.0001295895628848218 -1.7842128027304344e-5 5.4231267103847656e-5 3.952052499672205e-5 -8.189394359787317e-5 -7.09591761099469e-5 -1.9373120476562817e-5 5.160932848385687e-5; -7.923409045649304e-5 -7.570196224301594e-6 9.442299435870841e-5 -5.665038386102682e-5 -6.4766496300919e-5 -6.323151662688528e-5 -0.00011693449393821139 7.148804818537372e-5 -4.897550731873056e-5 0.00010429941560398894 3.80019703032551e-5 -0.00014357596571532865 -3.4779666542064923e-5 -1.2792237406003304e-5 3.5681158103063064e-5 0.00010522481554910645 0.0001369163767111125 4.344698775352445e-5 -0.00019875751220131186 -7.381059892683991e-5 1.4466310155355671e-5 -1.8576185332829483e-5 5.008759783488835e-5 -2.6785566454393467e-5 -3.5839544373761774e-5 0.00012197666021787196 -0.00011532659461697469 8.403694321425882e-5 -0.00012407386727606787 0.0001001544189665124 -0.00011512875405347932 0.0001011474052819118; 0.0001233550239041024 2.634195995720673e-6 7.941997603407367e-5 0.00013125093676276848 -1.7911913230679333e-5 0.0001086862204166697 1.9192847618920505e-5 3.1744522063627435e-5 4.050228301562713e-5 2.5937554825901e-5 0.00022457709849473415 2.7966732093004386e-5 -7.639874536184391e-5 -6.800467301988405e-6 6.500496522416472e-5 1.1761580142006772e-5 -4.7107122439087137e-5 -4.3524440909899965e-5 2.3743544377026143e-5 -4.6770623951350115e-5 5.231614989239022e-5 4.867407472115042e-5 -1.3226475742017294e-5 -5.2329721141036216e-5 -3.8181981720228886e-5 -2.790873702257031e-5 3.097892033761677e-5 0.00010431526163952301 -1.661524845334837e-5 1.459899260876196e-5 -0.00011832902680259316 3.652251395704443e-6; -2.8260223285790273e-6 1.3295454745602193e-5 0.0001354552542483205 5.787880135302134e-5 -0.00013302088783404502 1.1837590217856658e-5 -0.0003152906837482448 0.00012097718998973196 -1.030193644566294e-5 -9.146448875127744e-5 -3.924535204588754e-5 7.469378314397726e-5 6.0473993552825355e-5 -0.00017798736817951032 1.3494582244101045e-5 -7.160398391589894e-5 -6.842214216759775e-5 0.00010432608823005876 -4.092454119983505e-5 9.437423713223206e-5 -1.8410285438783166e-5 9.450398200839809e-5 6.181862689971448e-5 0.00010917862081385706 0.00013304138255015307 -0.0001648593724081675 0.0001989364128252271 9.623538343419386e-5 0.00028123813089167386 4.863054627037491e-5 0.00021277014738130048 1.8765430549467068e-5; 0.00020731906530684744 5.127101300288545e-5 4.631925818515209e-5 -5.719830463519086e-5 -1.8642770588000753e-5 -9.590096764028852e-5 -3.187904699619065e-6 -6.806580953295295e-5 -5.867422899924916e-5 5.453620448364157e-5 0.0001321192202927102 2.669701913376518e-5 0.000256707887242131 0.00013054524148586346 -0.0001418282672925401 4.950718990997625e-5 -6.779352864711325e-5 0.00010117571488492138 1.6520928479197705e-5 -3.500987535104989e-5 8.299408633732433e-5 -7.481771629997701e-5 -3.979215856403858e-5 8.352328128651912e-5 0.00018409355376616799 -5.325607074653012e-5 -2.1330720333450102e-5 -0.00020594343826402825 -0.0001694114662636514 -8.223914941069638e-5 -8.586542847795008e-5 7.546131652318562e-5; 3.154109318293463e-5 3.536413597195021e-5 2.9559060483186072e-5 5.945782932703545e-5 -0.00012007268496072491 -2.040289409033149e-5 3.0148516611103858e-6 0.00011454023361457142 -2.1531335089642805e-5 9.529384870565706e-5 5.551667591807606e-5 -6.080294160037954e-5 0.00011015533410263506 -3.3877474645670614e-5 1.8245915914490362e-5 6.0835724714158276e-5 2.809915413981493e-5 -6.335871266180674e-5 -1.7919546091982902e-5 -5.4945065063297755e-6 -0.0002445260700302267 0.00015829563809111817 -0.00016907961370871138 -0.00011736551213525923 4.6799012768544025e-5 8.013727957168783e-5 -9.651309782621265e-5 -0.00019136307791322918 -0.00011032929004850627 3.544035526592878e-5 -9.168168734739605e-5 -7.021537139061442e-5; -0.00011151542757454656 -0.0001167365383431639 -7.134042725249435e-5 0.00011842121849019783 -9.905850792586131e-5 -3.730810321451422e-5 -0.00015447046943757196 1.6477537405902708e-7 -5.284708940473613e-5 -8.245295366089095e-5 2.1591883255837038e-5 5.02172139980526e-5 2.6484575487642894e-5 0.00019929420041569243 -0.00011476730041488437 -1.2005206640083375e-5 3.686661757944851e-5 -2.5805858326411852e-5 8.277047259411739e-6 1.4364566327798674e-5 0.00019493563987032264 3.649362288831361e-5 0.00010192503642008395 9.98576458235899e-5 4.289482122236741e-5 -0.0001397386634331515 -3.5242909957219376e-5 3.896550579645388e-5 -5.7157686837513434e-5 -4.940363332446033e-5 0.00021121956770491288 3.5752533863499285e-5; 0.00014716079534976827 -2.641145907561628e-5 0.0001752321965248829 -4.8743566694758174e-5 0.00014663913829266416 6.44407824800849e-5 5.106187531039724e-5 -0.00017406216862597297 7.325505215689046e-5 -3.0333516733809932e-5 -3.290620075873599e-5 -2.885398713275422e-5 -0.00010346820850842252 0.00010980521405130015 6.33054202260911e-5 0.00010580349557116111 0.00013098878629520027 -9.472940619324102e-6 -8.607256277320514e-6 -7.78908600843019e-5 -3.935075652421991e-5 -3.799489273675791e-5 -2.9534125460633956e-5 -4.671630322411254e-6 -2.3549777652255285e-5 -3.852119549280157e-5 -0.00021246919855926692 4.008846317298144e-6 0.00018809201505049565 -9.240325231090934e-5 0.00011848190264556968 -8.009321969454423e-5; -0.00012020967654630444 0.00016060745007611773 -0.00013262470123104556 8.184531804194882e-5 0.00010457436916671864 0.00010485894642092461 -0.00019663653905250167 3.495236057110264e-5 -0.0001772373517577329 -2.6318724061504816e-5 3.12207874832416e-5 -9.912189956035279e-5 -0.0001296721467350404 9.973453312900088e-5 -4.8081411785952726e-6 -0.0001229504715719054 2.4496882239097885e-5 3.486484535291516e-5 -0.0002326689146992544 8.925542248666441e-5 7.762544452466263e-5 5.531887714956414e-6 3.3539522397538664e-5 5.509565277511446e-5 -5.4877852684387734e-5 0.00011217357574982597 -1.2346334950529898e-6 0.00019266559581060106 4.035999415113797e-5 7.287408413153271e-5 9.475820012643837e-5 -6.576203407699251e-5], bias = [-3.984060427337718e-10, -3.6220861999069013e-11, -1.677651797705e-9, 5.59984816741856e-10, -2.480965476587697e-9, -9.850675824529835e-11, 9.052803104856694e-11, 4.035324779008351e-10, 2.2140641303782147e-10, -6.105066009507857e-10, 3.3048440391400475e-9, 2.352485609995227e-9, 4.08259142636652e-9, 3.8879226771262375e-9, -3.9063886060447805e-10, -7.384531561969631e-10, 1.6708489619959403e-9, -2.9316310508200342e-9, 1.1862644435950507e-9, 2.292981203171935e-9, -2.923163995862929e-9, -1.5642660290027388e-9, 1.646107344126006e-9, -7.414476326908213e-10, -9.056504248706273e-10, 2.556705254773653e-9, 3.1314715527015514e-9, 8.394313533953629e-10, -1.798337146897226e-9, 2.934988285790656e-10, 1.060143257221312e-9, 3.336628174683956e-10]), layer_4 = (weight = [-0.0007788534505924469 -0.0006685830389952254 -0.000688716987191034 -0.0006542395072238465 -0.0006149025421776231 -0.00081659508330541 -0.0006478916014476879 -0.000749967054858272 -0.0005892286392886316 -0.0007119907802648645 -0.0007500193367682273 -0.0006822095209065945 -0.0007797070160926858 -0.0007522449748653104 -0.0006594741368423112 -0.0007969439236190279 -0.0006549193992018953 -0.0006106590559730796 -0.0008090121770929926 -0.000620521836429977 -0.0006054688376952504 -0.0007318573278050192 -0.0006780007670647525 -0.000565240734261215 -0.000638054666204854 -0.0006129880701127028 -0.0006384380101355744 -0.0006894606397778792 -0.0007330832427160187 -0.0007676011839588111 -0.0008188282810970212 -0.0006903661698708907; 0.0003245993159796126 0.00031691102393929387 0.00025862815255747214 0.0002386690866586987 0.00042179008933688464 0.00028225686738572416 0.00039923915367428507 0.00019782936787099414 0.00026801874025249095 0.00029064249278658256 0.00021962347088651664 0.00016200238480021362 0.0002067543200442797 2.5395698882187308e-5 0.00017155234894268115 0.0003376355658037225 0.00019695894545278714 0.00027573378158736456 0.00020768779579119 0.0003904691802563027 8.741270119787773e-5 0.00018457219015582057 0.00015858274434813938 0.0003211215798211186 0.00027279834140029273 0.00035529054072876593 0.00014283905766451167 0.00024238950764526578 0.00016205058459968245 0.00022644538420962983 -4.9050983374899115e-5 0.0001550530385393652], bias = [-0.000685165866837486, 0.00024344589819855688]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
+
+    lines!(ax, losses; linewidth=4, alpha=0.75)
+    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
+
+    fig
+end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
+waveform_nn_trained = first(compute_waveform(
+    dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
+    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
+        alpha=0.5, strokewidth=2, markersize=12)
+
+    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
+        position=:lb)
+
+    fig
+end
    Appendix ​
    julia
    using InteractiveUtils
+InteractiveUtils.versioninfo()
+
+if @isdefined(MLDataDevices)
+    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
+        println()
+        CUDA.versioninfo()
+    end
+
+    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
+        println()
+        AMDGPU.versioninfo()
+    end
+end
    Julia Version 1.10.6
+Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
+Build Info:
+  Official https://julialang.org/ release
+Platform Info:
+  OS: Linux (x86_64-linux-gnu)
+  CPU: 128 × AMD EPYC 7502 32-Core Processor
+  WORD_SIZE: 64
+  LIBM: libopenlibm
+  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
+Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
+Environment:
+  JULIA_CPU_THREADS = 16
+  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
+  JULIA_PKG_SERVER = 
+  JULIA_NUM_THREADS = 16
+  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
+  JULIA_PKG_PRECOMPILE_AUTO = 0
+  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const H=h(e,[["render",R]]);export{L as __pageData,H as default};
diff --git a/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.D_HX1t4_.lean.js b/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.D_HX1t4_.lean.js
new file mode 100644
index 0000000000..e8c06c33be
--- /dev/null
+++ b/dev/assets/tutorials_advanced_1_GravitationalWaveForm.md.D_HX1t4_.lean.js
@@ -0,0 +1,286 @@
+import{_ as h,c as i,a2 as n,j as A,a as t,o as a}from"./chunks/framework.CCjWn1F9.js";const L=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),e={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},m={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},c={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},X={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},D={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function R(N,s,U,Z,j,w){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
+      OptimizationOptimJL, Printf, Random, SciMLSensitivity
+using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=t("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=t(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=t(", ")),A("mjx-container",r,[(a(),i("svg",d,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=t(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
+    M = m₁ + m₂
+    r₁ = m₂ / M .* path
+    r₂ = -m₁ / M .* path
+    return r₁, r₂
+end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=t("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",C,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
+    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
+
+    if size(soln, 1) == 2
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+
+        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
+        p, M, e = model_params
+    else
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+        p = soln[3, :]
+        e = soln[4, :]
+    end
+
+    r = p ./ (1 .+ e .* cos.(χ))
+    x = r .* cos.(ϕ)
+    y = r .* sin.(ϕ)
+
+    orbit = vcat(x', y')
+    return orbit
+end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
+    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
+    b = (v[3:end] .- v[1:(end - 2)]) / 2
+    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
+    return [a; b; c] / dt
+end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
+    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
+    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
+    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
+    return [a; b; c] / (dt^2)
+end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
+    x = orbit[1, :]
+    y = orbit[2, :]
+
+    Ixx = x .^ 2
+    Iyy = y .^ 2
+    Ixy = x .* y
+    trace = Ixx .+ Iyy
+
+    if component[1] == 1 && component[2] == 1
+        tmp = Ixx .- trace ./ 3
+    elseif component[1] == 2 && component[2] == 2
+        tmp = Iyy .- trace ./ 3
+    else
+        tmp = Ixy
+    end
+
+    return mass .* tmp
+end
+
+function h_22_quadrupole_components(dt, orbit, component, mass=1)
+    mtensor = orbit2tensor(orbit, component, mass)
+    mtensor_ddot = d2_dt2(mtensor, dt)
+    return 2 * mtensor_ddot
+end
+
+function h_22_quadrupole(dt, orbit, mass=1)
+    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
+    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
+    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
+    return h11, h12, h22
+end
+
+function h_22_strain_one_body(dt::T, orbit) where {T}
+    h11, h12, h22 = h_22_quadrupole(dt, orbit)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
+    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
+    h11 = h11_1 + h11_2
+    h12 = h12_1 + h12_2
+    h22 = h22_1 + h22_2
+    return h11, h12, h22
+end
+
+function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
+    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
+
+    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
+
+    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
+    @assert mass_ratio≤1 "mass_ratio must be <= 1"
+    @assert mass_ratio≥0 "mass_ratio must be non-negative"
+
+    orbit = soln2orbit(soln, model_params)
+    if mass_ratio > 0
+        m₂ = inv(T(1) + mass_ratio)
+        m₁ = mass_ratio * m₂
+
+        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
+        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
+    else
+        waveform = h_22_strain_one_body(dt, orbit)
+    end
+    return waveform
+end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",g,[(a(),i("svg",o,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=t("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=t(", ")),A("mjx-container",F,[(a(),i("svg",u,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=t(", and ")),A("mjx-container",T,[(a(),i("svg",m,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=t(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
+    χ, ϕ = u
+
+    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
+    denom = sqrt((p - 2)^2 - 4 * e^2)
+
+    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
+    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
+
+    return [χ̇, ϕ̇]
+end
+
+mass_ratio = 0.0         # test particle
+u0 = Float64[π, 0.0]     # initial conditions
+datasize = 250
+tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
+tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
+dt_data = tsteps[2] - tsteps[1]
+dt = 100.0
+const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
+soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
+waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
+
+    axislegend(ax, [[l, s]], ["Waveform Data"])
+
+    fig
+end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
+    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
+ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[0.000101433325; -0.00014970472; -3.5847308f-5; -3.0095046f-6; -5.837021f-5; -5.530635f-5; -0.00014913769; -8.2657076f-5; 1.8703817f-6; 2.8848355f-5; 1.4417893f-5; 4.5086595f-5; 4.1238396f-5; 5.863067f-5; 0.000118168115; -2.7185877f-5; -2.2295013f-5; 0.00021102077; -2.7826984f-5; -7.046538f-5; 3.6819965f-5; -0.000122086; 4.216995f-5; 3.7783706f-5; -0.00029299365; 0.00012795332; 2.003964f-5; -7.480524f-5; -1.2783274f-5; 8.586675f-5; -0.00016691082; -0.00014107962;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-5.25561f-5 5.0319617f-5 0.000121188066 -0.0001252668 -0.00012370043 1.3147543f-5 6.7192734f-5 0.00028501067 -4.615136f-5 5.010297f-5 0.0001109413 0.00012366676 0.00012497939 -5.900952f-6 1.5949308f-5 -0.00010800126 -0.00019976492 8.1667444f-5 -0.00020031813 9.338467f-5 -6.742277f-5 4.060099f-5 -8.8431494f-5 -9.7043405f-5 4.6010473f-6 -0.0001405853 5.4320717f-5 8.4588464f-5 -7.09648f-5 -8.0628546f-5 6.95526f-5 -9.106946f-5; 5.6481567f-5 -3.276788f-5 -0.0001785962 -9.946028f-5 -0.00015278214 -6.958734f-5 -8.3435276f-5 6.671795f-5 0.00021108559 3.278153f-5 0.000119060285 5.4152362f-5 6.264283f-5 7.926161f-5 -0.00019341416 -6.553496f-6 7.6160606f-5 -9.66632f-5 -9.5061914f-5 -4.2958072f-5 0.00013052954 -0.00019412015 8.9826804f-5 7.1718045f-5 0.00019083949 6.192604f-6 -3.821931f-6 4.3647802f-7 -0.00014268934 5.3210428f-5 8.25371f-5 -1.2173807f-6; 4.5715973f-5 -5.495184f-5 -0.00011186839 -5.385712f-5 2.0592182f-5 -0.00011375335 -0.00012681524 -5.3366508f-5 8.4260675f-5 -0.000104376966 -9.465132f-5 9.176428f-5 -0.00019846226 4.180067f-5 2.441713f-5 -2.2916829f-5 -3.6965794f-5 8.515369f-5 -4.7670128f-5 -2.4597295f-5 -0.000115552284 5.7741276f-5 -2.149032f-5 0.0001659628 -3.8053244f-5 9.7449265f-5 1.5938956f-5 0.0001071606 0.00015616491 -0.0002005834 -2.687522f-5 1.2993707f-5; 5.1912615f-5 -9.1409915f-5 4.7006324f-5 0.0001274501 0.00010122003 3.093884f-5 4.6041696f-5 -2.819437f-5 7.968095f-6 5.1355753f-5 -9.012398f-5 -0.00016678765 4.6333556f-5 -1.850147f-5 -0.00011283306 2.4248398f-5 -0.0001531043 -0.00020931265 0.00019833563 5.7953574f-5 -8.870693f-5 -9.257044f-5 2.1976448f-5 -3.2718603f-5 -5.585929f-5 7.810437f-5 0.00022372045 3.585174f-5 -2.6321854f-5 9.45166f-5 0.00018568635 -0.000118498785; -8.421658f-5 0.00017808175 0.00015194518 -0.000113057125 -0.00012066419 -0.00018935993 -1.0009757f-5 -6.276838f-5 -2.1114492f-5 -1.8010991f-5 -0.00012293975 0.00010749552 4.4360055f-5 1.2918086f-5 -0.00012972708 1.2805929f-5 0.00014863386 -0.00019670448 -0.00028618306 -8.474048f-5 4.8318067f-5 -0.00013781365 -2.8484217f-5 1.2321886f-5 3.064775f-5 -6.4827924f-5 3.7555023f-6 1.5734184f-5 2.1832733f-5 0.0001064967 -3.4004093f-5 0.00016090772; -4.9504306f-5 1.5704822f-5 9.3366885f-5 6.956209f-5 -5.8176258f-5 -3.641004f-5 1.0974424f-5 -3.3011274f-5 6.968066f-5 -0.00012221691 -2.241614f-5 -0.00021536481 9.290237f-5 -4.6644625f-5 -1.2897161f-5 0.00013195348 -0.00010431149 -0.00011168598 9.9419194f-5 -0.00010479866 -2.6149884f-5 4.048816f-5 -5.4505414f-5 9.3042545f-5 -3.5675401f-7 3.167108f-5 0.00010144765 -2.0471824f-5 7.499174f-5 0.00012745548 -1.9343881f-5 -4.0860472f-5; 3.480437f-5 7.530175f-5 -0.00013266527 0.00024938013 0.00016262576 -8.980506f-5 7.534006f-6 7.878023f-5 4.098693f-5 7.753387f-7 6.413197f-5 -4.2840365f-5 -0.00011310589 4.09235f-5 -4.548707f-5 -4.3824835f-5 -3.7166952f-5 -0.0001460987 -8.8912086f-5 -4.724518f-5 -6.30762f-5 0.0001248795 -2.1304417f-5 0.000109929126 9.500971f-5 -6.1527586f-5 3.875689f-5 -1.0796941f-5 -8.256911f-5 3.3877268f-6 2.0451178f-5 -9.746762f-5; -8.7214146f-5 0.00010691131 -7.3489726f-5 -0.00017095437 1.7527753f-5 8.161695f-5 -8.896779f-5 1.6436641f-5 0.00014055088 3.1515363f-5 -8.762248f-5 -3.935946f-5 -0.00022448925 0.00015336409 1.1825823f-5 0.00012907668 -0.00011167856 0.00010605673 -1.8883887f-6 6.4180625f-5 -0.00012380801 -2.5067176f-5 4.1593237f-5 5.507989f-5 0.00016835274 0.00010800524 -5.2248088f-5 -2.9934452f-5 5.9419897f-5 7.210349f-5 -9.05596f-5 -4.981542f-5; -8.3190884f-5 -0.00013716334 9.333262f-5 3.0901905f-5 0.00018078604 3.6113823f-5 -2.474314f-5 -1.1500239f-5 -2.5257072f-5 -7.240517f-5 -7.065769f-5 -0.0002254977 1.833079f-5 -7.38984f-5 8.5560125f-5 0.0001946904 8.675337f-5 6.741229f-5 -6.8029505f-5 -5.6107692f-5 -0.00012546312 -9.770373f-6 -3.158323f-5 -2.5108455f-5 0.00012979835 5.185795f-5 -3.5403886f-5 0.00020563042 -0.00017649106 9.987908f-5 8.9251225f-5 -6.09851f-5; 0.00015459415 -1.9436993f-5 3.1855652f-5 -2.8804572f-5 0.00010122694 4.723484f-5 1.5064943f-6 5.0922765f-5 1.0278612f-6 -4.2696785f-5 -5.4365966f-5 -0.00020424122 5.340165f-6 -5.1149924f-5 5.5771772f-5 -0.00011463864 -1.7955612f-5 -3.0032486f-5 -0.00012965355 1.8046007f-5 -0.00013679774 2.9808103f-5 4.8935217f-5 -3.587022f-5 -5.076042f-5 -1.201444f-5 -4.7412987f-6 0.00022637536 6.3304215f-5 -1.0715298f-5 -1.32741f-5 -3.9808005f-5; -0.00017152919 8.569154f-5 -0.00015427994 0.0001518926 -7.394991f-5 6.4593505f-6 0.00010508767 0.0001652748 -0.00011657923 8.97161f-6 3.1926393f-5 -0.00014740185 3.5467187f-5 0.00016369366 0.00010648671 0.000103018654 0.00016443468 1.6788868f-5 6.344208f-5 2.0847823f-5 9.915987f-6 0.00013888412 -0.00020764141 0.00014734587 3.127726f-5 0.0001764748 6.793151f-5 -2.079785f-5 -7.642907f-5 0.00010638243 4.9633407f-5 -0.00011511093; 4.7663376f-5 5.5237564f-5 0.00010843301 -5.452355f-5 3.5808926f-6 -3.5294324f-5 2.4117173f-6 0.00012329305 -0.0001269051 -7.7190236f-7 -0.000119915334 -6.727885f-5 0.0002076928 -6.7405424f-5 -4.3333755f-5 5.2977874f-5 -0.00011457315 0.00019208078 -5.5476045f-5 -5.8278398f-5 9.9641446f-5 7.6247874f-5 -3.2882388f-5 2.0014795f-5 5.4410673f-5 2.1790565f-5 6.1696584f-5 8.6672895f-5 0.00012334628 1.748379f-5 0.0001684268 -0.00013439129; 6.154388f-6 -0.000103090264 1.8575416f-5 4.248712f-5 -2.0510362f-5 2.7966196f-5 -4.1805106f-6 0.000100216464 -3.9296287f-5 0.00015753518 -0.00010629292 0.00015053408 5.062221f-6 0.00017994971 2.6507929f-5 6.469896f-5 3.0912197f-5 -3.787202f-5 -9.180992f-5 9.380597f-6 1.7937424f-5 0.0001484272 0.00014581943 -0.00015441813 2.5858413f-5 0.00018289514 5.500523f-5 -1.8279512f-5 0.00018384538 -2.7853945f-5 1.1434443f-5 9.653837f-5; 1.1493156f-5 0.00012901722 -0.00022245743 0.00010968725 -5.9707843f-5 -5.9917904f-5 -3.61098f-5 8.152264f-5 -7.820346f-5 -3.3505003f-5 -3.7833874f-5 0.00011963542 -0.000101400816 -2.5519417f-5 -3.9913408f-5 -1.4419071f-5 1.0952946f-5 3.0841573f-5 -3.0217423f-5 -8.11942f-5 0.00012752488 0.00019579084 -7.06113f-6 0.00011654196 0.00023443296 0.000111647285 0.00014851333 4.3228825f-5 0.00012218115 0.00017196768 -2.9187097f-5 0.00010714686; 2.3869788f-5 -2.8650662f-5 0.00021611001 -1.7511142f-5 2.731428f-5 7.015406f-5 0.00011499756 3.0493577f-5 -1.3791175f-5 -9.098295f-5 -8.075956f-5 6.571922f-5 -4.6558005f-5 3.8361726f-5 -6.263231f-5 9.8436394f-5 -5.444673f-5 -0.0001525929 -2.8079692f-5 0.00020451599 -0.00011710578 8.0096115f-6 1.7739687f-5 -0.00017146657 -1.1216441f-5 5.582888f-5 2.102432f-5 -3.2097083f-5 -5.654088f-5 -0.0002029514 0.00012005623 -4.6535497f-5; 4.0600895f-5 -0.00016371673 -0.00017366491 2.4281542f-5 3.0944266f-5 9.9873614f-5 -2.5847257f-5 0.00020838533 -8.538478f-5 -4.1817697f-5 0.00021643919 1.0913612f-5 -0.00013417867 -0.00011062858 -5.303284f-5 -1.5524723f-6 -2.2601067f-5 3.064764f-5 -0.00012356581 9.590809f-5 0.00014010115 5.2191863f-6 -0.0001268768 -5.8050922f-5 -8.722839f-5 -7.693819f-5 -9.255185f-5 0.00025757347 3.3973716f-5 -0.00011420436 -3.4002467f-5 0.00013258631; 3.1190964f-5 0.00014198378 -0.000109832414 3.0770072f-5 -9.37026f-5 -6.4940985f-5 -6.932041f-6 0.00013815345 0.00014333393 -6.7661975f-5 -4.511986f-5 -2.4029983f-5 -5.3136577f-5 0.00026887178 -0.00015120419 6.6838904f-5 0.00013478157 0.00014922114 2.1898917f-5 -9.330819f-5 2.4437604f-5 -3.936817f-5 -0.00010707723 -1.4017039f-5 6.4641405f-5 4.3191016f-5 7.486379f-5 -9.5469855f-5 -1.4938067f-5 0.00012933458 9.605733f-5 -0.00014525898; 0.00016662048 9.602477f-6 -5.0595074f-5 -0.00010192549 -0.00010132676 -7.178766f-5 -4.139689f-5 6.0371243f-5 -1.2646691f-5 -0.00015618675 -4.537753f-6 0.00019719187 1.20949235f-5 -0.00019831482 -0.00012459549 -3.3654065f-5 1.3041183f-6 8.898287f-5 -4.7517875f-5 -9.4529514f-7 -2.2393571f-5 -7.230482f-5 -5.2415697f-5 0.00015502863 -1.2729995f-5 -3.9127535f-5 -6.104951f-5 -2.6234753f-5 -0.00013506036 -5.3432243f-5 -0.000111437635 8.188779f-5; 9.936778f-5 9.609397f-5 1.8602841f-5 8.174792f-5 3.0406422f-5 5.6035467f-5 2.6022355f-5 8.32676f-5 -0.00011590395 0.0001480582 3.0665713f-5 -8.2545f-5 -3.1364634f-5 3.6824866f-5 6.449255f-5 5.598043f-5 0.00011874976 0.00021320413 -9.1623195f-5 6.0359876f-6 -1.1393765f-5 -0.00022938936 -9.551152f-5 6.278853f-5 0.00015201893 4.6772777f-5 3.4416792f-6 -5.5460823f-5 -3.434725f-5 -0.0001919227 -0.00016547991 -8.913108f-6; -1.0548262f-5 -0.00022290568 6.221811f-5 -3.8161495f-5 -3.203011f-5 5.9161786f-5 2.3230212f-5 3.1728236f-5 8.554471f-5 1.3226349f-5 0.00014533396 7.015475f-5 0.00011469073 8.4968866f-5 0.000119700926 3.688433f-5 0.000105032515 -0.0001410187 4.3452623f-5 -0.00012144251 4.901844f-6 4.2582407f-5 0.0001310712 2.195234f-5 8.650805f-5 4.625665f-5 -7.943035f-5 -7.426819f-5 -9.102839f-5 -4.7004334f-5 6.47525f-5 6.314009f-5; -4.3685573f-5 -7.0061484f-5 -8.564832f-5 -2.5506959f-5 -3.3139113f-5 -0.00012296431 5.0306004f-5 -2.598046f-5 3.1052587f-5 -2.203405f-5 -2.4192013f-5 -1.2408228f-7 0.00018310527 8.597894f-5 -0.00016243967 -8.256807f-5 4.5036827f-6 -4.9110735f-5 -9.529562f-5 0.00011435464 1.8392873f-5 1.6941232f-7 -5.0154013f-5 7.089302f-5 2.9975381f-5 -5.9508693f-5 -2.4033567f-5 0.00012779109 -0.00018340634 -6.473091f-5 -0.00015620314 -8.254381f-5; -2.5668716f-5 4.8003552f-5 -3.2946562f-5 4.2170883f-5 8.5557236f-5 0.0001284341 -5.9376423f-5 -2.011942f-5 -3.5316505f-5 4.182381f-5 -2.0610467f-5 -8.933989f-6 2.7255755f-5 -0.00017342126 2.8045699f-5 2.8669716f-5 -7.505687f-5 -0.00023754647 -0.0002875935 -0.00011733108 -7.4658514f-5 8.5557935f-5 5.5539273f-5 0.00014043871 7.694104f-5 0.0001427624 -7.485815f-5 3.3148644f-5 -9.5938376f-5 -7.424506f-5 8.785261f-5 -4.980164f-5; -9.200019f-5 -0.00024000471 4.1095227f-5 -4.7544443f-5 -8.818522f-6 0.00020778744 7.85882f-6 6.168399f-5 0.0001310395 -2.998988f-5 -2.7495777f-5 0.00013481667 9.5430165f-5 -0.000120056735 -5.0351315f-5 1.976981f-5 -3.72669f-5 4.0843f-6 3.3779917f-5 1.3540122f-5 0.00011324852 3.395571f-5 3.1503678f-5 -0.00010043496 7.797303f-5 0.00017457195 -1.5625114f-5 5.3623786f-5 -0.00012425627 -1.9527579f-5 4.4190932f-5 6.129173f-5; -0.00019794701 0.0002352912 -4.594587f-5 3.4971752f-5 -6.8977897f-6 4.942128f-5 5.3490952f-5 5.9408554f-5 -0.00011715945 2.3265313f-6 0.00010682193 -8.7325025f-5 0.00018497492 -7.528666f-5 6.642244f-5 -8.35431f-6 5.2583666f-5 -1.5171533f-5 -0.00012801825 6.694276f-5 -0.00013729051 -2.2952234f-5 -2.2788714f-5 -6.407778f-5 -0.00012958882 -1.7841387f-5 5.423201f-5 3.9521266f-5 -8.18932f-5 -7.0958435f-5 -1.9372379f-5 5.161007f-5; -7.9233185f-5 -7.5692906f-6 9.44239f-5 -5.664948f-5 -6.476559f-5 -6.323061f-5 -0.00011693359 7.1488954f-5 -4.89746f-5 0.00010430032 3.8002876f-5 -0.00014357506 -3.477876f-5 -1.2791332f-5 3.5682064f-5 0.00010522572 0.00013691728 4.3447893f-5 -0.0001987566 -7.380969f-5 1.4467216f-5 -1.857528f-5 5.0088503f-5 -2.678466f-5 -3.583864f-5 0.000121977566 -0.00011532569 8.403785f-5 -0.00012407296 0.000100155325 -0.00011512785 0.00010114831; 0.00012335247 2.6316393f-6 7.941742f-5 0.00013124838 -1.791447f-5 0.000108683664 1.9190291f-5 3.1741965f-5 4.0499726f-5 2.5934998f-5 0.00022457454 2.7964175f-5 -7.64013f-5 -6.803024f-6 6.500241f-5 1.1759023f-5 -4.710968f-5 -4.3526998f-5 2.3740988f-5 -4.677318f-5 5.2313593f-5 4.8671518f-5 -1.3229032f-5 -5.2332278f-5 -3.818454f-5 -2.7911294f-5 3.0976364f-5 0.000104312705 -1.6617805f-5 1.4596436f-5 -0.00011833158 3.6496947f-6; -2.8291538f-6 1.3292323f-5 0.00013545212 5.787567f-5 -0.00013302402 1.1834459f-5 -0.00031529382 0.00012097406 -1.0305068f-5 -9.146762f-5 -3.9248484f-5 7.469065f-5 6.0470862f-5 -0.0001779905 1.3491451f-5 -7.1607115f-5 -6.8425274f-5 0.00010432296 -4.0927673f-5 9.4371106f-5 -1.8413417f-5 9.450085f-5 6.1815495f-5 0.00010917549 0.00013303825 -0.0001648625 0.00019893328 9.623225f-5 0.000281235 4.8627415f-5 0.00021276702 1.8762299f-5; 0.00020731823 5.1270174f-5 4.631842f-5 -5.7199144f-5 -1.864361f-5 -9.590181f-5 -3.1887441f-6 -6.806665f-5 -5.867507f-5 5.4535365f-5 0.00013211838 2.669618f-5 0.00025670705 0.0001305444 -0.0001418291 4.950635f-5 -6.779437f-5 0.000101174875 1.6520089f-5 -3.5010715f-5 8.299325f-5 -7.4818556f-5 -3.9792998f-5 8.352244f-5 0.00018409271 -5.325691f-5 -2.133156f-5 -0.00020594428 -0.0001694123 -8.223999f-5 -8.586627f-5 7.546048f-5; 3.154289f-5 3.5365934f-5 2.9560859f-5 5.9459628f-5 -0.00012007089 -2.0401096f-5 3.01665f-6 0.00011454203 -2.1529537f-5 9.529565f-5 5.5518474f-5 -6.0801143f-5 0.00011015713 -3.3875676f-5 1.8247714f-5 6.0837523f-5 2.8100952f-5 -6.3356914f-5 -1.7917748f-5 -5.492708f-6 -0.00024452427 0.00015829744 -0.00016907782 -0.000117363714 4.680081f-5 8.013908f-5 -9.65113f-5 -0.00019136128 -0.00011032749 3.5442154f-5 -9.167989f-5 -7.021357f-5; -0.00011151572 -0.00011673683 -7.134072f-5 0.000118420925 -9.90588f-5 -3.7308397f-5 -0.00015447076 1.6448188f-7 -5.2847383f-5 -8.245325f-5 2.159159f-5 5.021692f-5 2.6484282f-5 0.0001992939 -0.000114767594 -1.20055f-5 3.6866324f-5 -2.5806152f-5 8.276754f-6 1.4364273f-5 0.00019493535 3.649333f-5 0.00010192474 9.985735f-5 4.2894528f-5 -0.00013973896 -3.5243203f-5 3.8965212f-5 -5.715798f-5 -4.9403927f-5 0.00021121927 3.575224f-5; 0.00014715974 -2.641252f-5 0.00017523114 -4.8744627f-5 0.00014663808 6.443972f-5 5.1060815f-5 -0.00017406323 7.325399f-5 -3.0334577f-5 -3.290726f-5 -2.8855047f-5 -0.00010346927 0.000109804154 6.330436f-5 0.000105802435 0.00013098773 -9.474001f-6 -8.608316f-6 -7.789192f-5 -3.9351817f-5 -3.7995953f-5 -2.9535186f-5 -4.6726905f-6 -2.3550838f-5 -3.8522256f-5 -0.00021247026 4.007786f-6 0.00018809095 -9.240431f-5 0.00011848084 -8.009428f-5; -0.00012021001 0.00016060712 -0.00013262503 8.1844984f-5 0.000104574036 0.00010485861 -0.00019663687 3.4952027f-5 -0.00017723769 -2.6319058f-5 3.1220454f-5 -9.912223f-5 -0.00012967248 9.97342f-5 -4.808475f-6 -0.0001229508 2.4496549f-5 3.486451f-5 -0.00023266925 8.925509f-5 7.762511f-5 5.531554f-6 3.353919f-5 5.509532f-5 -5.4878186f-5 0.00011217324 -1.2349672f-6 0.00019266526 4.035966f-5 7.287375f-5 9.475787f-5 -6.576237f-5], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-9.368759f-5 1.6582828f-5 -3.5511866f-6 3.0926352f-5 7.026318f-5 -0.00013142922 3.7274265f-5 -6.480119f-5 9.593723f-5 -2.6824922f-5 -6.485373f-5 2.9562175f-6 -9.454155f-5 -6.707946f-5 2.5691726f-5 -0.00011177807 3.0246403f-5 7.450661f-5 -0.00012384634 6.464391f-5 7.969684f-5 -4.669152f-5 7.165037f-6 0.00011992512 4.711118f-5 7.2177645f-5 4.6727633f-5 -4.2947895f-6 -4.7917452f-5 -8.243532f-5 -0.00013366244 -5.2003056f-6; 8.115342f-5 7.3465126f-5 1.5182278f-5 -4.776809f-6 0.00017834424 3.881097f-5 0.00015579326 -4.561653f-5 2.4572842f-5 4.7196598f-5 -2.3822335f-5 -8.144347f-5 -3.6691436f-5 -0.00021805007 -7.189355f-5 9.418967f-5 -4.648693f-5 3.2287953f-5 -3.575809f-5 0.00014702333 -0.00015603313 -5.8873688f-5 -8.486313f-5 7.7675686f-5 2.935245f-5 0.000111844696 -0.00010060676 -1.0563847f-6 -8.139529f-5 -1.7000513f-5 -0.00029249687 -8.839286f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
+
+const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
+    Chain(
+        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
+        layer_2 = Dense(1 => 32, cos),  # 64 parameters
+        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
+        layer_4 = Dense(32 => 2),       # 66 parameters
+    ),
+)         # Total: 1_186 parameters,
+          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",c,[(a(),i("svg",B,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",z,[(a(),i("svg",V,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=t("where, ")),A("mjx-container",q,[(a(),i("svg",b,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=t(", ")),A("mjx-container",X,[(a(),i("svg",D,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=t(", and ")),A("mjx-container",P,[(a(),i("svg",K,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=t(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
+    χ, ϕ = u
+    p, M, e = ode_model_params
+
+    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
+    # it, however, in general, we should use \`st\` to store the state of the neural network.
+    y = 1 .+ nn_model([first(u)], nn_params)
+
+    numer = (1 + e * cos(χ))^2
+    denom = M * (p^(3 / 2))
+
+    χ̇ = (numer / denom) * y[1]
+    ϕ̇ = (numer / denom) * y[2]
+
+    return [χ̇, ϕ̇]
+end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
+waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    axislegend(ax, [[l1, s1], [l2, s2]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
+
+    fig
+end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
+
+function loss(θ)
+    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
+    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
+    return mseloss(pred_waveform, waveform)
+end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0006994921614618403
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
+
+function callback(θ, l)
+    push!(losses, l)
+    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
+    return false
+end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
+optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
+optprob = Optimization.OptimizationProblem(optf, params)
+res = Optimization.solve(
+    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
+    callback, maxiters=1000)
    retcode: Success
+u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [0.00010143332474399266; -0.00014970471966083986; -3.584730802686131e-5; -3.0095045531156137e-6; -5.837020944455422e-5; -5.530635098688162e-5; -0.00014913768973205503; -8.265707583622401e-5; 1.870381652225233e-6; 2.8848355213978774e-5; 1.4417892998606726e-5; 4.508659549173389e-5; 4.123839607928261e-5; 5.863067053716329e-5; 0.00011816811456800735; -2.7185877115678053e-5; -2.229501296823788e-5; 0.0002110207715302368; -2.7826983568923675e-5; -7.046537939462137e-5; 3.681996531663573e-5; -0.00012208599946444672; 4.2169951484519065e-5; 3.778370592044982e-5; -0.00029299364541612773; 0.00012795331713263893; 2.003964073079977e-5; -7.480524072890363e-5; -1.278327363249883e-5; 8.586674812241179e-5; -0.00016691081691508447; -0.0001410796248819983;;], bias = [7.6713176236549e-19, 3.696847498061422e-17, 2.4055689200670786e-18, -4.74888813316211e-18, 8.733758721019089e-18, -6.464424972716262e-17, 4.685057209353382e-17, -1.141839913481628e-16, -5.842869798050569e-19, 1.8510923759576366e-17, 1.3679823189400309e-18, 6.669020875018921e-18, -4.063228054300107e-18, 8.877328713490479e-17, 1.1229820843349717e-16, -2.356901393761471e-17, -6.315824425403808e-18, 2.5453541817767217e-16, -3.7526064434901786e-17, 3.5326060372288575e-17, 5.257347495428192e-17, -1.9470226826847666e-16, 2.2141221075579763e-17, -3.109513202642436e-18, -5.151280825108199e-16, 1.6121381886129855e-16, 3.3708327975928686e-17, 1.2404157686012208e-17, -2.664045442200898e-17, 1.0050086139984667e-16, -2.8553122257375194e-16, 3.2154322768126957e-17]), layer_3 = (weight = [-5.25564992604587e-5 5.03192185969298e-5 0.00012118766785344962 -0.000125267199157021 -0.00012370083100183914 1.3147144225444096e-5 6.719233528922508e-5 0.00028501027004306206 -4.615175753504133e-5 5.010257332098586e-5 0.00011094090036560689 0.00012366636105791602 0.0001249789874672603 -5.901350552518186e-6 1.5948909956864962e-5 -0.00010800165732878346 -0.00019976532171501866 8.166704533135075e-5 -0.0002003185273243402 9.338426930109939e-5 -6.742316877723749e-5 4.060059104442697e-5 -8.84318924837101e-5 -9.704380322734901e-5 4.6006488882410714e-6 -0.00014058569472770083 5.4320318620669685e-5 8.458806582705095e-5 -7.096519953127111e-5 -8.062894453548825e-5 6.955220482989485e-5 -9.106985640034847e-5; 5.648153107789875e-5 -3.276791524081877e-5 -0.00017859623023947177 -9.946031771074462e-5 -0.0001527821803622454 -6.958737487844602e-5 -8.343531210380995e-5 6.671791213624706e-5 0.000211085549540022 3.278149249557925e-5 0.0001190602489939384 5.4152326060594614e-5 6.264279379569514e-5 7.926157575160787e-5 -0.00019341419813395872 -6.553532171218581e-6 7.616056989240049e-5 -9.666323587703881e-5 -9.506195040126744e-5 -4.2958108520232484e-5 0.0001305295083282009 -0.0001941201843012629 8.982676824847756e-5 7.171800841637475e-5 0.000190839455330881 6.192567684357778e-6 -3.821967152597545e-6 4.3644179731121464e-7 -0.0001426893794134766 5.32103915017989e-5 8.253706635036125e-5 -1.2174169344313458e-6; 4.57142954495314e-5 -5.4953518517909694e-5 -0.00011187006439073063 -5.385879612518597e-5 2.0590504757222437e-5 -0.0001137550249019633 -0.00012681691859838386 -5.336818557921545e-5 8.42589974148187e-5 -0.00010437864384371862 -9.465299499211943e-5 9.176260519083579e-5 -0.0001984639426133598 4.179899351797133e-5 2.4415452028142198e-5 -2.291850669067434e-5 -3.696747134896089e-5 8.515201207261988e-5 -4.76718056212653e-5 -2.4598972775180253e-5 -0.0001155539617663254 5.773959801658752e-5 -2.1491996992752195e-5 0.00016596112564215394 -3.80549214220148e-5 9.74475873611945e-5 1.5937278842412807e-5 0.00010715892440416103 0.00015616323380740252 -0.00020058507343278912 -2.6876896843207e-5 1.2992029407862942e-5; 5.19131751478338e-5 -9.140935461817854e-5 4.700688415124854e-5 0.00012745065733708302 0.00010122059003134992 3.093940150376217e-5 4.6042255880650304e-5 -2.8193810330943946e-5 7.968654890480487e-6 5.1356313369812806e-5 -9.012341642135895e-5 -0.00016678708952002404 4.633411636843179e-5 -1.8500910564565225e-5 -0.00011283250231843832 2.4248958388686553e-5 -0.00015310373445589243 -0.0002093120931842273 0.00019833619418006238 5.7954133544459536e-5 -8.870637274429363e-5 -9.256987713359704e-5 2.1977007881848314e-5 -3.271804261219849e-5 -5.58587310390065e-5 7.810493089874772e-5 0.00022372100621379767 3.5852299069862294e-5 -2.6321293526218726e-5 9.451716296991064e-5 0.0001856869127639691 -0.0001184982250289855; -8.421906372894266e-5 0.0001780792737459982 0.00015194269521963324 -0.0001130596063834579 -0.00012066667100078717 -0.00018936240653637186 -1.0012237771997909e-5 -6.27708576306609e-5 -2.1116973448797237e-5 -1.8013471936127532e-5 -0.00012294222674462564 0.00010749304069541923 4.4357573778699496e-5 1.291560524770234e-5 -0.00012972956014925983 1.2803448180066917e-5 0.00014863137780558722 -0.00019670696187890814 -0.0002861855390168684 -8.474296178099866e-5 4.831558558145079e-5 -0.00013781613220123864 -2.84866981477426e-5 1.2319405095342203e-5 3.064526767928963e-5 -6.483040474483915e-5 3.7530213827364957e-6 1.5731702796634066e-5 2.1830252216760097e-5 0.00010649421906202128 -3.400657428789124e-5 0.0001609052343607146; -4.950440444129138e-5 1.5704723128396553e-5 9.336678675997424e-5 6.956199257790459e-5 -5.817635638716272e-5 -3.641014000943504e-5 1.0974325678648426e-5 -3.3011372860824714e-5 6.968056158318532e-5 -0.0001222170070007688 -2.2416237979383453e-5 -0.00021536490777118708 9.290227507396954e-5 -4.6644723906252673e-5 -1.289725906583554e-5 0.00013195338005123078 -0.00010431158835610957 -0.00011168607698830286 9.941909565047711e-5 -0.0001047987573741247 -2.6149982940593018e-5 4.0488062668986844e-5 -5.450551207952779e-5 9.304244639740679e-5 -3.5685252152966485e-7 3.167098083456447e-5 0.00010144755259777768 -2.0471922024922518e-5 7.499163956770091e-5 0.0001274553830541428 -1.934397942870884e-5 -4.0860570433619975e-5; 3.480445915249443e-5 7.530184260451466e-5 -0.00013266518041143833 0.00024938021991749594 0.00016262584862181272 -8.980496610399982e-5 7.534096345463743e-6 7.878031786878868e-5 4.098702223620674e-5 7.754292474808159e-7 6.413206168565263e-5 -4.28402749670361e-5 -0.00011310579835788056 4.0923590437726185e-5 -4.548698094675063e-5 -4.382474477404435e-5 -3.7166861569291764e-5 -0.00014609860633152372 -8.891199510196862e-5 -4.724508877495954e-5 -6.307611007594211e-5 0.00012487959205520594 -2.1304326276892678e-5 0.00010992921628405631 9.500979971255219e-5 -6.152749525733976e-5 3.875698124522638e-5 -1.0796850313797592e-5 -8.256901898073669e-5 3.3878173204562125e-6 2.0451268516464432e-5 -9.746753172601335e-5; -8.72137428008044e-5 0.00010691171672865347 -7.348932223189744e-5 -0.00017095396861927688 1.7528156321214866e-5 8.161735212402272e-5 -8.896738682901885e-5 1.6437044622444454e-5 0.0001405512845746231 3.151576646692004e-5 -8.762207681806867e-5 -3.9359056528472726e-5 -0.00022448884948184845 0.00015336449331575854 1.1826226626589792e-5 0.00012907708486514048 -0.00011167815347230175 0.00010605713367899186 -1.8879852053473542e-6 6.418102901866489e-5 -0.00012380761137260248 -2.5066772933633293e-5 4.159364078864327e-5 5.508029397665399e-5 0.00016835314062394827 0.00010800564240403152 -5.224768420825822e-5 -2.9934048426502315e-5 5.942030083010798e-5 7.210389202432504e-5 -9.055919900797779e-5 -4.981501507368307e-5; -8.31906623362331e-5 -0.00013716311928972227 9.333284418874019e-5 3.090212652653063e-5 0.0001807862603881634 3.611404412270109e-5 -2.474291846836073e-5 -1.1500018045876406e-5 -2.525685027746876e-5 -7.240495011599202e-5 -7.06574688238783e-5 -0.00022549747933323905 1.8331010723091334e-5 -7.389818034523583e-5 8.556034612313352e-5 0.0001946906153888681 8.675358861994533e-5 6.741251439864033e-5 -6.802928379427108e-5 -5.610747081034315e-5 -0.00012546289923291126 -9.770151847147264e-6 -3.1583008022675615e-5 -2.5108233386230962e-5 0.00012979857587098995 5.185817229987542e-5 -3.540366429758654e-5 0.0002056306433941474 -0.00017649083434760035 9.987929974060993e-5 8.925144669676538e-5 -6.098487989318162e-5; 0.00015459353811902082 -1.9437603677126076e-5 3.18550416894993e-5 -2.880518264565269e-5 0.00010122633169966698 4.723422966062034e-5 1.5058837933777822e-6 5.092215427875536e-5 1.027250724194729e-6 -4.269739589208838e-5 -5.436657671371589e-5 -0.0002042418339070592 5.339554482719717e-6 -5.115053432664885e-5 5.577116166104724e-5 -0.00011463925262351623 -1.7956222344861515e-5 -3.003309689947565e-5 -0.0001296541588627464 1.804539601006992e-5 -0.00013679834888453683 2.98074926294218e-5 4.893460640313953e-5 -3.587083052398726e-5 -5.076103048767231e-5 -1.2015050068933292e-5 -4.7419091598444446e-6 0.00022637474534430537 6.330360405708549e-5 -1.0715908737533902e-5 -1.3274710235328225e-5 -3.9808615164567725e-5; -0.0001715258825111625 8.569484790716378e-5 -0.00015427663882797108 0.00015189590495942777 -7.394660341156399e-5 6.462655359833105e-6 0.00010509097202377471 0.00016527810813790064 -0.00011657592248007039 8.974915285260097e-6 3.19296975380428e-5 -0.00014739854512804722 3.547049137928025e-5 0.00016369696978876612 0.00010649001498171168 0.00010302195888875587 0.0001644379824160282 1.6792173223071718e-5 6.344538425259823e-5 2.0851128023826387e-5 9.919291837329557e-6 0.00013888742406782632 -0.00020763810451676645 0.0001473491735982382 3.128056570344334e-5 0.00017647810610872474 6.793481744757494e-5 -2.07945452398759e-5 -7.642576227731405e-5 0.00010638573595719233 4.96367117324698e-5 -0.00011510762695759204; 4.766572836640417e-5 5.523991656924728e-5 0.00010843535945721786 -5.452119641151299e-5 3.58324503875292e-6 -3.529197184863753e-5 2.414069818099648e-6 0.0001232954042145835 -0.0001269027465375235 -7.69549878640041e-7 -0.00011991298144013656 -6.727649944397793e-5 0.00020769515921099894 -6.740307200263535e-5 -4.3331402971243175e-5 5.2980226051005725e-5 -0.00011457079873682055 0.0001920831369582128 -5.547369202278632e-5 -5.8276045287786293e-5 9.964379804412713e-5 7.625022643448559e-5 -3.288003555528041e-5 2.0017147640174854e-5 5.441302581421965e-5 2.179291785548102e-5 6.169893693848465e-5 8.667524760792516e-5 0.00012334863512049402 1.748614211944747e-5 0.00016842915882558278 -0.0001343889356710264; 6.158470438758418e-6 -0.0001030861814281599 1.8579498698038935e-5 4.249120398230365e-5 -2.0506279855235563e-5 2.7970278876400933e-5 -4.17642799519663e-6 0.00010022054708018434 -3.929220396753885e-5 0.0001575392605961893 -0.00010628883969116538 0.0001505381595568376 5.066303726145864e-6 0.0001799537939011677 2.6512011442468293e-5 6.470304002208984e-5 3.091627955358855e-5 -3.786793890260071e-5 -9.180583505736319e-5 9.384679157608353e-6 1.794150635463917e-5 0.0001484312896143118 0.0001458235136458298 -0.00015441404606759293 2.5862495801101184e-5 0.00018289921795869367 5.500931079654415e-5 -1.8275429413973564e-5 0.00018384945802312768 -2.7849862038479912e-5 1.1438525558973062e-5 9.654245591874819e-5; 1.1497044230215683e-5 0.00012902111012968843 -0.0002224535393759746 0.00010969113901973412 -5.970395539753857e-5 -5.991401593183841e-5 -3.610591118907857e-5 8.152652555161868e-5 -7.81995742580044e-5 -3.350111472519508e-5 -3.7829985725519594e-5 0.00011963930848304569 -0.00010139692784269844 -2.5515529326924823e-5 -3.990951989528911e-5 -1.4415182871570897e-5 1.095683348120538e-5 3.0845460802201694e-5 -3.021353512286594e-5 -8.119031297957508e-5 0.0001275287675673003 0.000195794726838513 -7.057242040728919e-6 0.00011654584779188501 0.00023443684296223387 0.0001116511728616103 0.00014851721317990345 4.32327131567796e-5 0.00012218504236097142 0.00017197156813943473 -2.918320859302846e-5 0.0001071507456946745; 2.38693978535884e-5 -2.865105262750728e-5 0.00021610962220441316 -1.7511532416364836e-5 2.7313888938834663e-5 7.015367235729255e-5 0.00011499716719120773 3.049318592096048e-5 -1.3791565390392404e-5 -9.098333749713619e-5 -8.075994766110806e-5 6.571883067229534e-5 -4.655839576295809e-5 3.8361335544351264e-5 -6.263269962070584e-5 9.843600354363898e-5 -5.444711997549296e-5 -0.0001525932871226935 -2.8080082767673397e-5 0.00020451560202494075 -0.00011710617173955082 8.009220880951566e-6 1.7739296243262974e-5 -0.00017146696110282325 -1.1216831364397432e-5 5.582849052010059e-5 2.1023929082758117e-5 -3.209747366051371e-5 -5.654127154409205e-5 -0.00020295179048113686 0.00012005584220608799 -4.65358875880763e-5; 4.060015640986507e-5 -0.00016371746818351842 -0.0001736656521987529 2.4280803756635753e-5 3.0943527292192e-5 9.98728756368737e-5 -2.5847995800928283e-5 0.0002083845939179537 -8.538551986733346e-5 -4.181843536374034e-5 0.00021643845326449829 1.0912873278510902e-5 -0.0001341794085763753 -0.00011062931656654735 -5.303357824034544e-5 -1.55321077984955e-6 -2.2601805483492784e-5 3.0646901052186956e-5 -0.0001235665512811753 9.590735504586477e-5 0.00014010041564885663 5.218447824674669e-6 -0.00012687753400117012 -5.805166068769852e-5 -8.722913111142489e-5 -7.693892615664247e-5 -9.255258549986294e-5 0.0002575727303903201 3.397297774624188e-5 -0.00011420510048795209 -3.400320559905991e-5 0.0001325855737942889; 3.119263521317027e-5 0.00014198544987980186 -0.00010983074287845628 3.0771742893068196e-5 -9.37009271020907e-5 -6.493931403578232e-5 -6.93036998395775e-6 0.00013815512385723008 0.00014333560567853233 -6.766030390475728e-5 -4.5118190320989156e-5 -2.4028312402278282e-5 -5.313490590391579e-5 0.00026887345040255226 -0.00015120252181272583 6.684057472544877e-5 0.00013478324137200767 0.0001492228158147876 2.1900587960646143e-5 -9.330651926769951e-5 2.443927507361983e-5 -3.9366498533260655e-5 -0.00010707555600869373 -1.4015367816808866e-5 6.46430754548516e-5 4.319268656932053e-5 7.486546120877551e-5 -9.54681844470009e-5 -1.4936395820894294e-5 0.00012933625577521304 9.60590015145911e-5 -0.00014525730774234935; 0.00016661754547165592 9.5995452626037e-6 -5.059800547535577e-5 -0.00010192842397594061 -0.00010132969269982383 -7.179059263073716e-5 -4.1399819895018836e-5 6.03683116765555e-5 -1.264962280622851e-5 -0.0001561896855150079 -4.540684460542714e-6 0.00019718893657916247 1.2091991845930406e-5 -0.0001983177565875168 -0.00012459842342870078 -3.365699682256304e-5 1.301186682732619e-6 8.897994179061968e-5 -4.75208065494625e-5 -9.482267727648984e-7 -2.2396502902144953e-5 -7.230775314602677e-5 -5.241862833138554e-5 0.00015502569352754357 -1.2732927063860792e-5 -3.9130466905054665e-5 -6.105244334542321e-5 -2.6237684653481535e-5 -0.00013506328774604167 -5.343517419753061e-5 -0.00011144056712728296 8.188485720674376e-5; 9.93689685051731e-5 9.609515865262418e-5 1.8604027274294636e-5 8.174910936900174e-5 3.0407607765980987e-5 5.603665368794104e-5 2.602354135118717e-5 8.32687877041703e-5 -0.00011590276670945117 0.00014805938582098773 3.066689924848949e-5 -8.254381072079152e-5 -3.136344793206388e-5 3.682605193857661e-5 6.449373932950347e-5 5.598161470656944e-5 0.00011875094816018567 0.00021320531260383206 -9.162200848409247e-5 6.037173817773901e-6 -1.1392578643109326e-5 -0.00022938817682048194 -9.551033116481576e-5 6.278971733222418e-5 0.00015202011155589635 4.6773963105390396e-5 3.442865419446369e-6 -5.545963694062543e-5 -3.434606304695145e-5 -0.0001919215207530704 -0.00016547872627458 -8.91192183958915e-6; -1.0545969271800749e-5 -0.00022290339151415838 6.222040352394661e-5 -3.815920248644475e-5 -3.202781848859344e-5 5.916407932044661e-5 2.3232504790464034e-5 3.173052860755846e-5 8.55469999437402e-5 1.3228642419102902e-5 0.00014533625759167532 7.015703991737016e-5 0.00011469302367867727 8.497115883027829e-5 0.00011970321898796433 3.6886621418938875e-5 0.00010503480840224687 -0.00014101640303173694 4.3454915757587986e-5 -0.00012144021351606074 4.904137003444741e-6 4.258470031298464e-5 0.00013107348891889498 2.1954633811668815e-5 8.651034400775136e-5 4.625894249282434e-5 -7.94280557242975e-5 -7.426589476427531e-5 -9.102609950833082e-5 -4.70020412108269e-5 6.475479418392951e-5 6.314238376897153e-5; -4.368849663732603e-5 -7.006440687544458e-5 -8.565123976295764e-5 -2.550988215516858e-5 -3.3142036093340545e-5 -0.00012296723257604767 5.030308052229863e-5 -2.598338328679915e-5 3.104966340033266e-5 -2.2036972998469065e-5 -2.4194936534334495e-5 -1.2700544089169337e-7 0.00018310234185919255 8.597602022491649e-5 -0.0001624425913658241 -8.257099221082406e-5 4.500759539220936e-6 -4.911365799922083e-5 -9.529854110296813e-5 0.00011435171649936983 1.8389950080016263e-5 1.664891530819169e-7 -5.015693573368616e-5 7.089009336341235e-5 2.9972457875633954e-5 -5.951161624833396e-5 -2.4036489824359646e-5 0.0001277881692178221 -0.00018340926731325743 -6.473383660049485e-5 -0.00015620606250447454 -8.254673416812025e-5; -2.56702805278856e-5 4.80019880990992e-5 -3.29481262530745e-5 4.216931854108423e-5 8.555567189552139e-5 0.00012843253359398103 -5.9377987716903e-5 -2.0120984053104614e-5 -3.5318069357061005e-5 4.1822244448951675e-5 -2.0612031156531617e-5 -8.935553458910985e-6 2.725419114659701e-5 -0.0001734228207430405 2.8044134588817553e-5 2.8668151636623035e-5 -7.505843671305004e-5 -0.0002375480325360175 -0.0002875950521490153 -0.00011733264539424001 -7.466007803367306e-5 8.555637038745822e-5 5.5537708314034874e-5 0.00014043715061353668 7.69394756128898e-5 0.00014276082954104035 -7.485971575869152e-5 3.314708016779633e-5 -9.59399403006389e-5 -7.424662174224403e-5 8.785104734791074e-5 -4.980320584643474e-5; -9.199854510901844e-5 -0.00024000306265029188 4.109687316864863e-5 -4.7542796970296486e-5 -8.816875457193028e-6 0.0002077890839374628 7.860466134072651e-6 6.168563587758528e-5 0.00013104114797366984 -2.9988234096740654e-5 -2.749413041404722e-5 0.0001348183158499198 9.543181105451772e-5 -0.00012005508877867815 -5.034966860497914e-5 1.977145613106131e-5 -3.726525218174617e-5 4.085946251633898e-6 3.378156353770916e-5 1.3541768488869374e-5 0.00011325016559036582 3.395735431163909e-5 3.150532385385703e-5 -0.00010043331538857763 7.797467871033851e-5 0.00017457359528103169 -1.5623467412146784e-5 5.362543189600014e-5 -0.00012425462599443082 -1.9525932735486006e-5 4.419257849271984e-5 6.129337445068244e-5; -0.00019794775219478724 0.00023529045644837644 -4.594661298169121e-5 3.497101058174528e-5 -6.898531164818111e-6 4.9420538534566074e-5 5.349021082084834e-5 5.9407812632078533e-5 -0.00011716019001168527 2.3257898826220205e-6 0.00010682118660925566 -8.732576609899568e-5 0.00018497418274792557 -7.52874031880243e-5 6.642170116693722e-5 -8.35505145941473e-6 5.258292437220608e-5 -1.5172274497928558e-5 -0.00012801898922615992 6.694201944784054e-5 -0.0001372912513309467 -2.2952975296256756e-5 -2.278945542483876e-5 -6.40785212729435e-5 -0.0001295895628848218 -1.7842128027304344e-5 5.4231267103847656e-5 3.952052499672205e-5 -8.189394359787317e-5 -7.09591761099469e-5 -1.9373120476562817e-5 5.160932848385687e-5; -7.923409045649304e-5 -7.570196224301594e-6 9.442299435870841e-5 -5.665038386102682e-5 -6.4766496300919e-5 -6.323151662688528e-5 -0.00011693449393821139 7.148804818537372e-5 -4.897550731873056e-5 0.00010429941560398894 3.80019703032551e-5 -0.00014357596571532865 -3.4779666542064923e-5 -1.2792237406003304e-5 3.5681158103063064e-5 0.00010522481554910645 0.0001369163767111125 4.344698775352445e-5 -0.00019875751220131186 -7.381059892683991e-5 1.4466310155355671e-5 -1.8576185332829483e-5 5.008759783488835e-5 -2.6785566454393467e-5 -3.5839544373761774e-5 0.00012197666021787196 -0.00011532659461697469 8.403694321425882e-5 -0.00012407386727606787 0.0001001544189665124 -0.00011512875405347932 0.0001011474052819118; 0.0001233550239041024 2.634195995720673e-6 7.941997603407367e-5 0.00013125093676276848 -1.7911913230679333e-5 0.0001086862204166697 1.9192847618920505e-5 3.1744522063627435e-5 4.050228301562713e-5 2.5937554825901e-5 0.00022457709849473415 2.7966732093004386e-5 -7.639874536184391e-5 -6.800467301988405e-6 6.500496522416472e-5 1.1761580142006772e-5 -4.7107122439087137e-5 -4.3524440909899965e-5 2.3743544377026143e-5 -4.6770623951350115e-5 5.231614989239022e-5 4.867407472115042e-5 -1.3226475742017294e-5 -5.2329721141036216e-5 -3.8181981720228886e-5 -2.790873702257031e-5 3.097892033761677e-5 0.00010431526163952301 -1.661524845334837e-5 1.459899260876196e-5 -0.00011832902680259316 3.652251395704443e-6; -2.8260223285790273e-6 1.3295454745602193e-5 0.0001354552542483205 5.787880135302134e-5 -0.00013302088783404502 1.1837590217856658e-5 -0.0003152906837482448 0.00012097718998973196 -1.030193644566294e-5 -9.146448875127744e-5 -3.924535204588754e-5 7.469378314397726e-5 6.0473993552825355e-5 -0.00017798736817951032 1.3494582244101045e-5 -7.160398391589894e-5 -6.842214216759775e-5 0.00010432608823005876 -4.092454119983505e-5 9.437423713223206e-5 -1.8410285438783166e-5 9.450398200839809e-5 6.181862689971448e-5 0.00010917862081385706 0.00013304138255015307 -0.0001648593724081675 0.0001989364128252271 9.623538343419386e-5 0.00028123813089167386 4.863054627037491e-5 0.00021277014738130048 1.8765430549467068e-5; 0.00020731906530684744 5.127101300288545e-5 4.631925818515209e-5 -5.719830463519086e-5 -1.8642770588000753e-5 -9.590096764028852e-5 -3.187904699619065e-6 -6.806580953295295e-5 -5.867422899924916e-5 5.453620448364157e-5 0.0001321192202927102 2.669701913376518e-5 0.000256707887242131 0.00013054524148586346 -0.0001418282672925401 4.950718990997625e-5 -6.779352864711325e-5 0.00010117571488492138 1.6520928479197705e-5 -3.500987535104989e-5 8.299408633732433e-5 -7.481771629997701e-5 -3.979215856403858e-5 8.352328128651912e-5 0.00018409355376616799 -5.325607074653012e-5 -2.1330720333450102e-5 -0.00020594343826402825 -0.0001694114662636514 -8.223914941069638e-5 -8.586542847795008e-5 7.546131652318562e-5; 3.154109318293463e-5 3.536413597195021e-5 2.9559060483186072e-5 5.945782932703545e-5 -0.00012007268496072491 -2.040289409033149e-5 3.0148516611103858e-6 0.00011454023361457142 -2.1531335089642805e-5 9.529384870565706e-5 5.551667591807606e-5 -6.080294160037954e-5 0.00011015533410263506 -3.3877474645670614e-5 1.8245915914490362e-5 6.0835724714158276e-5 2.809915413981493e-5 -6.335871266180674e-5 -1.7919546091982902e-5 -5.4945065063297755e-6 -0.0002445260700302267 0.00015829563809111817 -0.00016907961370871138 -0.00011736551213525923 4.6799012768544025e-5 8.013727957168783e-5 -9.651309782621265e-5 -0.00019136307791322918 -0.00011032929004850627 3.544035526592878e-5 -9.168168734739605e-5 -7.021537139061442e-5; -0.00011151542757454656 -0.0001167365383431639 -7.134042725249435e-5 0.00011842121849019783 -9.905850792586131e-5 -3.730810321451422e-5 -0.00015447046943757196 1.6477537405902708e-7 -5.284708940473613e-5 -8.245295366089095e-5 2.1591883255837038e-5 5.02172139980526e-5 2.6484575487642894e-5 0.00019929420041569243 -0.00011476730041488437 -1.2005206640083375e-5 3.686661757944851e-5 -2.5805858326411852e-5 8.277047259411739e-6 1.4364566327798674e-5 0.00019493563987032264 3.649362288831361e-5 0.00010192503642008395 9.98576458235899e-5 4.289482122236741e-5 -0.0001397386634331515 -3.5242909957219376e-5 3.896550579645388e-5 -5.7157686837513434e-5 -4.940363332446033e-5 0.00021121956770491288 3.5752533863499285e-5; 0.00014716079534976827 -2.641145907561628e-5 0.0001752321965248829 -4.8743566694758174e-5 0.00014663913829266416 6.44407824800849e-5 5.106187531039724e-5 -0.00017406216862597297 7.325505215689046e-5 -3.0333516733809932e-5 -3.290620075873599e-5 -2.885398713275422e-5 -0.00010346820850842252 0.00010980521405130015 6.33054202260911e-5 0.00010580349557116111 0.00013098878629520027 -9.472940619324102e-6 -8.607256277320514e-6 -7.78908600843019e-5 -3.935075652421991e-5 -3.799489273675791e-5 -2.9534125460633956e-5 -4.671630322411254e-6 -2.3549777652255285e-5 -3.852119549280157e-5 -0.00021246919855926692 4.008846317298144e-6 0.00018809201505049565 -9.240325231090934e-5 0.00011848190264556968 -8.009321969454423e-5; -0.00012020967654630444 0.00016060745007611773 -0.00013262470123104556 8.184531804194882e-5 0.00010457436916671864 0.00010485894642092461 -0.00019663653905250167 3.495236057110264e-5 -0.0001772373517577329 -2.6318724061504816e-5 3.12207874832416e-5 -9.912189956035279e-5 -0.0001296721467350404 9.973453312900088e-5 -4.8081411785952726e-6 -0.0001229504715719054 2.4496882239097885e-5 3.486484535291516e-5 -0.0002326689146992544 8.925542248666441e-5 7.762544452466263e-5 5.531887714956414e-6 3.3539522397538664e-5 5.509565277511446e-5 -5.4877852684387734e-5 0.00011217357574982597 -1.2346334950529898e-6 0.00019266559581060106 4.035999415113797e-5 7.287408413153271e-5 9.475820012643837e-5 -6.576203407699251e-5], bias = [-3.984060427337718e-10, -3.6220861999069013e-11, -1.677651797705e-9, 5.59984816741856e-10, -2.480965476587697e-9, -9.850675824529835e-11, 9.052803104856694e-11, 4.035324779008351e-10, 2.2140641303782147e-10, -6.105066009507857e-10, 3.3048440391400475e-9, 2.352485609995227e-9, 4.08259142636652e-9, 3.8879226771262375e-9, -3.9063886060447805e-10, -7.384531561969631e-10, 1.6708489619959403e-9, -2.9316310508200342e-9, 1.1862644435950507e-9, 2.292981203171935e-9, -2.923163995862929e-9, -1.5642660290027388e-9, 1.646107344126006e-9, -7.414476326908213e-10, -9.056504248706273e-10, 2.556705254773653e-9, 3.1314715527015514e-9, 8.394313533953629e-10, -1.798337146897226e-9, 2.934988285790656e-10, 1.060143257221312e-9, 3.336628174683956e-10]), layer_4 = (weight = [-0.0007788534505924469 -0.0006685830389952254 -0.000688716987191034 -0.0006542395072238465 -0.0006149025421776231 -0.00081659508330541 -0.0006478916014476879 -0.000749967054858272 -0.0005892286392886316 -0.0007119907802648645 -0.0007500193367682273 -0.0006822095209065945 -0.0007797070160926858 -0.0007522449748653104 -0.0006594741368423112 -0.0007969439236190279 -0.0006549193992018953 -0.0006106590559730796 -0.0008090121770929926 -0.000620521836429977 -0.0006054688376952504 -0.0007318573278050192 -0.0006780007670647525 -0.000565240734261215 -0.000638054666204854 -0.0006129880701127028 -0.0006384380101355744 -0.0006894606397778792 -0.0007330832427160187 -0.0007676011839588111 -0.0008188282810970212 -0.0006903661698708907; 0.0003245993159796126 0.00031691102393929387 0.00025862815255747214 0.0002386690866586987 0.00042179008933688464 0.00028225686738572416 0.00039923915367428507 0.00019782936787099414 0.00026801874025249095 0.00029064249278658256 0.00021962347088651664 0.00016200238480021362 0.0002067543200442797 2.5395698882187308e-5 0.00017155234894268115 0.0003376355658037225 0.00019695894545278714 0.00027573378158736456 0.00020768779579119 0.0003904691802563027 8.741270119787773e-5 0.00018457219015582057 0.00015858274434813938 0.0003211215798211186 0.00027279834140029273 0.00035529054072876593 0.00014283905766451167 0.00024238950764526578 0.00016205058459968245 0.00022644538420962983 -4.9050983374899115e-5 0.0001550530385393652], bias = [-0.000685165866837486, 0.00024344589819855688]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
+
+    lines!(ax, losses; linewidth=4, alpha=0.75)
+    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
+
+    fig
+end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
+waveform_nn_trained = first(compute_waveform(
+    dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
+    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
+        alpha=0.5, strokewidth=2, markersize=12)
+
+    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
+        position=:lb)
+
+    fig
+end
    Appendix ​
    julia
    using InteractiveUtils
+InteractiveUtils.versioninfo()
+
+if @isdefined(MLDataDevices)
+    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
+        println()
+        CUDA.versioninfo()
+    end
+
+    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
+        println()
+        AMDGPU.versioninfo()
+    end
+end
    Julia Version 1.10.6
+Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
+Build Info:
+  Official https://julialang.org/ release
+Platform Info:
+  OS: Linux (x86_64-linux-gnu)
+  CPU: 128 × AMD EPYC 7502 32-Core Processor
+  WORD_SIZE: 64
+  LIBM: libopenlibm
+  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
+Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
+Environment:
+  JULIA_CPU_THREADS = 16
+  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
+  JULIA_PKG_SERVER = 
+  JULIA_NUM_THREADS = 16
+  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
+  JULIA_PKG_PRECOMPILE_AUTO = 0
+  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const H=h(e,[["render",R]]);export{L as __pageData,H as default};
diff --git a/dev/assets/tutorials_beginner_3_SimpleRNN.md.CSIho-Mj.js b/dev/assets/tutorials_beginner_3_SimpleRNN.md.BM5rnJGw.js
similarity index 67%
rename from dev/assets/tutorials_beginner_3_SimpleRNN.md.CSIho-Mj.js
rename to dev/assets/tutorials_beginner_3_SimpleRNN.md.BM5rnJGw.js
index f9f4414906..106e9e6a89 100644
--- a/dev/assets/tutorials_beginner_3_SimpleRNN.md.CSIho-Mj.js
+++ b/dev/assets/tutorials_beginner_3_SimpleRNN.md.BM5rnJGw.js
@@ -97,455 +97,455 @@ import{_ as a,c as n,a2 as i,o as p}from"./chunks/framework.CCjWn1F9.js";const r
     return (train_state.parameters, train_state.states) |> cpu_device()
 end
 
-ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.60876
-Epoch [  1]: Loss 0.58947
-Epoch [  1]: Loss 0.56685
-Epoch [  1]: Loss 0.55803
-Epoch [  1]: Loss 0.52538
-Epoch [  1]: Loss 0.49019
-Epoch [  1]: Loss 0.48089
-Validation: Loss 0.47082 Accuracy 1.00000
-Validation: Loss 0.48051 Accuracy 1.00000
-Epoch [  2]: Loss 0.46293
-Epoch [  2]: Loss 0.45149
-Epoch [  2]: Loss 0.44166
-Epoch [  2]: Loss 0.43669
-Epoch [  2]: Loss 0.41220
-Epoch [  2]: Loss 0.39264
-Epoch [  2]: Loss 0.40298
-Validation: Loss 0.37454 Accuracy 1.00000
-Validation: Loss 0.38588 Accuracy 1.00000
-Epoch [  3]: Loss 0.36447
-Epoch [  3]: Loss 0.36221
-Epoch [  3]: Loss 0.33435
-Epoch [  3]: Loss 0.32705
-Epoch [  3]: Loss 0.33662
-Epoch [  3]: Loss 0.32659
-Epoch [  3]: Loss 0.28894
-Validation: Loss 0.29056 Accuracy 1.00000
-Validation: Loss 0.30277 Accuracy 1.00000
-Epoch [  4]: Loss 0.28560
-Epoch [  4]: Loss 0.27737
-Epoch [  4]: Loss 0.27287
-Epoch [  4]: Loss 0.24844
-Epoch [  4]: Loss 0.25384
-Epoch [  4]: Loss 0.23238
-Epoch [  4]: Loss 0.24520
-Validation: Loss 0.22104 Accuracy 1.00000
-Validation: Loss 0.23303 Accuracy 1.00000
-Epoch [  5]: Loss 0.20778
-Epoch [  5]: Loss 0.21737
-Epoch [  5]: Loss 0.17930
-Epoch [  5]: Loss 0.19193
-Epoch [  5]: Loss 0.18534
-Epoch [  5]: Loss 0.20299
-Epoch [  5]: Loss 0.17725
-Validation: Loss 0.16526 Accuracy 1.00000
-Validation: Loss 0.17602 Accuracy 1.00000
-Epoch [  6]: Loss 0.15675
-Epoch [  6]: Loss 0.16486
-Epoch [  6]: Loss 0.15707
-Epoch [  6]: Loss 0.13994
-Epoch [  6]: Loss 0.13805
-Epoch [  6]: Loss 0.12533
-Epoch [  6]: Loss 0.12193
-Validation: Loss 0.12156 Accuracy 1.00000
-Validation: Loss 0.13025 Accuracy 1.00000
-Epoch [  7]: Loss 0.12611
-Epoch [  7]: Loss 0.11352
-Epoch [  7]: Loss 0.09855
-Epoch [  7]: Loss 0.10333
-Epoch [  7]: Loss 0.10313
-Epoch [  7]: Loss 0.09685
-Epoch [  7]: Loss 0.09368
-Validation: Loss 0.08737 Accuracy 1.00000
-Validation: Loss 0.09384 Accuracy 1.00000
-Epoch [  8]: Loss 0.08845
-Epoch [  8]: Loss 0.07892
-Epoch [  8]: Loss 0.07708
-Epoch [  8]: Loss 0.07038
-Epoch [  8]: Loss 0.07520
-Epoch [  8]: Loss 0.06707
-Epoch [  8]: Loss 0.05914
-Validation: Loss 0.06083 Accuracy 1.00000
-Validation: Loss 0.06521 Accuracy 1.00000
-Epoch [  9]: Loss 0.05626
-Epoch [  9]: Loss 0.05812
-Epoch [  9]: Loss 0.05727
-Epoch [  9]: Loss 0.04751
-Epoch [  9]: Loss 0.04994
-Epoch [  9]: Loss 0.04992
-Epoch [  9]: Loss 0.05267
-Validation: Loss 0.04513 Accuracy 1.00000
-Validation: Loss 0.04821 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04409
-Epoch [ 10]: Loss 0.04117
-Epoch [ 10]: Loss 0.04124
-Epoch [ 10]: Loss 0.03994
-Epoch [ 10]: Loss 0.04008
-Epoch [ 10]: Loss 0.03930
-Epoch [ 10]: Loss 0.03853
-Validation: Loss 0.03653 Accuracy 1.00000
-Validation: Loss 0.03901 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03718
-Epoch [ 11]: Loss 0.03205
-Epoch [ 11]: Loss 0.03301
-Epoch [ 11]: Loss 0.03540
-Epoch [ 11]: Loss 0.03260
-Epoch [ 11]: Loss 0.03353
-Epoch [ 11]: Loss 0.02814
-Validation: Loss 0.03100 Accuracy 1.00000
-Validation: Loss 0.03314 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03047
-Epoch [ 12]: Loss 0.03012
-Epoch [ 12]: Loss 0.03002
-Epoch [ 12]: Loss 0.03018
-Epoch [ 12]: Loss 0.02835
-Epoch [ 12]: Loss 0.02538
-Epoch [ 12]: Loss 0.02536
-Validation: Loss 0.02699 Accuracy 1.00000
-Validation: Loss 0.02888 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02532
-Epoch [ 13]: Loss 0.02466
-Epoch [ 13]: Loss 0.02676
-Epoch [ 13]: Loss 0.02630
-Epoch [ 13]: Loss 0.02537
-Epoch [ 13]: Loss 0.02363
-Epoch [ 13]: Loss 0.02499
-Validation: Loss 0.02390 Accuracy 1.00000
-Validation: Loss 0.02561 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02328
-Epoch [ 14]: Loss 0.02500
-Epoch [ 14]: Loss 0.02232
-Epoch [ 14]: Loss 0.02259
-Epoch [ 14]: Loss 0.02060
-Epoch [ 14]: Loss 0.02168
-Epoch [ 14]: Loss 0.02158
-Validation: Loss 0.02141 Accuracy 1.00000
-Validation: Loss 0.02297 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02163
-Epoch [ 15]: Loss 0.02014
-Epoch [ 15]: Loss 0.02060
-Epoch [ 15]: Loss 0.02013
-Epoch [ 15]: Loss 0.01927
-Epoch [ 15]: Loss 0.01935
-Epoch [ 15]: Loss 0.02165
-Validation: Loss 0.01935 Accuracy 1.00000
-Validation: Loss 0.02078 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01828
-Epoch [ 16]: Loss 0.01926
-Epoch [ 16]: Loss 0.01919
-Epoch [ 16]: Loss 0.01910
-Epoch [ 16]: Loss 0.01779
-Epoch [ 16]: Loss 0.01724
-Epoch [ 16]: Loss 0.01498
-Validation: Loss 0.01759 Accuracy 1.00000
-Validation: Loss 0.01892 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01661
-Epoch [ 17]: Loss 0.01720
-Epoch [ 17]: Loss 0.01668
-Epoch [ 17]: Loss 0.01709
-Epoch [ 17]: Loss 0.01608
-Epoch [ 17]: Loss 0.01699
-Epoch [ 17]: Loss 0.01488
-Validation: Loss 0.01609 Accuracy 1.00000
+ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.61512
+Epoch [  1]: Loss 0.60524
+Epoch [  1]: Loss 0.54823
+Epoch [  1]: Loss 0.54691
+Epoch [  1]: Loss 0.52434
+Epoch [  1]: Loss 0.50498
+Epoch [  1]: Loss 0.49311
+Validation: Loss 0.46981 Accuracy 1.00000
+Validation: Loss 0.45988 Accuracy 1.00000
+Epoch [  2]: Loss 0.47294
+Epoch [  2]: Loss 0.45229
+Epoch [  2]: Loss 0.44374
+Epoch [  2]: Loss 0.42407
+Epoch [  2]: Loss 0.41602
+Epoch [  2]: Loss 0.40209
+Epoch [  2]: Loss 0.39644
+Validation: Loss 0.37335 Accuracy 1.00000
+Validation: Loss 0.36179 Accuracy 1.00000
+Epoch [  3]: Loss 0.37796
+Epoch [  3]: Loss 0.36535
+Epoch [  3]: Loss 0.34490
+Epoch [  3]: Loss 0.33645
+Epoch [  3]: Loss 0.32186
+Epoch [  3]: Loss 0.31185
+Epoch [  3]: Loss 0.29527
+Validation: Loss 0.28839 Accuracy 1.00000
+Validation: Loss 0.27590 Accuracy 1.00000
+Epoch [  4]: Loss 0.29135
+Epoch [  4]: Loss 0.28370
+Epoch [  4]: Loss 0.26049
+Epoch [  4]: Loss 0.26873
+Epoch [  4]: Loss 0.24797
+Epoch [  4]: Loss 0.23672
+Epoch [  4]: Loss 0.18915
+Validation: Loss 0.21823 Accuracy 1.00000
+Validation: Loss 0.20570 Accuracy 1.00000
+Epoch [  5]: Loss 0.21983
+Epoch [  5]: Loss 0.22068
+Epoch [  5]: Loss 0.20337
+Epoch [  5]: Loss 0.18072
+Epoch [  5]: Loss 0.18741
+Epoch [  5]: Loss 0.18099
+Epoch [  5]: Loss 0.15911
+Validation: Loss 0.16261 Accuracy 1.00000
+Validation: Loss 0.15102 Accuracy 1.00000
+Epoch [  6]: Loss 0.16786
+Epoch [  6]: Loss 0.14418
+Epoch [  6]: Loss 0.14681
+Epoch [  6]: Loss 0.15733
+Epoch [  6]: Loss 0.13193
+Epoch [  6]: Loss 0.13234
+Epoch [  6]: Loss 0.13811
+Validation: Loss 0.11957 Accuracy 1.00000
+Validation: Loss 0.11001 Accuracy 1.00000
+Epoch [  7]: Loss 0.12181
+Epoch [  7]: Loss 0.10795
+Epoch [  7]: Loss 0.10904
+Epoch [  7]: Loss 0.09474
+Epoch [  7]: Loss 0.10197
+Epoch [  7]: Loss 0.10666
+Epoch [  7]: Loss 0.09703
+Validation: Loss 0.08569 Accuracy 1.00000
+Validation: Loss 0.07868 Accuracy 1.00000
+Epoch [  8]: Loss 0.09038
+Epoch [  8]: Loss 0.08012
+Epoch [  8]: Loss 0.08368
+Epoch [  8]: Loss 0.07743
+Epoch [  8]: Loss 0.06634
+Epoch [  8]: Loss 0.06049
+Epoch [  8]: Loss 0.06252
+Validation: Loss 0.05944 Accuracy 1.00000
+Validation: Loss 0.05485 Accuracy 1.00000
+Epoch [  9]: Loss 0.05594
+Epoch [  9]: Loss 0.05566
+Epoch [  9]: Loss 0.05580
+Epoch [  9]: Loss 0.05207
+Epoch [  9]: Loss 0.05399
+Epoch [  9]: Loss 0.04829
+Epoch [  9]: Loss 0.03966
+Validation: Loss 0.04414 Accuracy 1.00000
+Validation: Loss 0.04086 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04453
+Epoch [ 10]: Loss 0.04691
+Epoch [ 10]: Loss 0.03821
+Epoch [ 10]: Loss 0.03967
+Epoch [ 10]: Loss 0.04043
+Epoch [ 10]: Loss 0.03839
+Epoch [ 10]: Loss 0.03051
+Validation: Loss 0.03575 Accuracy 1.00000
+Validation: Loss 0.03306 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03488
+Epoch [ 11]: Loss 0.03574
+Epoch [ 11]: Loss 0.03342
+Epoch [ 11]: Loss 0.03265
+Epoch [ 11]: Loss 0.03160
+Epoch [ 11]: Loss 0.03473
+Epoch [ 11]: Loss 0.03210
+Validation: Loss 0.03038 Accuracy 1.00000
+Validation: Loss 0.02803 Accuracy 1.00000
+Epoch [ 12]: Loss 0.02933
+Epoch [ 12]: Loss 0.03052
+Epoch [ 12]: Loss 0.02711
+Epoch [ 12]: Loss 0.03082
+Epoch [ 12]: Loss 0.02924
+Epoch [ 12]: Loss 0.02724
+Epoch [ 12]: Loss 0.02788
+Validation: Loss 0.02645 Accuracy 1.00000
+Validation: Loss 0.02436 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02505
+Epoch [ 13]: Loss 0.02758
+Epoch [ 13]: Loss 0.02597
+Epoch [ 13]: Loss 0.02615
+Epoch [ 13]: Loss 0.02482
+Epoch [ 13]: Loss 0.02354
+Epoch [ 13]: Loss 0.02294
+Validation: Loss 0.02339 Accuracy 1.00000
+Validation: Loss 0.02151 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02298
+Epoch [ 14]: Loss 0.02550
+Epoch [ 14]: Loss 0.02284
+Epoch [ 14]: Loss 0.02288
+Epoch [ 14]: Loss 0.02034
+Epoch [ 14]: Loss 0.02120
+Epoch [ 14]: Loss 0.02200
+Validation: Loss 0.02094 Accuracy 1.00000
+Validation: Loss 0.01923 Accuracy 1.00000
+Epoch [ 15]: Loss 0.02073
+Epoch [ 15]: Loss 0.02081
+Epoch [ 15]: Loss 0.01883
+Epoch [ 15]: Loss 0.02020
+Epoch [ 15]: Loss 0.01974
+Epoch [ 15]: Loss 0.02074
+Epoch [ 15]: Loss 0.02274
+Validation: Loss 0.01891 Accuracy 1.00000
 Validation: Loss 0.01733 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01658
-Epoch [ 18]: Loss 0.01422
-Epoch [ 18]: Loss 0.01632
-Epoch [ 18]: Loss 0.01551
-Epoch [ 18]: Loss 0.01564
-Epoch [ 18]: Loss 0.01396
-Epoch [ 18]: Loss 0.01366
-Validation: Loss 0.01479 Accuracy 1.00000
-Validation: Loss 0.01595 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01597
-Epoch [ 19]: Loss 0.01463
-Epoch [ 19]: Loss 0.01458
-Epoch [ 19]: Loss 0.01428
-Epoch [ 19]: Loss 0.01315
-Epoch [ 19]: Loss 0.01243
-Epoch [ 19]: Loss 0.01202
-Validation: Loss 0.01364 Accuracy 1.00000
-Validation: Loss 0.01472 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01785
+Epoch [ 16]: Loss 0.01951
+Epoch [ 16]: Loss 0.01925
+Epoch [ 16]: Loss 0.01797
+Epoch [ 16]: Loss 0.01738
+Epoch [ 16]: Loss 0.01879
+Epoch [ 16]: Loss 0.01609
+Validation: Loss 0.01717 Accuracy 1.00000
+Validation: Loss 0.01572 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01831
+Epoch [ 17]: Loss 0.01727
+Epoch [ 17]: Loss 0.01735
+Epoch [ 17]: Loss 0.01683
+Epoch [ 17]: Loss 0.01505
+Epoch [ 17]: Loss 0.01624
+Epoch [ 17]: Loss 0.01405
+Validation: Loss 0.01568 Accuracy 1.00000
+Validation: Loss 0.01433 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01569
+Epoch [ 18]: Loss 0.01639
+Epoch [ 18]: Loss 0.01485
+Epoch [ 18]: Loss 0.01613
+Epoch [ 18]: Loss 0.01420
+Epoch [ 18]: Loss 0.01461
+Epoch [ 18]: Loss 0.01515
+Validation: Loss 0.01440 Accuracy 1.00000
+Validation: Loss 0.01314 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01552
+Epoch [ 19]: Loss 0.01451
+Epoch [ 19]: Loss 0.01378
+Epoch [ 19]: Loss 0.01444
+Epoch [ 19]: Loss 0.01317
+Epoch [ 19]: Loss 0.01395
+Epoch [ 19]: Loss 0.01025
+Validation: Loss 0.01325 Accuracy 1.00000
+Validation: Loss 0.01208 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01351
 Epoch [ 20]: Loss 0.01270
-Epoch [ 20]: Loss 0.01350
-Epoch [ 20]: Loss 0.01361
-Epoch [ 20]: Loss 0.01174
-Epoch [ 20]: Loss 0.01357
-Epoch [ 20]: Loss 0.01328
-Epoch [ 20]: Loss 0.01111
-Validation: Loss 0.01260 Accuracy 1.00000
-Validation: Loss 0.01361 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01262
-Epoch [ 21]: Loss 0.01259
-Epoch [ 21]: Loss 0.01207
-Epoch [ 21]: Loss 0.01204
-Epoch [ 21]: Loss 0.01107
-Epoch [ 21]: Loss 0.01176
+Epoch [ 20]: Loss 0.01306
+Epoch [ 20]: Loss 0.01216
+Epoch [ 20]: Loss 0.01388
+Epoch [ 20]: Loss 0.01223
+Epoch [ 20]: Loss 0.01327
+Validation: Loss 0.01217 Accuracy 1.00000
+Validation: Loss 0.01110 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01264
+Epoch [ 21]: Loss 0.01238
+Epoch [ 21]: Loss 0.01103
+Epoch [ 21]: Loss 0.01147
+Epoch [ 21]: Loss 0.01289
+Epoch [ 21]: Loss 0.01081
 Epoch [ 21]: Loss 0.01118
-Validation: Loss 0.01157 Accuracy 1.00000
-Validation: Loss 0.01250 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01136
-Epoch [ 22]: Loss 0.01084
-Epoch [ 22]: Loss 0.01072
-Epoch [ 22]: Loss 0.01103
-Epoch [ 22]: Loss 0.01129
-Epoch [ 22]: Loss 0.01068
-Epoch [ 22]: Loss 0.01013
-Validation: Loss 0.01044 Accuracy 1.00000
-Validation: Loss 0.01127 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01126
-Epoch [ 23]: Loss 0.01079
-Epoch [ 23]: Loss 0.00862
-Epoch [ 23]: Loss 0.00926
-Epoch [ 23]: Loss 0.00956
-Epoch [ 23]: Loss 0.00950
-Epoch [ 23]: Loss 0.00933
-Validation: Loss 0.00923 Accuracy 1.00000
-Validation: Loss 0.00994 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00898
-Epoch [ 24]: Loss 0.00918
-Epoch [ 24]: Loss 0.00925
-Epoch [ 24]: Loss 0.00852
-Epoch [ 24]: Loss 0.00775
-Epoch [ 24]: Loss 0.00819
-Epoch [ 24]: Loss 0.00957
-Validation: Loss 0.00824 Accuracy 1.00000
-Validation: Loss 0.00886 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00832
-Epoch [ 25]: Loss 0.00852
-Epoch [ 25]: Loss 0.00766
-Epoch [ 25]: Loss 0.00720
-Epoch [ 25]: Loss 0.00726
-Epoch [ 25]: Loss 0.00801
-Epoch [ 25]: Loss 0.00786
-Validation: Loss 0.00755 Accuracy 1.00000
-Validation: Loss 0.00809 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.61512
-Epoch [  1]: Loss 0.60242
-Epoch [  1]: Loss 0.55694
-Epoch [  1]: Loss 0.53467
-Epoch [  1]: Loss 0.51785
-Epoch [  1]: Loss 0.50398
-Epoch [  1]: Loss 0.46705
-Validation: Loss 0.46429 Accuracy 1.00000
-Validation: Loss 0.49806 Accuracy 1.00000
-Epoch [  2]: Loss 0.46969
-Epoch [  2]: Loss 0.44272
-Epoch [  2]: Loss 0.43718
-Epoch [  2]: Loss 0.42512
-Epoch [  2]: Loss 0.42391
-Epoch [  2]: Loss 0.39088
-Epoch [  2]: Loss 0.38009
-Validation: Loss 0.36741 Accuracy 1.00000
-Validation: Loss 0.40898 Accuracy 1.00000
-Epoch [  3]: Loss 0.36482
-Epoch [  3]: Loss 0.36399
-Epoch [  3]: Loss 0.34310
-Epoch [  3]: Loss 0.33063
-Epoch [  3]: Loss 0.31571
-Epoch [  3]: Loss 0.31828
-Epoch [  3]: Loss 0.28813
-Validation: Loss 0.28291 Accuracy 1.00000
-Validation: Loss 0.32788 Accuracy 1.00000
-Epoch [  4]: Loss 0.28119
-Epoch [  4]: Loss 0.26874
-Epoch [  4]: Loss 0.27318
-Epoch [  4]: Loss 0.24913
-Epoch [  4]: Loss 0.24549
-Epoch [  4]: Loss 0.23824
-Epoch [  4]: Loss 0.23989
-Validation: Loss 0.21332 Accuracy 1.00000
-Validation: Loss 0.25775 Accuracy 1.00000
-Epoch [  5]: Loss 0.22898
-Epoch [  5]: Loss 0.21470
-Epoch [  5]: Loss 0.18468
-Epoch [  5]: Loss 0.18042
-Epoch [  5]: Loss 0.17828
-Epoch [  5]: Loss 0.18276
-Epoch [  5]: Loss 0.17560
-Validation: Loss 0.15792 Accuracy 1.00000
-Validation: Loss 0.19791 Accuracy 1.00000
-Epoch [  6]: Loss 0.16034
-Epoch [  6]: Loss 0.16505
-Epoch [  6]: Loss 0.13701
-Epoch [  6]: Loss 0.14388
-Epoch [  6]: Loss 0.13054
-Epoch [  6]: Loss 0.12183
-Epoch [  6]: Loss 0.15076
-Validation: Loss 0.11535 Accuracy 1.00000
-Validation: Loss 0.14781 Accuracy 1.00000
-Epoch [  7]: Loss 0.12367
-Epoch [  7]: Loss 0.11161
-Epoch [  7]: Loss 0.11171
-Epoch [  7]: Loss 0.09255
-Epoch [  7]: Loss 0.10395
-Epoch [  7]: Loss 0.09041
-Epoch [  7]: Loss 0.07027
-Validation: Loss 0.08214 Accuracy 1.00000
-Validation: Loss 0.10572 Accuracy 1.00000
-Epoch [  8]: Loss 0.08794
-Epoch [  8]: Loss 0.07938
-Epoch [  8]: Loss 0.07477
-Epoch [  8]: Loss 0.06629
-Epoch [  8]: Loss 0.07276
-Epoch [  8]: Loss 0.06286
-Epoch [  8]: Loss 0.05999
-Validation: Loss 0.05727 Accuracy 1.00000
-Validation: Loss 0.07302 Accuracy 1.00000
-Epoch [  9]: Loss 0.05903
-Epoch [  9]: Loss 0.05738
-Epoch [  9]: Loss 0.05522
-Epoch [  9]: Loss 0.05230
-Epoch [  9]: Loss 0.04565
-Epoch [  9]: Loss 0.04576
-Epoch [  9]: Loss 0.03811
-Validation: Loss 0.04256 Accuracy 1.00000
-Validation: Loss 0.05375 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04291
-Epoch [ 10]: Loss 0.04001
-Epoch [ 10]: Loss 0.03806
-Epoch [ 10]: Loss 0.04058
-Epoch [ 10]: Loss 0.04058
-Epoch [ 10]: Loss 0.03729
-Epoch [ 10]: Loss 0.03779
-Validation: Loss 0.03443 Accuracy 1.00000
-Validation: Loss 0.04360 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03438
-Epoch [ 11]: Loss 0.03180
-Epoch [ 11]: Loss 0.03152
-Epoch [ 11]: Loss 0.03531
-Epoch [ 11]: Loss 0.03175
-Epoch [ 11]: Loss 0.03104
-Epoch [ 11]: Loss 0.03834
-Validation: Loss 0.02918 Accuracy 1.00000
-Validation: Loss 0.03715 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03147
-Epoch [ 12]: Loss 0.02983
-Epoch [ 12]: Loss 0.02744
-Epoch [ 12]: Loss 0.02762
+Validation: Loss 0.01103 Accuracy 1.00000
+Validation: Loss 0.01008 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01158
+Epoch [ 22]: Loss 0.00968
+Epoch [ 22]: Loss 0.01026
+Epoch [ 22]: Loss 0.01091
+Epoch [ 22]: Loss 0.01081
+Epoch [ 22]: Loss 0.01023
+Epoch [ 22]: Loss 0.01178
+Validation: Loss 0.00977 Accuracy 1.00000
+Validation: Loss 0.00896 Accuracy 1.00000
+Epoch [ 23]: Loss 0.00874
+Epoch [ 23]: Loss 0.00915
+Epoch [ 23]: Loss 0.01013
+Epoch [ 23]: Loss 0.00944
+Epoch [ 23]: Loss 0.00958
+Epoch [ 23]: Loss 0.00952
+Epoch [ 23]: Loss 0.00815
+Validation: Loss 0.00866 Accuracy 1.00000
+Validation: Loss 0.00796 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00897
+Epoch [ 24]: Loss 0.00868
+Epoch [ 24]: Loss 0.00832
+Epoch [ 24]: Loss 0.00880
+Epoch [ 24]: Loss 0.00824
+Epoch [ 24]: Loss 0.00784
+Epoch [ 24]: Loss 0.00653
+Validation: Loss 0.00786 Accuracy 1.00000
+Validation: Loss 0.00724 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00807
+Epoch [ 25]: Loss 0.00791
+Epoch [ 25]: Loss 0.00809
+Epoch [ 25]: Loss 0.00746
+Epoch [ 25]: Loss 0.00784
+Epoch [ 25]: Loss 0.00691
+Epoch [ 25]: Loss 0.00709
+Validation: Loss 0.00729 Accuracy 1.00000
+Validation: Loss 0.00672 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.62544
+Epoch [  1]: Loss 0.58391
+Epoch [  1]: Loss 0.57736
+Epoch [  1]: Loss 0.53167
+Epoch [  1]: Loss 0.51671
+Epoch [  1]: Loss 0.49884
+Epoch [  1]: Loss 0.48786
+Validation: Loss 0.47871 Accuracy 1.00000
+Validation: Loss 0.46396 Accuracy 1.00000
+Epoch [  2]: Loss 0.47308
+Epoch [  2]: Loss 0.45026
+Epoch [  2]: Loss 0.43611
+Epoch [  2]: Loss 0.42725
+Epoch [  2]: Loss 0.40983
+Epoch [  2]: Loss 0.39966
+Epoch [  2]: Loss 0.37452
+Validation: Loss 0.38291 Accuracy 1.00000
+Validation: Loss 0.36610 Accuracy 1.00000
+Epoch [  3]: Loss 0.37263
+Epoch [  3]: Loss 0.36606
+Epoch [  3]: Loss 0.34123
+Epoch [  3]: Loss 0.33349
+Epoch [  3]: Loss 0.32480
+Epoch [  3]: Loss 0.29689
+Epoch [  3]: Loss 0.28596
+Validation: Loss 0.29870 Accuracy 1.00000
+Validation: Loss 0.28058 Accuracy 1.00000
+Epoch [  4]: Loss 0.29523
+Epoch [  4]: Loss 0.28169
+Epoch [  4]: Loss 0.25398
+Epoch [  4]: Loss 0.25350
+Epoch [  4]: Loss 0.24640
+Epoch [  4]: Loss 0.23055
+Epoch [  4]: Loss 0.19666
+Validation: Loss 0.22878 Accuracy 1.00000
+Validation: Loss 0.21061 Accuracy 1.00000
+Epoch [  5]: Loss 0.20867
+Epoch [  5]: Loss 0.21100
+Epoch [  5]: Loss 0.19381
+Epoch [  5]: Loss 0.18173
+Epoch [  5]: Loss 0.19679
+Epoch [  5]: Loss 0.17571
+Epoch [  5]: Loss 0.16454
+Validation: Loss 0.17258 Accuracy 1.00000
+Validation: Loss 0.15561 Accuracy 1.00000
+Epoch [  6]: Loss 0.15738
+Epoch [  6]: Loss 0.15615
+Epoch [  6]: Loss 0.15424
+Epoch [  6]: Loss 0.13024
+Epoch [  6]: Loss 0.13682
+Epoch [  6]: Loss 0.13057
+Epoch [  6]: Loss 0.11993
+Validation: Loss 0.12772 Accuracy 1.00000
+Validation: Loss 0.11373 Accuracy 1.00000
+Epoch [  7]: Loss 0.12709
+Epoch [  7]: Loss 0.11010
+Epoch [  7]: Loss 0.10527
+Epoch [  7]: Loss 0.10227
+Epoch [  7]: Loss 0.09745
+Epoch [  7]: Loss 0.08796
+Epoch [  7]: Loss 0.09235
+Validation: Loss 0.09154 Accuracy 1.00000
+Validation: Loss 0.08131 Accuracy 1.00000
+Epoch [  8]: Loss 0.08486
+Epoch [  8]: Loss 0.07846
+Epoch [  8]: Loss 0.08010
+Epoch [  8]: Loss 0.06830
+Epoch [  8]: Loss 0.07391
+Epoch [  8]: Loss 0.05747
+Epoch [  8]: Loss 0.07270
+Validation: Loss 0.06335 Accuracy 1.00000
+Validation: Loss 0.05660 Accuracy 1.00000
+Epoch [  9]: Loss 0.05451
+Epoch [  9]: Loss 0.05514
+Epoch [  9]: Loss 0.05569
+Epoch [  9]: Loss 0.05111
+Epoch [  9]: Loss 0.04776
+Epoch [  9]: Loss 0.04815
+Epoch [  9]: Loss 0.04527
+Validation: Loss 0.04674 Accuracy 1.00000
+Validation: Loss 0.04191 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04214
+Epoch [ 10]: Loss 0.04455
+Epoch [ 10]: Loss 0.04306
+Epoch [ 10]: Loss 0.03731
+Epoch [ 10]: Loss 0.03861
+Epoch [ 10]: Loss 0.03507
+Epoch [ 10]: Loss 0.03270
+Validation: Loss 0.03776 Accuracy 1.00000
+Validation: Loss 0.03378 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03550
+Epoch [ 11]: Loss 0.03377
+Epoch [ 11]: Loss 0.03181
+Epoch [ 11]: Loss 0.03216
+Epoch [ 11]: Loss 0.03193
+Epoch [ 11]: Loss 0.03174
+Epoch [ 11]: Loss 0.03156
+Validation: Loss 0.03209 Accuracy 1.00000
+Validation: Loss 0.02861 Accuracy 1.00000
+Epoch [ 12]: Loss 0.03212
+Epoch [ 12]: Loss 0.02678
+Epoch [ 12]: Loss 0.02960
+Epoch [ 12]: Loss 0.02662
 Epoch [ 12]: Loss 0.02777
-Epoch [ 12]: Loss 0.02654
-Epoch [ 12]: Loss 0.02210
-Validation: Loss 0.02531 Accuracy 1.00000
-Validation: Loss 0.03235 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02393
+Epoch [ 12]: Loss 0.02676
+Epoch [ 12]: Loss 0.02374
+Validation: Loss 0.02795 Accuracy 1.00000
+Validation: Loss 0.02484 Accuracy 1.00000
 Epoch [ 13]: Loss 0.02618
-Epoch [ 13]: Loss 0.02483
-Epoch [ 13]: Loss 0.02462
-Epoch [ 13]: Loss 0.02438
+Epoch [ 13]: Loss 0.02595
+Epoch [ 13]: Loss 0.02439
+Epoch [ 13]: Loss 0.02525
 Epoch [ 13]: Loss 0.02377
-Epoch [ 13]: Loss 0.02443
-Validation: Loss 0.02236 Accuracy 1.00000
-Validation: Loss 0.02872 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02299
-Epoch [ 14]: Loss 0.02280
-Epoch [ 14]: Loss 0.02283
-Epoch [ 14]: Loss 0.02205
-Epoch [ 14]: Loss 0.02132
-Epoch [ 14]: Loss 0.01942
-Epoch [ 14]: Loss 0.02104
-Validation: Loss 0.01997 Accuracy 1.00000
-Validation: Loss 0.02578 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02167
-Epoch [ 15]: Loss 0.01947
-Epoch [ 15]: Loss 0.02176
-Epoch [ 15]: Loss 0.01962
-Epoch [ 15]: Loss 0.01752
+Epoch [ 13]: Loss 0.02294
+Epoch [ 13]: Loss 0.02093
+Validation: Loss 0.02477 Accuracy 1.00000
+Validation: Loss 0.02195 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02317
+Epoch [ 14]: Loss 0.02142
+Epoch [ 14]: Loss 0.02263
+Epoch [ 14]: Loss 0.02210
+Epoch [ 14]: Loss 0.02096
+Epoch [ 14]: Loss 0.02121
+Epoch [ 14]: Loss 0.02011
+Validation: Loss 0.02221 Accuracy 1.00000
+Validation: Loss 0.01962 Accuracy 1.00000
 Epoch [ 15]: Loss 0.01816
-Epoch [ 15]: Loss 0.01755
-Validation: Loss 0.01800 Accuracy 1.00000
-Validation: Loss 0.02335 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01851
-Epoch [ 16]: Loss 0.01911
-Epoch [ 16]: Loss 0.01715
-Epoch [ 16]: Loss 0.01675
-Epoch [ 16]: Loss 0.01767
+Epoch [ 15]: Loss 0.02028
+Epoch [ 15]: Loss 0.01997
+Epoch [ 15]: Loss 0.01986
+Epoch [ 15]: Loss 0.02098
+Epoch [ 15]: Loss 0.01841
+Epoch [ 15]: Loss 0.01955
+Validation: Loss 0.02010 Accuracy 1.00000
+Validation: Loss 0.01770 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01771
+Epoch [ 16]: Loss 0.01766
+Epoch [ 16]: Loss 0.01857
+Epoch [ 16]: Loss 0.01865
+Epoch [ 16]: Loss 0.01644
+Epoch [ 16]: Loss 0.01765
 Epoch [ 16]: Loss 0.01711
-Epoch [ 16]: Loss 0.01785
-Validation: Loss 0.01634 Accuracy 1.00000
-Validation: Loss 0.02131 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01649
-Epoch [ 17]: Loss 0.01484
-Epoch [ 17]: Loss 0.01512
-Epoch [ 17]: Loss 0.01680
-Epoch [ 17]: Loss 0.01660
-Epoch [ 17]: Loss 0.01670
-Epoch [ 17]: Loss 0.01695
-Validation: Loss 0.01494 Accuracy 1.00000
-Validation: Loss 0.01957 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01493
-Epoch [ 18]: Loss 0.01534
-Epoch [ 18]: Loss 0.01605
-Epoch [ 18]: Loss 0.01395
-Epoch [ 18]: Loss 0.01464
-Epoch [ 18]: Loss 0.01375
-Epoch [ 18]: Loss 0.01502
-Validation: Loss 0.01372 Accuracy 1.00000
-Validation: Loss 0.01803 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01358
-Epoch [ 19]: Loss 0.01467
-Epoch [ 19]: Loss 0.01368
-Epoch [ 19]: Loss 0.01324
-Epoch [ 19]: Loss 0.01336
-Epoch [ 19]: Loss 0.01362
-Epoch [ 19]: Loss 0.01173
-Validation: Loss 0.01266 Accuracy 1.00000
-Validation: Loss 0.01667 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01371
-Epoch [ 20]: Loss 0.01225
-Epoch [ 20]: Loss 0.01174
-Epoch [ 20]: Loss 0.01224
-Epoch [ 20]: Loss 0.01298
-Epoch [ 20]: Loss 0.01183
-Epoch [ 20]: Loss 0.01526
-Validation: Loss 0.01170 Accuracy 1.00000
-Validation: Loss 0.01541 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01193
-Epoch [ 21]: Loss 0.01288
-Epoch [ 21]: Loss 0.01119
-Epoch [ 21]: Loss 0.01071
-Epoch [ 21]: Loss 0.01207
-Epoch [ 21]: Loss 0.01124
-Epoch [ 21]: Loss 0.00995
-Validation: Loss 0.01074 Accuracy 1.00000
-Validation: Loss 0.01412 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01029
-Epoch [ 22]: Loss 0.01079
-Epoch [ 22]: Loss 0.01077
-Epoch [ 22]: Loss 0.01157
-Epoch [ 22]: Loss 0.01110
-Epoch [ 22]: Loss 0.00930
-Epoch [ 22]: Loss 0.00945
-Validation: Loss 0.00970 Accuracy 1.00000
-Validation: Loss 0.01269 Accuracy 1.00000
-Epoch [ 23]: Loss 0.00912
-Epoch [ 23]: Loss 0.00974
-Epoch [ 23]: Loss 0.00977
-Epoch [ 23]: Loss 0.00978
-Epoch [ 23]: Loss 0.00924
-Epoch [ 23]: Loss 0.00950
-Epoch [ 23]: Loss 0.00840
-Validation: Loss 0.00862 Accuracy 1.00000
-Validation: Loss 0.01118 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00903
-Epoch [ 24]: Loss 0.00880
-Epoch [ 24]: Loss 0.00833
-Epoch [ 24]: Loss 0.00852
-Epoch [ 24]: Loss 0.00776
-Epoch [ 24]: Loss 0.00803
-Epoch [ 24]: Loss 0.00902
-Validation: Loss 0.00774 Accuracy 1.00000
-Validation: Loss 0.00995 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00774
-Epoch [ 25]: Loss 0.00778
-Epoch [ 25]: Loss 0.00731
-Epoch [ 25]: Loss 0.00812
-Epoch [ 25]: Loss 0.00823
-Epoch [ 25]: Loss 0.00713
-Epoch [ 25]: Loss 0.00560
-Validation: Loss 0.00710 Accuracy 1.00000
-Validation: Loss 0.00909 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
+Validation: Loss 0.01829 Accuracy 1.00000
+Validation: Loss 0.01607 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01641
+Epoch [ 17]: Loss 0.01635
+Epoch [ 17]: Loss 0.01721
+Epoch [ 17]: Loss 0.01516
+Epoch [ 17]: Loss 0.01558
+Epoch [ 17]: Loss 0.01598
+Epoch [ 17]: Loss 0.01744
+Validation: Loss 0.01676 Accuracy 1.00000
+Validation: Loss 0.01468 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01574
+Epoch [ 18]: Loss 0.01612
+Epoch [ 18]: Loss 0.01448
+Epoch [ 18]: Loss 0.01332
+Epoch [ 18]: Loss 0.01484
+Epoch [ 18]: Loss 0.01454
+Epoch [ 18]: Loss 0.01437
+Validation: Loss 0.01542 Accuracy 1.00000
+Validation: Loss 0.01349 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01310
+Epoch [ 19]: Loss 0.01445
+Epoch [ 19]: Loss 0.01387
+Epoch [ 19]: Loss 0.01386
+Epoch [ 19]: Loss 0.01291
+Epoch [ 19]: Loss 0.01382
+Epoch [ 19]: Loss 0.01327
+Validation: Loss 0.01428 Accuracy 1.00000
+Validation: Loss 0.01247 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01262
+Epoch [ 20]: Loss 0.01312
+Epoch [ 20]: Loss 0.01335
+Epoch [ 20]: Loss 0.01283
+Epoch [ 20]: Loss 0.01258
+Epoch [ 20]: Loss 0.01185
+Epoch [ 20]: Loss 0.01087
+Validation: Loss 0.01326 Accuracy 1.00000
+Validation: Loss 0.01157 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01242
+Epoch [ 21]: Loss 0.01076
+Epoch [ 21]: Loss 0.01183
+Epoch [ 21]: Loss 0.01136
+Epoch [ 21]: Loss 0.01204
+Epoch [ 21]: Loss 0.01192
+Epoch [ 21]: Loss 0.01236
+Validation: Loss 0.01233 Accuracy 1.00000
+Validation: Loss 0.01077 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01078
+Epoch [ 22]: Loss 0.01177
+Epoch [ 22]: Loss 0.01122
+Epoch [ 22]: Loss 0.01035
+Epoch [ 22]: Loss 0.01076
+Epoch [ 22]: Loss 0.01075
+Epoch [ 22]: Loss 0.01061
+Validation: Loss 0.01143 Accuracy 1.00000
+Validation: Loss 0.00999 Accuracy 1.00000
+Epoch [ 23]: Loss 0.01001
+Epoch [ 23]: Loss 0.00999
+Epoch [ 23]: Loss 0.00999
+Epoch [ 23]: Loss 0.01055
+Epoch [ 23]: Loss 0.00962
+Epoch [ 23]: Loss 0.01039
+Epoch [ 23]: Loss 0.01015
+Validation: Loss 0.01047 Accuracy 1.00000
+Validation: Loss 0.00918 Accuracy 1.00000
+Epoch [ 24]: Loss 0.01096
+Epoch [ 24]: Loss 0.00877
+Epoch [ 24]: Loss 0.00900
+Epoch [ 24]: Loss 0.00879
+Epoch [ 24]: Loss 0.00932
+Epoch [ 24]: Loss 0.00853
+Epoch [ 24]: Loss 0.00848
+Validation: Loss 0.00936 Accuracy 1.00000
+Validation: Loss 0.00825 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00799
+Epoch [ 25]: Loss 0.00883
+Epoch [ 25]: Loss 0.00804
+Epoch [ 25]: Loss 0.00830
+Epoch [ 25]: Loss 0.00800
+Epoch [ 25]: Loss 0.00805
+Epoch [ 25]: Loss 0.00808
+Validation: Loss 0.00831 Accuracy 1.00000
+Validation: Loss 0.00736 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
  :ps_trained
  :st_trained
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
@@ -607,4 +607,4 @@ import{_ as a,c as n,a2 as i,o as p}from"./chunks/framework.CCjWn1F9.js";const r
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.453 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,45)]))}const d=a(l,[["render",e]]);export{r as __pageData,d as default};
+  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.422 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,45)]))}const d=a(l,[["render",e]]);export{r as __pageData,d as default};
diff --git a/dev/assets/tutorials_beginner_3_SimpleRNN.md.CSIho-Mj.lean.js b/dev/assets/tutorials_beginner_3_SimpleRNN.md.BM5rnJGw.lean.js
similarity index 67%
rename from dev/assets/tutorials_beginner_3_SimpleRNN.md.CSIho-Mj.lean.js
rename to dev/assets/tutorials_beginner_3_SimpleRNN.md.BM5rnJGw.lean.js
index f9f4414906..106e9e6a89 100644
--- a/dev/assets/tutorials_beginner_3_SimpleRNN.md.CSIho-Mj.lean.js
+++ b/dev/assets/tutorials_beginner_3_SimpleRNN.md.BM5rnJGw.lean.js
@@ -97,455 +97,455 @@ import{_ as a,c as n,a2 as i,o as p}from"./chunks/framework.CCjWn1F9.js";const r
     return (train_state.parameters, train_state.states) |> cpu_device()
 end
 
-ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.60876
-Epoch [  1]: Loss 0.58947
-Epoch [  1]: Loss 0.56685
-Epoch [  1]: Loss 0.55803
-Epoch [  1]: Loss 0.52538
-Epoch [  1]: Loss 0.49019
-Epoch [  1]: Loss 0.48089
-Validation: Loss 0.47082 Accuracy 1.00000
-Validation: Loss 0.48051 Accuracy 1.00000
-Epoch [  2]: Loss 0.46293
-Epoch [  2]: Loss 0.45149
-Epoch [  2]: Loss 0.44166
-Epoch [  2]: Loss 0.43669
-Epoch [  2]: Loss 0.41220
-Epoch [  2]: Loss 0.39264
-Epoch [  2]: Loss 0.40298
-Validation: Loss 0.37454 Accuracy 1.00000
-Validation: Loss 0.38588 Accuracy 1.00000
-Epoch [  3]: Loss 0.36447
-Epoch [  3]: Loss 0.36221
-Epoch [  3]: Loss 0.33435
-Epoch [  3]: Loss 0.32705
-Epoch [  3]: Loss 0.33662
-Epoch [  3]: Loss 0.32659
-Epoch [  3]: Loss 0.28894
-Validation: Loss 0.29056 Accuracy 1.00000
-Validation: Loss 0.30277 Accuracy 1.00000
-Epoch [  4]: Loss 0.28560
-Epoch [  4]: Loss 0.27737
-Epoch [  4]: Loss 0.27287
-Epoch [  4]: Loss 0.24844
-Epoch [  4]: Loss 0.25384
-Epoch [  4]: Loss 0.23238
-Epoch [  4]: Loss 0.24520
-Validation: Loss 0.22104 Accuracy 1.00000
-Validation: Loss 0.23303 Accuracy 1.00000
-Epoch [  5]: Loss 0.20778
-Epoch [  5]: Loss 0.21737
-Epoch [  5]: Loss 0.17930
-Epoch [  5]: Loss 0.19193
-Epoch [  5]: Loss 0.18534
-Epoch [  5]: Loss 0.20299
-Epoch [  5]: Loss 0.17725
-Validation: Loss 0.16526 Accuracy 1.00000
-Validation: Loss 0.17602 Accuracy 1.00000
-Epoch [  6]: Loss 0.15675
-Epoch [  6]: Loss 0.16486
-Epoch [  6]: Loss 0.15707
-Epoch [  6]: Loss 0.13994
-Epoch [  6]: Loss 0.13805
-Epoch [  6]: Loss 0.12533
-Epoch [  6]: Loss 0.12193
-Validation: Loss 0.12156 Accuracy 1.00000
-Validation: Loss 0.13025 Accuracy 1.00000
-Epoch [  7]: Loss 0.12611
-Epoch [  7]: Loss 0.11352
-Epoch [  7]: Loss 0.09855
-Epoch [  7]: Loss 0.10333
-Epoch [  7]: Loss 0.10313
-Epoch [  7]: Loss 0.09685
-Epoch [  7]: Loss 0.09368
-Validation: Loss 0.08737 Accuracy 1.00000
-Validation: Loss 0.09384 Accuracy 1.00000
-Epoch [  8]: Loss 0.08845
-Epoch [  8]: Loss 0.07892
-Epoch [  8]: Loss 0.07708
-Epoch [  8]: Loss 0.07038
-Epoch [  8]: Loss 0.07520
-Epoch [  8]: Loss 0.06707
-Epoch [  8]: Loss 0.05914
-Validation: Loss 0.06083 Accuracy 1.00000
-Validation: Loss 0.06521 Accuracy 1.00000
-Epoch [  9]: Loss 0.05626
-Epoch [  9]: Loss 0.05812
-Epoch [  9]: Loss 0.05727
-Epoch [  9]: Loss 0.04751
-Epoch [  9]: Loss 0.04994
-Epoch [  9]: Loss 0.04992
-Epoch [  9]: Loss 0.05267
-Validation: Loss 0.04513 Accuracy 1.00000
-Validation: Loss 0.04821 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04409
-Epoch [ 10]: Loss 0.04117
-Epoch [ 10]: Loss 0.04124
-Epoch [ 10]: Loss 0.03994
-Epoch [ 10]: Loss 0.04008
-Epoch [ 10]: Loss 0.03930
-Epoch [ 10]: Loss 0.03853
-Validation: Loss 0.03653 Accuracy 1.00000
-Validation: Loss 0.03901 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03718
-Epoch [ 11]: Loss 0.03205
-Epoch [ 11]: Loss 0.03301
-Epoch [ 11]: Loss 0.03540
-Epoch [ 11]: Loss 0.03260
-Epoch [ 11]: Loss 0.03353
-Epoch [ 11]: Loss 0.02814
-Validation: Loss 0.03100 Accuracy 1.00000
-Validation: Loss 0.03314 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03047
-Epoch [ 12]: Loss 0.03012
-Epoch [ 12]: Loss 0.03002
-Epoch [ 12]: Loss 0.03018
-Epoch [ 12]: Loss 0.02835
-Epoch [ 12]: Loss 0.02538
-Epoch [ 12]: Loss 0.02536
-Validation: Loss 0.02699 Accuracy 1.00000
-Validation: Loss 0.02888 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02532
-Epoch [ 13]: Loss 0.02466
-Epoch [ 13]: Loss 0.02676
-Epoch [ 13]: Loss 0.02630
-Epoch [ 13]: Loss 0.02537
-Epoch [ 13]: Loss 0.02363
-Epoch [ 13]: Loss 0.02499
-Validation: Loss 0.02390 Accuracy 1.00000
-Validation: Loss 0.02561 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02328
-Epoch [ 14]: Loss 0.02500
-Epoch [ 14]: Loss 0.02232
-Epoch [ 14]: Loss 0.02259
-Epoch [ 14]: Loss 0.02060
-Epoch [ 14]: Loss 0.02168
-Epoch [ 14]: Loss 0.02158
-Validation: Loss 0.02141 Accuracy 1.00000
-Validation: Loss 0.02297 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02163
-Epoch [ 15]: Loss 0.02014
-Epoch [ 15]: Loss 0.02060
-Epoch [ 15]: Loss 0.02013
-Epoch [ 15]: Loss 0.01927
-Epoch [ 15]: Loss 0.01935
-Epoch [ 15]: Loss 0.02165
-Validation: Loss 0.01935 Accuracy 1.00000
-Validation: Loss 0.02078 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01828
-Epoch [ 16]: Loss 0.01926
-Epoch [ 16]: Loss 0.01919
-Epoch [ 16]: Loss 0.01910
-Epoch [ 16]: Loss 0.01779
-Epoch [ 16]: Loss 0.01724
-Epoch [ 16]: Loss 0.01498
-Validation: Loss 0.01759 Accuracy 1.00000
-Validation: Loss 0.01892 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01661
-Epoch [ 17]: Loss 0.01720
-Epoch [ 17]: Loss 0.01668
-Epoch [ 17]: Loss 0.01709
-Epoch [ 17]: Loss 0.01608
-Epoch [ 17]: Loss 0.01699
-Epoch [ 17]: Loss 0.01488
-Validation: Loss 0.01609 Accuracy 1.00000
+ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.61512
+Epoch [  1]: Loss 0.60524
+Epoch [  1]: Loss 0.54823
+Epoch [  1]: Loss 0.54691
+Epoch [  1]: Loss 0.52434
+Epoch [  1]: Loss 0.50498
+Epoch [  1]: Loss 0.49311
+Validation: Loss 0.46981 Accuracy 1.00000
+Validation: Loss 0.45988 Accuracy 1.00000
+Epoch [  2]: Loss 0.47294
+Epoch [  2]: Loss 0.45229
+Epoch [  2]: Loss 0.44374
+Epoch [  2]: Loss 0.42407
+Epoch [  2]: Loss 0.41602
+Epoch [  2]: Loss 0.40209
+Epoch [  2]: Loss 0.39644
+Validation: Loss 0.37335 Accuracy 1.00000
+Validation: Loss 0.36179 Accuracy 1.00000
+Epoch [  3]: Loss 0.37796
+Epoch [  3]: Loss 0.36535
+Epoch [  3]: Loss 0.34490
+Epoch [  3]: Loss 0.33645
+Epoch [  3]: Loss 0.32186
+Epoch [  3]: Loss 0.31185
+Epoch [  3]: Loss 0.29527
+Validation: Loss 0.28839 Accuracy 1.00000
+Validation: Loss 0.27590 Accuracy 1.00000
+Epoch [  4]: Loss 0.29135
+Epoch [  4]: Loss 0.28370
+Epoch [  4]: Loss 0.26049
+Epoch [  4]: Loss 0.26873
+Epoch [  4]: Loss 0.24797
+Epoch [  4]: Loss 0.23672
+Epoch [  4]: Loss 0.18915
+Validation: Loss 0.21823 Accuracy 1.00000
+Validation: Loss 0.20570 Accuracy 1.00000
+Epoch [  5]: Loss 0.21983
+Epoch [  5]: Loss 0.22068
+Epoch [  5]: Loss 0.20337
+Epoch [  5]: Loss 0.18072
+Epoch [  5]: Loss 0.18741
+Epoch [  5]: Loss 0.18099
+Epoch [  5]: Loss 0.15911
+Validation: Loss 0.16261 Accuracy 1.00000
+Validation: Loss 0.15102 Accuracy 1.00000
+Epoch [  6]: Loss 0.16786
+Epoch [  6]: Loss 0.14418
+Epoch [  6]: Loss 0.14681
+Epoch [  6]: Loss 0.15733
+Epoch [  6]: Loss 0.13193
+Epoch [  6]: Loss 0.13234
+Epoch [  6]: Loss 0.13811
+Validation: Loss 0.11957 Accuracy 1.00000
+Validation: Loss 0.11001 Accuracy 1.00000
+Epoch [  7]: Loss 0.12181
+Epoch [  7]: Loss 0.10795
+Epoch [  7]: Loss 0.10904
+Epoch [  7]: Loss 0.09474
+Epoch [  7]: Loss 0.10197
+Epoch [  7]: Loss 0.10666
+Epoch [  7]: Loss 0.09703
+Validation: Loss 0.08569 Accuracy 1.00000
+Validation: Loss 0.07868 Accuracy 1.00000
+Epoch [  8]: Loss 0.09038
+Epoch [  8]: Loss 0.08012
+Epoch [  8]: Loss 0.08368
+Epoch [  8]: Loss 0.07743
+Epoch [  8]: Loss 0.06634
+Epoch [  8]: Loss 0.06049
+Epoch [  8]: Loss 0.06252
+Validation: Loss 0.05944 Accuracy 1.00000
+Validation: Loss 0.05485 Accuracy 1.00000
+Epoch [  9]: Loss 0.05594
+Epoch [  9]: Loss 0.05566
+Epoch [  9]: Loss 0.05580
+Epoch [  9]: Loss 0.05207
+Epoch [  9]: Loss 0.05399
+Epoch [  9]: Loss 0.04829
+Epoch [  9]: Loss 0.03966
+Validation: Loss 0.04414 Accuracy 1.00000
+Validation: Loss 0.04086 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04453
+Epoch [ 10]: Loss 0.04691
+Epoch [ 10]: Loss 0.03821
+Epoch [ 10]: Loss 0.03967
+Epoch [ 10]: Loss 0.04043
+Epoch [ 10]: Loss 0.03839
+Epoch [ 10]: Loss 0.03051
+Validation: Loss 0.03575 Accuracy 1.00000
+Validation: Loss 0.03306 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03488
+Epoch [ 11]: Loss 0.03574
+Epoch [ 11]: Loss 0.03342
+Epoch [ 11]: Loss 0.03265
+Epoch [ 11]: Loss 0.03160
+Epoch [ 11]: Loss 0.03473
+Epoch [ 11]: Loss 0.03210
+Validation: Loss 0.03038 Accuracy 1.00000
+Validation: Loss 0.02803 Accuracy 1.00000
+Epoch [ 12]: Loss 0.02933
+Epoch [ 12]: Loss 0.03052
+Epoch [ 12]: Loss 0.02711
+Epoch [ 12]: Loss 0.03082
+Epoch [ 12]: Loss 0.02924
+Epoch [ 12]: Loss 0.02724
+Epoch [ 12]: Loss 0.02788
+Validation: Loss 0.02645 Accuracy 1.00000
+Validation: Loss 0.02436 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02505
+Epoch [ 13]: Loss 0.02758
+Epoch [ 13]: Loss 0.02597
+Epoch [ 13]: Loss 0.02615
+Epoch [ 13]: Loss 0.02482
+Epoch [ 13]: Loss 0.02354
+Epoch [ 13]: Loss 0.02294
+Validation: Loss 0.02339 Accuracy 1.00000
+Validation: Loss 0.02151 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02298
+Epoch [ 14]: Loss 0.02550
+Epoch [ 14]: Loss 0.02284
+Epoch [ 14]: Loss 0.02288
+Epoch [ 14]: Loss 0.02034
+Epoch [ 14]: Loss 0.02120
+Epoch [ 14]: Loss 0.02200
+Validation: Loss 0.02094 Accuracy 1.00000
+Validation: Loss 0.01923 Accuracy 1.00000
+Epoch [ 15]: Loss 0.02073
+Epoch [ 15]: Loss 0.02081
+Epoch [ 15]: Loss 0.01883
+Epoch [ 15]: Loss 0.02020
+Epoch [ 15]: Loss 0.01974
+Epoch [ 15]: Loss 0.02074
+Epoch [ 15]: Loss 0.02274
+Validation: Loss 0.01891 Accuracy 1.00000
 Validation: Loss 0.01733 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01658
-Epoch [ 18]: Loss 0.01422
-Epoch [ 18]: Loss 0.01632
-Epoch [ 18]: Loss 0.01551
-Epoch [ 18]: Loss 0.01564
-Epoch [ 18]: Loss 0.01396
-Epoch [ 18]: Loss 0.01366
-Validation: Loss 0.01479 Accuracy 1.00000
-Validation: Loss 0.01595 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01597
-Epoch [ 19]: Loss 0.01463
-Epoch [ 19]: Loss 0.01458
-Epoch [ 19]: Loss 0.01428
-Epoch [ 19]: Loss 0.01315
-Epoch [ 19]: Loss 0.01243
-Epoch [ 19]: Loss 0.01202
-Validation: Loss 0.01364 Accuracy 1.00000
-Validation: Loss 0.01472 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01785
+Epoch [ 16]: Loss 0.01951
+Epoch [ 16]: Loss 0.01925
+Epoch [ 16]: Loss 0.01797
+Epoch [ 16]: Loss 0.01738
+Epoch [ 16]: Loss 0.01879
+Epoch [ 16]: Loss 0.01609
+Validation: Loss 0.01717 Accuracy 1.00000
+Validation: Loss 0.01572 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01831
+Epoch [ 17]: Loss 0.01727
+Epoch [ 17]: Loss 0.01735
+Epoch [ 17]: Loss 0.01683
+Epoch [ 17]: Loss 0.01505
+Epoch [ 17]: Loss 0.01624
+Epoch [ 17]: Loss 0.01405
+Validation: Loss 0.01568 Accuracy 1.00000
+Validation: Loss 0.01433 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01569
+Epoch [ 18]: Loss 0.01639
+Epoch [ 18]: Loss 0.01485
+Epoch [ 18]: Loss 0.01613
+Epoch [ 18]: Loss 0.01420
+Epoch [ 18]: Loss 0.01461
+Epoch [ 18]: Loss 0.01515
+Validation: Loss 0.01440 Accuracy 1.00000
+Validation: Loss 0.01314 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01552
+Epoch [ 19]: Loss 0.01451
+Epoch [ 19]: Loss 0.01378
+Epoch [ 19]: Loss 0.01444
+Epoch [ 19]: Loss 0.01317
+Epoch [ 19]: Loss 0.01395
+Epoch [ 19]: Loss 0.01025
+Validation: Loss 0.01325 Accuracy 1.00000
+Validation: Loss 0.01208 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01351
 Epoch [ 20]: Loss 0.01270
-Epoch [ 20]: Loss 0.01350
-Epoch [ 20]: Loss 0.01361
-Epoch [ 20]: Loss 0.01174
-Epoch [ 20]: Loss 0.01357
-Epoch [ 20]: Loss 0.01328
-Epoch [ 20]: Loss 0.01111
-Validation: Loss 0.01260 Accuracy 1.00000
-Validation: Loss 0.01361 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01262
-Epoch [ 21]: Loss 0.01259
-Epoch [ 21]: Loss 0.01207
-Epoch [ 21]: Loss 0.01204
-Epoch [ 21]: Loss 0.01107
-Epoch [ 21]: Loss 0.01176
+Epoch [ 20]: Loss 0.01306
+Epoch [ 20]: Loss 0.01216
+Epoch [ 20]: Loss 0.01388
+Epoch [ 20]: Loss 0.01223
+Epoch [ 20]: Loss 0.01327
+Validation: Loss 0.01217 Accuracy 1.00000
+Validation: Loss 0.01110 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01264
+Epoch [ 21]: Loss 0.01238
+Epoch [ 21]: Loss 0.01103
+Epoch [ 21]: Loss 0.01147
+Epoch [ 21]: Loss 0.01289
+Epoch [ 21]: Loss 0.01081
 Epoch [ 21]: Loss 0.01118
-Validation: Loss 0.01157 Accuracy 1.00000
-Validation: Loss 0.01250 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01136
-Epoch [ 22]: Loss 0.01084
-Epoch [ 22]: Loss 0.01072
-Epoch [ 22]: Loss 0.01103
-Epoch [ 22]: Loss 0.01129
-Epoch [ 22]: Loss 0.01068
-Epoch [ 22]: Loss 0.01013
-Validation: Loss 0.01044 Accuracy 1.00000
-Validation: Loss 0.01127 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01126
-Epoch [ 23]: Loss 0.01079
-Epoch [ 23]: Loss 0.00862
-Epoch [ 23]: Loss 0.00926
-Epoch [ 23]: Loss 0.00956
-Epoch [ 23]: Loss 0.00950
-Epoch [ 23]: Loss 0.00933
-Validation: Loss 0.00923 Accuracy 1.00000
-Validation: Loss 0.00994 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00898
-Epoch [ 24]: Loss 0.00918
-Epoch [ 24]: Loss 0.00925
-Epoch [ 24]: Loss 0.00852
-Epoch [ 24]: Loss 0.00775
-Epoch [ 24]: Loss 0.00819
-Epoch [ 24]: Loss 0.00957
-Validation: Loss 0.00824 Accuracy 1.00000
-Validation: Loss 0.00886 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00832
-Epoch [ 25]: Loss 0.00852
-Epoch [ 25]: Loss 0.00766
-Epoch [ 25]: Loss 0.00720
-Epoch [ 25]: Loss 0.00726
-Epoch [ 25]: Loss 0.00801
-Epoch [ 25]: Loss 0.00786
-Validation: Loss 0.00755 Accuracy 1.00000
-Validation: Loss 0.00809 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.61512
-Epoch [  1]: Loss 0.60242
-Epoch [  1]: Loss 0.55694
-Epoch [  1]: Loss 0.53467
-Epoch [  1]: Loss 0.51785
-Epoch [  1]: Loss 0.50398
-Epoch [  1]: Loss 0.46705
-Validation: Loss 0.46429 Accuracy 1.00000
-Validation: Loss 0.49806 Accuracy 1.00000
-Epoch [  2]: Loss 0.46969
-Epoch [  2]: Loss 0.44272
-Epoch [  2]: Loss 0.43718
-Epoch [  2]: Loss 0.42512
-Epoch [  2]: Loss 0.42391
-Epoch [  2]: Loss 0.39088
-Epoch [  2]: Loss 0.38009
-Validation: Loss 0.36741 Accuracy 1.00000
-Validation: Loss 0.40898 Accuracy 1.00000
-Epoch [  3]: Loss 0.36482
-Epoch [  3]: Loss 0.36399
-Epoch [  3]: Loss 0.34310
-Epoch [  3]: Loss 0.33063
-Epoch [  3]: Loss 0.31571
-Epoch [  3]: Loss 0.31828
-Epoch [  3]: Loss 0.28813
-Validation: Loss 0.28291 Accuracy 1.00000
-Validation: Loss 0.32788 Accuracy 1.00000
-Epoch [  4]: Loss 0.28119
-Epoch [  4]: Loss 0.26874
-Epoch [  4]: Loss 0.27318
-Epoch [  4]: Loss 0.24913
-Epoch [  4]: Loss 0.24549
-Epoch [  4]: Loss 0.23824
-Epoch [  4]: Loss 0.23989
-Validation: Loss 0.21332 Accuracy 1.00000
-Validation: Loss 0.25775 Accuracy 1.00000
-Epoch [  5]: Loss 0.22898
-Epoch [  5]: Loss 0.21470
-Epoch [  5]: Loss 0.18468
-Epoch [  5]: Loss 0.18042
-Epoch [  5]: Loss 0.17828
-Epoch [  5]: Loss 0.18276
-Epoch [  5]: Loss 0.17560
-Validation: Loss 0.15792 Accuracy 1.00000
-Validation: Loss 0.19791 Accuracy 1.00000
-Epoch [  6]: Loss 0.16034
-Epoch [  6]: Loss 0.16505
-Epoch [  6]: Loss 0.13701
-Epoch [  6]: Loss 0.14388
-Epoch [  6]: Loss 0.13054
-Epoch [  6]: Loss 0.12183
-Epoch [  6]: Loss 0.15076
-Validation: Loss 0.11535 Accuracy 1.00000
-Validation: Loss 0.14781 Accuracy 1.00000
-Epoch [  7]: Loss 0.12367
-Epoch [  7]: Loss 0.11161
-Epoch [  7]: Loss 0.11171
-Epoch [  7]: Loss 0.09255
-Epoch [  7]: Loss 0.10395
-Epoch [  7]: Loss 0.09041
-Epoch [  7]: Loss 0.07027
-Validation: Loss 0.08214 Accuracy 1.00000
-Validation: Loss 0.10572 Accuracy 1.00000
-Epoch [  8]: Loss 0.08794
-Epoch [  8]: Loss 0.07938
-Epoch [  8]: Loss 0.07477
-Epoch [  8]: Loss 0.06629
-Epoch [  8]: Loss 0.07276
-Epoch [  8]: Loss 0.06286
-Epoch [  8]: Loss 0.05999
-Validation: Loss 0.05727 Accuracy 1.00000
-Validation: Loss 0.07302 Accuracy 1.00000
-Epoch [  9]: Loss 0.05903
-Epoch [  9]: Loss 0.05738
-Epoch [  9]: Loss 0.05522
-Epoch [  9]: Loss 0.05230
-Epoch [  9]: Loss 0.04565
-Epoch [  9]: Loss 0.04576
-Epoch [  9]: Loss 0.03811
-Validation: Loss 0.04256 Accuracy 1.00000
-Validation: Loss 0.05375 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04291
-Epoch [ 10]: Loss 0.04001
-Epoch [ 10]: Loss 0.03806
-Epoch [ 10]: Loss 0.04058
-Epoch [ 10]: Loss 0.04058
-Epoch [ 10]: Loss 0.03729
-Epoch [ 10]: Loss 0.03779
-Validation: Loss 0.03443 Accuracy 1.00000
-Validation: Loss 0.04360 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03438
-Epoch [ 11]: Loss 0.03180
-Epoch [ 11]: Loss 0.03152
-Epoch [ 11]: Loss 0.03531
-Epoch [ 11]: Loss 0.03175
-Epoch [ 11]: Loss 0.03104
-Epoch [ 11]: Loss 0.03834
-Validation: Loss 0.02918 Accuracy 1.00000
-Validation: Loss 0.03715 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03147
-Epoch [ 12]: Loss 0.02983
-Epoch [ 12]: Loss 0.02744
-Epoch [ 12]: Loss 0.02762
+Validation: Loss 0.01103 Accuracy 1.00000
+Validation: Loss 0.01008 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01158
+Epoch [ 22]: Loss 0.00968
+Epoch [ 22]: Loss 0.01026
+Epoch [ 22]: Loss 0.01091
+Epoch [ 22]: Loss 0.01081
+Epoch [ 22]: Loss 0.01023
+Epoch [ 22]: Loss 0.01178
+Validation: Loss 0.00977 Accuracy 1.00000
+Validation: Loss 0.00896 Accuracy 1.00000
+Epoch [ 23]: Loss 0.00874
+Epoch [ 23]: Loss 0.00915
+Epoch [ 23]: Loss 0.01013
+Epoch [ 23]: Loss 0.00944
+Epoch [ 23]: Loss 0.00958
+Epoch [ 23]: Loss 0.00952
+Epoch [ 23]: Loss 0.00815
+Validation: Loss 0.00866 Accuracy 1.00000
+Validation: Loss 0.00796 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00897
+Epoch [ 24]: Loss 0.00868
+Epoch [ 24]: Loss 0.00832
+Epoch [ 24]: Loss 0.00880
+Epoch [ 24]: Loss 0.00824
+Epoch [ 24]: Loss 0.00784
+Epoch [ 24]: Loss 0.00653
+Validation: Loss 0.00786 Accuracy 1.00000
+Validation: Loss 0.00724 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00807
+Epoch [ 25]: Loss 0.00791
+Epoch [ 25]: Loss 0.00809
+Epoch [ 25]: Loss 0.00746
+Epoch [ 25]: Loss 0.00784
+Epoch [ 25]: Loss 0.00691
+Epoch [ 25]: Loss 0.00709
+Validation: Loss 0.00729 Accuracy 1.00000
+Validation: Loss 0.00672 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.62544
+Epoch [  1]: Loss 0.58391
+Epoch [  1]: Loss 0.57736
+Epoch [  1]: Loss 0.53167
+Epoch [  1]: Loss 0.51671
+Epoch [  1]: Loss 0.49884
+Epoch [  1]: Loss 0.48786
+Validation: Loss 0.47871 Accuracy 1.00000
+Validation: Loss 0.46396 Accuracy 1.00000
+Epoch [  2]: Loss 0.47308
+Epoch [  2]: Loss 0.45026
+Epoch [  2]: Loss 0.43611
+Epoch [  2]: Loss 0.42725
+Epoch [  2]: Loss 0.40983
+Epoch [  2]: Loss 0.39966
+Epoch [  2]: Loss 0.37452
+Validation: Loss 0.38291 Accuracy 1.00000
+Validation: Loss 0.36610 Accuracy 1.00000
+Epoch [  3]: Loss 0.37263
+Epoch [  3]: Loss 0.36606
+Epoch [  3]: Loss 0.34123
+Epoch [  3]: Loss 0.33349
+Epoch [  3]: Loss 0.32480
+Epoch [  3]: Loss 0.29689
+Epoch [  3]: Loss 0.28596
+Validation: Loss 0.29870 Accuracy 1.00000
+Validation: Loss 0.28058 Accuracy 1.00000
+Epoch [  4]: Loss 0.29523
+Epoch [  4]: Loss 0.28169
+Epoch [  4]: Loss 0.25398
+Epoch [  4]: Loss 0.25350
+Epoch [  4]: Loss 0.24640
+Epoch [  4]: Loss 0.23055
+Epoch [  4]: Loss 0.19666
+Validation: Loss 0.22878 Accuracy 1.00000
+Validation: Loss 0.21061 Accuracy 1.00000
+Epoch [  5]: Loss 0.20867
+Epoch [  5]: Loss 0.21100
+Epoch [  5]: Loss 0.19381
+Epoch [  5]: Loss 0.18173
+Epoch [  5]: Loss 0.19679
+Epoch [  5]: Loss 0.17571
+Epoch [  5]: Loss 0.16454
+Validation: Loss 0.17258 Accuracy 1.00000
+Validation: Loss 0.15561 Accuracy 1.00000
+Epoch [  6]: Loss 0.15738
+Epoch [  6]: Loss 0.15615
+Epoch [  6]: Loss 0.15424
+Epoch [  6]: Loss 0.13024
+Epoch [  6]: Loss 0.13682
+Epoch [  6]: Loss 0.13057
+Epoch [  6]: Loss 0.11993
+Validation: Loss 0.12772 Accuracy 1.00000
+Validation: Loss 0.11373 Accuracy 1.00000
+Epoch [  7]: Loss 0.12709
+Epoch [  7]: Loss 0.11010
+Epoch [  7]: Loss 0.10527
+Epoch [  7]: Loss 0.10227
+Epoch [  7]: Loss 0.09745
+Epoch [  7]: Loss 0.08796
+Epoch [  7]: Loss 0.09235
+Validation: Loss 0.09154 Accuracy 1.00000
+Validation: Loss 0.08131 Accuracy 1.00000
+Epoch [  8]: Loss 0.08486
+Epoch [  8]: Loss 0.07846
+Epoch [  8]: Loss 0.08010
+Epoch [  8]: Loss 0.06830
+Epoch [  8]: Loss 0.07391
+Epoch [  8]: Loss 0.05747
+Epoch [  8]: Loss 0.07270
+Validation: Loss 0.06335 Accuracy 1.00000
+Validation: Loss 0.05660 Accuracy 1.00000
+Epoch [  9]: Loss 0.05451
+Epoch [  9]: Loss 0.05514
+Epoch [  9]: Loss 0.05569
+Epoch [  9]: Loss 0.05111
+Epoch [  9]: Loss 0.04776
+Epoch [  9]: Loss 0.04815
+Epoch [  9]: Loss 0.04527
+Validation: Loss 0.04674 Accuracy 1.00000
+Validation: Loss 0.04191 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04214
+Epoch [ 10]: Loss 0.04455
+Epoch [ 10]: Loss 0.04306
+Epoch [ 10]: Loss 0.03731
+Epoch [ 10]: Loss 0.03861
+Epoch [ 10]: Loss 0.03507
+Epoch [ 10]: Loss 0.03270
+Validation: Loss 0.03776 Accuracy 1.00000
+Validation: Loss 0.03378 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03550
+Epoch [ 11]: Loss 0.03377
+Epoch [ 11]: Loss 0.03181
+Epoch [ 11]: Loss 0.03216
+Epoch [ 11]: Loss 0.03193
+Epoch [ 11]: Loss 0.03174
+Epoch [ 11]: Loss 0.03156
+Validation: Loss 0.03209 Accuracy 1.00000
+Validation: Loss 0.02861 Accuracy 1.00000
+Epoch [ 12]: Loss 0.03212
+Epoch [ 12]: Loss 0.02678
+Epoch [ 12]: Loss 0.02960
+Epoch [ 12]: Loss 0.02662
 Epoch [ 12]: Loss 0.02777
-Epoch [ 12]: Loss 0.02654
-Epoch [ 12]: Loss 0.02210
-Validation: Loss 0.02531 Accuracy 1.00000
-Validation: Loss 0.03235 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02393
+Epoch [ 12]: Loss 0.02676
+Epoch [ 12]: Loss 0.02374
+Validation: Loss 0.02795 Accuracy 1.00000
+Validation: Loss 0.02484 Accuracy 1.00000
 Epoch [ 13]: Loss 0.02618
-Epoch [ 13]: Loss 0.02483
-Epoch [ 13]: Loss 0.02462
-Epoch [ 13]: Loss 0.02438
+Epoch [ 13]: Loss 0.02595
+Epoch [ 13]: Loss 0.02439
+Epoch [ 13]: Loss 0.02525
 Epoch [ 13]: Loss 0.02377
-Epoch [ 13]: Loss 0.02443
-Validation: Loss 0.02236 Accuracy 1.00000
-Validation: Loss 0.02872 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02299
-Epoch [ 14]: Loss 0.02280
-Epoch [ 14]: Loss 0.02283
-Epoch [ 14]: Loss 0.02205
-Epoch [ 14]: Loss 0.02132
-Epoch [ 14]: Loss 0.01942
-Epoch [ 14]: Loss 0.02104
-Validation: Loss 0.01997 Accuracy 1.00000
-Validation: Loss 0.02578 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02167
-Epoch [ 15]: Loss 0.01947
-Epoch [ 15]: Loss 0.02176
-Epoch [ 15]: Loss 0.01962
-Epoch [ 15]: Loss 0.01752
+Epoch [ 13]: Loss 0.02294
+Epoch [ 13]: Loss 0.02093
+Validation: Loss 0.02477 Accuracy 1.00000
+Validation: Loss 0.02195 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02317
+Epoch [ 14]: Loss 0.02142
+Epoch [ 14]: Loss 0.02263
+Epoch [ 14]: Loss 0.02210
+Epoch [ 14]: Loss 0.02096
+Epoch [ 14]: Loss 0.02121
+Epoch [ 14]: Loss 0.02011
+Validation: Loss 0.02221 Accuracy 1.00000
+Validation: Loss 0.01962 Accuracy 1.00000
 Epoch [ 15]: Loss 0.01816
-Epoch [ 15]: Loss 0.01755
-Validation: Loss 0.01800 Accuracy 1.00000
-Validation: Loss 0.02335 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01851
-Epoch [ 16]: Loss 0.01911
-Epoch [ 16]: Loss 0.01715
-Epoch [ 16]: Loss 0.01675
-Epoch [ 16]: Loss 0.01767
+Epoch [ 15]: Loss 0.02028
+Epoch [ 15]: Loss 0.01997
+Epoch [ 15]: Loss 0.01986
+Epoch [ 15]: Loss 0.02098
+Epoch [ 15]: Loss 0.01841
+Epoch [ 15]: Loss 0.01955
+Validation: Loss 0.02010 Accuracy 1.00000
+Validation: Loss 0.01770 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01771
+Epoch [ 16]: Loss 0.01766
+Epoch [ 16]: Loss 0.01857
+Epoch [ 16]: Loss 0.01865
+Epoch [ 16]: Loss 0.01644
+Epoch [ 16]: Loss 0.01765
 Epoch [ 16]: Loss 0.01711
-Epoch [ 16]: Loss 0.01785
-Validation: Loss 0.01634 Accuracy 1.00000
-Validation: Loss 0.02131 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01649
-Epoch [ 17]: Loss 0.01484
-Epoch [ 17]: Loss 0.01512
-Epoch [ 17]: Loss 0.01680
-Epoch [ 17]: Loss 0.01660
-Epoch [ 17]: Loss 0.01670
-Epoch [ 17]: Loss 0.01695
-Validation: Loss 0.01494 Accuracy 1.00000
-Validation: Loss 0.01957 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01493
-Epoch [ 18]: Loss 0.01534
-Epoch [ 18]: Loss 0.01605
-Epoch [ 18]: Loss 0.01395
-Epoch [ 18]: Loss 0.01464
-Epoch [ 18]: Loss 0.01375
-Epoch [ 18]: Loss 0.01502
-Validation: Loss 0.01372 Accuracy 1.00000
-Validation: Loss 0.01803 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01358
-Epoch [ 19]: Loss 0.01467
-Epoch [ 19]: Loss 0.01368
-Epoch [ 19]: Loss 0.01324
-Epoch [ 19]: Loss 0.01336
-Epoch [ 19]: Loss 0.01362
-Epoch [ 19]: Loss 0.01173
-Validation: Loss 0.01266 Accuracy 1.00000
-Validation: Loss 0.01667 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01371
-Epoch [ 20]: Loss 0.01225
-Epoch [ 20]: Loss 0.01174
-Epoch [ 20]: Loss 0.01224
-Epoch [ 20]: Loss 0.01298
-Epoch [ 20]: Loss 0.01183
-Epoch [ 20]: Loss 0.01526
-Validation: Loss 0.01170 Accuracy 1.00000
-Validation: Loss 0.01541 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01193
-Epoch [ 21]: Loss 0.01288
-Epoch [ 21]: Loss 0.01119
-Epoch [ 21]: Loss 0.01071
-Epoch [ 21]: Loss 0.01207
-Epoch [ 21]: Loss 0.01124
-Epoch [ 21]: Loss 0.00995
-Validation: Loss 0.01074 Accuracy 1.00000
-Validation: Loss 0.01412 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01029
-Epoch [ 22]: Loss 0.01079
-Epoch [ 22]: Loss 0.01077
-Epoch [ 22]: Loss 0.01157
-Epoch [ 22]: Loss 0.01110
-Epoch [ 22]: Loss 0.00930
-Epoch [ 22]: Loss 0.00945
-Validation: Loss 0.00970 Accuracy 1.00000
-Validation: Loss 0.01269 Accuracy 1.00000
-Epoch [ 23]: Loss 0.00912
-Epoch [ 23]: Loss 0.00974
-Epoch [ 23]: Loss 0.00977
-Epoch [ 23]: Loss 0.00978
-Epoch [ 23]: Loss 0.00924
-Epoch [ 23]: Loss 0.00950
-Epoch [ 23]: Loss 0.00840
-Validation: Loss 0.00862 Accuracy 1.00000
-Validation: Loss 0.01118 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00903
-Epoch [ 24]: Loss 0.00880
-Epoch [ 24]: Loss 0.00833
-Epoch [ 24]: Loss 0.00852
-Epoch [ 24]: Loss 0.00776
-Epoch [ 24]: Loss 0.00803
-Epoch [ 24]: Loss 0.00902
-Validation: Loss 0.00774 Accuracy 1.00000
-Validation: Loss 0.00995 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00774
-Epoch [ 25]: Loss 0.00778
-Epoch [ 25]: Loss 0.00731
-Epoch [ 25]: Loss 0.00812
-Epoch [ 25]: Loss 0.00823
-Epoch [ 25]: Loss 0.00713
-Epoch [ 25]: Loss 0.00560
-Validation: Loss 0.00710 Accuracy 1.00000
-Validation: Loss 0.00909 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
+Validation: Loss 0.01829 Accuracy 1.00000
+Validation: Loss 0.01607 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01641
+Epoch [ 17]: Loss 0.01635
+Epoch [ 17]: Loss 0.01721
+Epoch [ 17]: Loss 0.01516
+Epoch [ 17]: Loss 0.01558
+Epoch [ 17]: Loss 0.01598
+Epoch [ 17]: Loss 0.01744
+Validation: Loss 0.01676 Accuracy 1.00000
+Validation: Loss 0.01468 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01574
+Epoch [ 18]: Loss 0.01612
+Epoch [ 18]: Loss 0.01448
+Epoch [ 18]: Loss 0.01332
+Epoch [ 18]: Loss 0.01484
+Epoch [ 18]: Loss 0.01454
+Epoch [ 18]: Loss 0.01437
+Validation: Loss 0.01542 Accuracy 1.00000
+Validation: Loss 0.01349 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01310
+Epoch [ 19]: Loss 0.01445
+Epoch [ 19]: Loss 0.01387
+Epoch [ 19]: Loss 0.01386
+Epoch [ 19]: Loss 0.01291
+Epoch [ 19]: Loss 0.01382
+Epoch [ 19]: Loss 0.01327
+Validation: Loss 0.01428 Accuracy 1.00000
+Validation: Loss 0.01247 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01262
+Epoch [ 20]: Loss 0.01312
+Epoch [ 20]: Loss 0.01335
+Epoch [ 20]: Loss 0.01283
+Epoch [ 20]: Loss 0.01258
+Epoch [ 20]: Loss 0.01185
+Epoch [ 20]: Loss 0.01087
+Validation: Loss 0.01326 Accuracy 1.00000
+Validation: Loss 0.01157 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01242
+Epoch [ 21]: Loss 0.01076
+Epoch [ 21]: Loss 0.01183
+Epoch [ 21]: Loss 0.01136
+Epoch [ 21]: Loss 0.01204
+Epoch [ 21]: Loss 0.01192
+Epoch [ 21]: Loss 0.01236
+Validation: Loss 0.01233 Accuracy 1.00000
+Validation: Loss 0.01077 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01078
+Epoch [ 22]: Loss 0.01177
+Epoch [ 22]: Loss 0.01122
+Epoch [ 22]: Loss 0.01035
+Epoch [ 22]: Loss 0.01076
+Epoch [ 22]: Loss 0.01075
+Epoch [ 22]: Loss 0.01061
+Validation: Loss 0.01143 Accuracy 1.00000
+Validation: Loss 0.00999 Accuracy 1.00000
+Epoch [ 23]: Loss 0.01001
+Epoch [ 23]: Loss 0.00999
+Epoch [ 23]: Loss 0.00999
+Epoch [ 23]: Loss 0.01055
+Epoch [ 23]: Loss 0.00962
+Epoch [ 23]: Loss 0.01039
+Epoch [ 23]: Loss 0.01015
+Validation: Loss 0.01047 Accuracy 1.00000
+Validation: Loss 0.00918 Accuracy 1.00000
+Epoch [ 24]: Loss 0.01096
+Epoch [ 24]: Loss 0.00877
+Epoch [ 24]: Loss 0.00900
+Epoch [ 24]: Loss 0.00879
+Epoch [ 24]: Loss 0.00932
+Epoch [ 24]: Loss 0.00853
+Epoch [ 24]: Loss 0.00848
+Validation: Loss 0.00936 Accuracy 1.00000
+Validation: Loss 0.00825 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00799
+Epoch [ 25]: Loss 0.00883
+Epoch [ 25]: Loss 0.00804
+Epoch [ 25]: Loss 0.00830
+Epoch [ 25]: Loss 0.00800
+Epoch [ 25]: Loss 0.00805
+Epoch [ 25]: Loss 0.00808
+Validation: Loss 0.00831 Accuracy 1.00000
+Validation: Loss 0.00736 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
  :ps_trained
  :st_trained
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
@@ -607,4 +607,4 @@ import{_ as a,c as n,a2 as i,o as p}from"./chunks/framework.CCjWn1F9.js";const r
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.453 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,45)]))}const d=a(l,[["render",e]]);export{r as __pageData,d as default};
+  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.422 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,45)]))}const d=a(l,[["render",e]]);export{r as __pageData,d as default};
diff --git a/dev/assets/tutorials_beginner_4_SimpleChains.md.UnuoVtSp.js b/dev/assets/tutorials_beginner_4_SimpleChains.md.BHQtKOk4.js
similarity index 94%
rename from dev/assets/tutorials_beginner_4_SimpleChains.md.UnuoVtSp.js
rename to dev/assets/tutorials_beginner_4_SimpleChains.md.BHQtKOk4.js
index 0684cca9a4..58b13ea589 100644
--- a/dev/assets/tutorials_beginner_4_SimpleChains.md.UnuoVtSp.js
+++ b/dev/assets/tutorials_beginner_4_SimpleChains.md.BHQtKOk4.js
@@ -90,25 +90,25 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const g
     end
 
     return tr_acc, te_acc
-end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 86.21s 	 Training Accuracy: 22.06% 	 Test Accuracy: 19.00%
-[ 2/10] 	 Time 80.57s 	 Training Accuracy: 47.28% 	 Test Accuracy: 46.50%
-[ 3/10] 	 Time 81.04s 	 Training Accuracy: 60.33% 	 Test Accuracy: 60.00%
-[ 4/10] 	 Time 85.15s 	 Training Accuracy: 69.61% 	 Test Accuracy: 66.50%
-[ 5/10] 	 Time 93.32s 	 Training Accuracy: 75.44% 	 Test Accuracy: 73.50%
-[ 6/10] 	 Time 88.87s 	 Training Accuracy: 79.00% 	 Test Accuracy: 76.50%
-[ 7/10] 	 Time 85.97s 	 Training Accuracy: 81.00% 	 Test Accuracy: 81.50%
-[ 8/10] 	 Time 92.55s 	 Training Accuracy: 83.00% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 109.11s 	 Training Accuracy: 84.89% 	 Test Accuracy: 81.50%
-[10/10] 	 Time 108.74s 	 Training Accuracy: 86.17% 	 Test Accuracy: 85.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.89s 	 Training Accuracy: 30.22% 	 Test Accuracy: 26.00%
-[ 2/10] 	 Time 17.83s 	 Training Accuracy: 46.56% 	 Test Accuracy: 44.50%
-[ 3/10] 	 Time 17.84s 	 Training Accuracy: 62.22% 	 Test Accuracy: 55.00%
-[ 4/10] 	 Time 17.83s 	 Training Accuracy: 66.39% 	 Test Accuracy: 58.50%
-[ 5/10] 	 Time 17.88s 	 Training Accuracy: 73.17% 	 Test Accuracy: 70.00%
-[ 6/10] 	 Time 17.84s 	 Training Accuracy: 78.78% 	 Test Accuracy: 74.50%
-[ 7/10] 	 Time 17.84s 	 Training Accuracy: 82.44% 	 Test Accuracy: 79.00%
-[ 8/10] 	 Time 17.85s 	 Training Accuracy: 84.06% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 17.84s 	 Training Accuracy: 86.50% 	 Test Accuracy: 82.00%
-[10/10] 	 Time 17.84s 	 Training Accuracy: 88.39% 	 Test Accuracy: 85.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
+end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 112.65s 	 Training Accuracy: 21.89% 	 Test Accuracy: 18.50%
+[ 2/10] 	 Time 100.45s 	 Training Accuracy: 46.44% 	 Test Accuracy: 43.00%
+[ 3/10] 	 Time 108.58s 	 Training Accuracy: 59.94% 	 Test Accuracy: 61.50%
+[ 4/10] 	 Time 110.91s 	 Training Accuracy: 67.94% 	 Test Accuracy: 65.50%
+[ 5/10] 	 Time 113.76s 	 Training Accuracy: 73.28% 	 Test Accuracy: 71.50%
+[ 6/10] 	 Time 114.94s 	 Training Accuracy: 77.33% 	 Test Accuracy: 76.00%
+[ 7/10] 	 Time 114.72s 	 Training Accuracy: 80.44% 	 Test Accuracy: 78.50%
+[ 8/10] 	 Time 114.19s 	 Training Accuracy: 83.00% 	 Test Accuracy: 82.50%
+[ 9/10] 	 Time 114.70s 	 Training Accuracy: 84.44% 	 Test Accuracy: 81.00%
+[10/10] 	 Time 113.65s 	 Training Accuracy: 87.06% 	 Test Accuracy: 83.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 19.01s 	 Training Accuracy: 30.39% 	 Test Accuracy: 26.00%
+[ 2/10] 	 Time 17.90s 	 Training Accuracy: 45.67% 	 Test Accuracy: 42.00%
+[ 3/10] 	 Time 17.91s 	 Training Accuracy: 60.89% 	 Test Accuracy: 55.50%
+[ 4/10] 	 Time 17.91s 	 Training Accuracy: 68.28% 	 Test Accuracy: 66.50%
+[ 5/10] 	 Time 17.93s 	 Training Accuracy: 77.39% 	 Test Accuracy: 73.50%
+[ 6/10] 	 Time 17.93s 	 Training Accuracy: 79.17% 	 Test Accuracy: 74.50%
+[ 7/10] 	 Time 17.90s 	 Training Accuracy: 81.28% 	 Test Accuracy: 76.00%
+[ 8/10] 	 Time 17.97s 	 Training Accuracy: 84.33% 	 Test Accuracy: 81.00%
+[ 9/10] 	 Time 17.95s 	 Training Accuracy: 86.22% 	 Test Accuracy: 84.50%
+[10/10] 	 Time 17.92s 	 Training Accuracy: 87.44% 	 Test Accuracy: 83.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/dev/assets/tutorials_beginner_4_SimpleChains.md.UnuoVtSp.lean.js b/dev/assets/tutorials_beginner_4_SimpleChains.md.BHQtKOk4.lean.js
similarity index 94%
rename from dev/assets/tutorials_beginner_4_SimpleChains.md.UnuoVtSp.lean.js
rename to dev/assets/tutorials_beginner_4_SimpleChains.md.BHQtKOk4.lean.js
index 0684cca9a4..58b13ea589 100644
--- a/dev/assets/tutorials_beginner_4_SimpleChains.md.UnuoVtSp.lean.js
+++ b/dev/assets/tutorials_beginner_4_SimpleChains.md.BHQtKOk4.lean.js
@@ -90,25 +90,25 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const g
     end
 
     return tr_acc, te_acc
-end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 86.21s 	 Training Accuracy: 22.06% 	 Test Accuracy: 19.00%
-[ 2/10] 	 Time 80.57s 	 Training Accuracy: 47.28% 	 Test Accuracy: 46.50%
-[ 3/10] 	 Time 81.04s 	 Training Accuracy: 60.33% 	 Test Accuracy: 60.00%
-[ 4/10] 	 Time 85.15s 	 Training Accuracy: 69.61% 	 Test Accuracy: 66.50%
-[ 5/10] 	 Time 93.32s 	 Training Accuracy: 75.44% 	 Test Accuracy: 73.50%
-[ 6/10] 	 Time 88.87s 	 Training Accuracy: 79.00% 	 Test Accuracy: 76.50%
-[ 7/10] 	 Time 85.97s 	 Training Accuracy: 81.00% 	 Test Accuracy: 81.50%
-[ 8/10] 	 Time 92.55s 	 Training Accuracy: 83.00% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 109.11s 	 Training Accuracy: 84.89% 	 Test Accuracy: 81.50%
-[10/10] 	 Time 108.74s 	 Training Accuracy: 86.17% 	 Test Accuracy: 85.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.89s 	 Training Accuracy: 30.22% 	 Test Accuracy: 26.00%
-[ 2/10] 	 Time 17.83s 	 Training Accuracy: 46.56% 	 Test Accuracy: 44.50%
-[ 3/10] 	 Time 17.84s 	 Training Accuracy: 62.22% 	 Test Accuracy: 55.00%
-[ 4/10] 	 Time 17.83s 	 Training Accuracy: 66.39% 	 Test Accuracy: 58.50%
-[ 5/10] 	 Time 17.88s 	 Training Accuracy: 73.17% 	 Test Accuracy: 70.00%
-[ 6/10] 	 Time 17.84s 	 Training Accuracy: 78.78% 	 Test Accuracy: 74.50%
-[ 7/10] 	 Time 17.84s 	 Training Accuracy: 82.44% 	 Test Accuracy: 79.00%
-[ 8/10] 	 Time 17.85s 	 Training Accuracy: 84.06% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 17.84s 	 Training Accuracy: 86.50% 	 Test Accuracy: 82.00%
-[10/10] 	 Time 17.84s 	 Training Accuracy: 88.39% 	 Test Accuracy: 85.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
+end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 112.65s 	 Training Accuracy: 21.89% 	 Test Accuracy: 18.50%
+[ 2/10] 	 Time 100.45s 	 Training Accuracy: 46.44% 	 Test Accuracy: 43.00%
+[ 3/10] 	 Time 108.58s 	 Training Accuracy: 59.94% 	 Test Accuracy: 61.50%
+[ 4/10] 	 Time 110.91s 	 Training Accuracy: 67.94% 	 Test Accuracy: 65.50%
+[ 5/10] 	 Time 113.76s 	 Training Accuracy: 73.28% 	 Test Accuracy: 71.50%
+[ 6/10] 	 Time 114.94s 	 Training Accuracy: 77.33% 	 Test Accuracy: 76.00%
+[ 7/10] 	 Time 114.72s 	 Training Accuracy: 80.44% 	 Test Accuracy: 78.50%
+[ 8/10] 	 Time 114.19s 	 Training Accuracy: 83.00% 	 Test Accuracy: 82.50%
+[ 9/10] 	 Time 114.70s 	 Training Accuracy: 84.44% 	 Test Accuracy: 81.00%
+[10/10] 	 Time 113.65s 	 Training Accuracy: 87.06% 	 Test Accuracy: 83.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 19.01s 	 Training Accuracy: 30.39% 	 Test Accuracy: 26.00%
+[ 2/10] 	 Time 17.90s 	 Training Accuracy: 45.67% 	 Test Accuracy: 42.00%
+[ 3/10] 	 Time 17.91s 	 Training Accuracy: 60.89% 	 Test Accuracy: 55.50%
+[ 4/10] 	 Time 17.91s 	 Training Accuracy: 68.28% 	 Test Accuracy: 66.50%
+[ 5/10] 	 Time 17.93s 	 Training Accuracy: 77.39% 	 Test Accuracy: 73.50%
+[ 6/10] 	 Time 17.93s 	 Training Accuracy: 79.17% 	 Test Accuracy: 74.50%
+[ 7/10] 	 Time 17.90s 	 Training Accuracy: 81.28% 	 Test Accuracy: 76.00%
+[ 8/10] 	 Time 17.97s 	 Training Accuracy: 84.33% 	 Test Accuracy: 81.00%
+[ 9/10] 	 Time 17.95s 	 Training Accuracy: 86.22% 	 Test Accuracy: 84.50%
+[10/10] 	 Time 17.92s 	 Training Accuracy: 87.44% 	 Test Accuracy: 83.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.DkpjLori.lean.js b/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.Cpu7s8HO.js
similarity index 58%
rename from dev/assets/tutorials_beginner_5_OptimizationIntegration.md.DkpjLori.lean.js
rename to dev/assets/tutorials_beginner_5_OptimizationIntegration.md.Cpu7s8HO.js
index 6a7dc4a0ec..cf06dc810e 100644
--- a/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.DkpjLori.lean.js
+++ b/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.Cpu7s8HO.js
@@ -80,31 +80,31 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.CCjWn1F9.js";const d
     return StatefulLuxLayer{true}(model, res.u, smodel.st)
 end
 
-trained_model = train_model(dataloader)
    Iteration:     1, Loss: 2.991784e-01
-Iteration:    26, Loss: 8.993525e-02
-Iteration:    51, Loss: 1.784816e-01
-Iteration:    76, Loss: 2.487762e-01
-Iteration:     1, Loss: 1.481039e-01
-Iteration:     1, Loss: 2.541728e-02
-Iteration:    26, Loss: 3.857330e-02
-Iteration:    51, Loss: 2.923982e-02
-Iteration:    76, Loss: 2.677913e-02
-Iteration:   101, Loss: 2.588445e-02
-Iteration:   126, Loss: 2.508674e-02
-Iteration:   151, Loss: 2.428122e-02
-Iteration:   176, Loss: 2.341577e-02
-Iteration:   201, Loss: 2.247446e-02
-Iteration:   226, Loss: 2.149151e-02
-Iteration:   251, Loss: 2.047292e-02
-Iteration:   276, Loss: 1.935390e-02
-Iteration:   301, Loss: 1.811985e-02
-Iteration:   326, Loss: 1.687357e-02
-Iteration:   351, Loss: 1.563711e-02
-Iteration:   376, Loss: 1.438919e-02
-Iteration:   401, Loss: 1.313944e-02
-Iteration:   426, Loss: 1.188992e-02
-Iteration:   451, Loss: 1.067601e-02
-Iteration:   476, Loss: 1.076645e-02

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
+trained_model = train_model(dataloader)
    Iteration:     1, Loss: 2.966199e-02
+Iteration:    26, Loss: 2.974361e-02
+Iteration:    51, Loss: 1.571838e-01
+Iteration:    76, Loss: 2.171316e-01
+Iteration:     1, Loss: 1.832428e-01
+Iteration:     1, Loss: 1.933464e-02
+Iteration:    26, Loss: 3.901346e-02
+Iteration:    51, Loss: 2.986553e-02
+Iteration:    76, Loss: 2.560794e-02
+Iteration:   101, Loss: 2.308636e-02
+Iteration:   126, Loss: 2.079723e-02
+Iteration:   151, Loss: 1.886188e-02
+Iteration:   176, Loss: 1.737600e-02
+Iteration:   201, Loss: 1.608310e-02
+Iteration:   226, Loss: 1.487726e-02
+Iteration:   251, Loss: 1.359568e-02
+Iteration:   276, Loss: 1.214328e-02
+Iteration:   301, Loss: 1.045338e-02
+Iteration:   326, Loss: 2.002797e-02
+Iteration:   351, Loss: 1.186746e-02
+Iteration:   376, Loss: 9.847032e-03
+Iteration:   401, Loss: 8.622359e-03
+Iteration:   426, Loss: 7.740140e-03
+Iteration:   451, Loss: 6.804725e-03
+Iteration:   476, Loss: 5.482031e-03

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
 prob = ODEProblem(dudt, gdev(u0), (tspan[1], tspan[2]), trained_model.ps)
 sol = solve(prob, Tsit5(); saveat=t)
 pred = convert(AbstractArray, sol) |> cdev
@@ -118,7 +118,7 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.CCjWn1F9.js";const d
     lines!(ax, t, pred[2, :]; label=L"\\hat{u}_2(t)", color=:red, linewidth=4)
     axislegend(ax; position=:lt)
     fig
-end

    Appendix ​

    julia
    using InteractiveUtils
+end

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -178,4 +178,4 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.CCjWn1F9.js";const d
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,29)]))}const g=s(t,[["render",p]]);export{d as __pageData,g as default}; + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.359 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,29)]))}const g=s(t,[["render",p]]);export{d as __pageData,g as default}; diff --git a/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.DkpjLori.js b/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.Cpu7s8HO.lean.js similarity index 58% rename from dev/assets/tutorials_beginner_5_OptimizationIntegration.md.DkpjLori.js rename to dev/assets/tutorials_beginner_5_OptimizationIntegration.md.Cpu7s8HO.lean.js index 6a7dc4a0ec..cf06dc810e 100644 --- a/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.DkpjLori.js +++ b/dev/assets/tutorials_beginner_5_OptimizationIntegration.md.Cpu7s8HO.lean.js @@ -80,31 +80,31 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.CCjWn1F9.js";const d return StatefulLuxLayer{true}(model, res.u, smodel.st) end -trained_model = train_model(dataloader)
    Iteration:     1, Loss: 2.991784e-01
-Iteration:    26, Loss: 8.993525e-02
-Iteration:    51, Loss: 1.784816e-01
-Iteration:    76, Loss: 2.487762e-01
-Iteration:     1, Loss: 1.481039e-01
-Iteration:     1, Loss: 2.541728e-02
-Iteration:    26, Loss: 3.857330e-02
-Iteration:    51, Loss: 2.923982e-02
-Iteration:    76, Loss: 2.677913e-02
-Iteration:   101, Loss: 2.588445e-02
-Iteration:   126, Loss: 2.508674e-02
-Iteration:   151, Loss: 2.428122e-02
-Iteration:   176, Loss: 2.341577e-02
-Iteration:   201, Loss: 2.247446e-02
-Iteration:   226, Loss: 2.149151e-02
-Iteration:   251, Loss: 2.047292e-02
-Iteration:   276, Loss: 1.935390e-02
-Iteration:   301, Loss: 1.811985e-02
-Iteration:   326, Loss: 1.687357e-02
-Iteration:   351, Loss: 1.563711e-02
-Iteration:   376, Loss: 1.438919e-02
-Iteration:   401, Loss: 1.313944e-02
-Iteration:   426, Loss: 1.188992e-02
-Iteration:   451, Loss: 1.067601e-02
-Iteration:   476, Loss: 1.076645e-02

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
+trained_model = train_model(dataloader)
    Iteration:     1, Loss: 2.966199e-02
+Iteration:    26, Loss: 2.974361e-02
+Iteration:    51, Loss: 1.571838e-01
+Iteration:    76, Loss: 2.171316e-01
+Iteration:     1, Loss: 1.832428e-01
+Iteration:     1, Loss: 1.933464e-02
+Iteration:    26, Loss: 3.901346e-02
+Iteration:    51, Loss: 2.986553e-02
+Iteration:    76, Loss: 2.560794e-02
+Iteration:   101, Loss: 2.308636e-02
+Iteration:   126, Loss: 2.079723e-02
+Iteration:   151, Loss: 1.886188e-02
+Iteration:   176, Loss: 1.737600e-02
+Iteration:   201, Loss: 1.608310e-02
+Iteration:   226, Loss: 1.487726e-02
+Iteration:   251, Loss: 1.359568e-02
+Iteration:   276, Loss: 1.214328e-02
+Iteration:   301, Loss: 1.045338e-02
+Iteration:   326, Loss: 2.002797e-02
+Iteration:   351, Loss: 1.186746e-02
+Iteration:   376, Loss: 9.847032e-03
+Iteration:   401, Loss: 8.622359e-03
+Iteration:   426, Loss: 7.740140e-03
+Iteration:   451, Loss: 6.804725e-03
+Iteration:   476, Loss: 5.482031e-03

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
 prob = ODEProblem(dudt, gdev(u0), (tspan[1], tspan[2]), trained_model.ps)
 sol = solve(prob, Tsit5(); saveat=t)
 pred = convert(AbstractArray, sol) |> cdev
@@ -118,7 +118,7 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.CCjWn1F9.js";const d
     lines!(ax, t, pred[2, :]; label=L"\\hat{u}_2(t)", color=:red, linewidth=4)
     axislegend(ax; position=:lt)
     fig
-end

    Appendix ​

    julia
    using InteractiveUtils
+end

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -178,4 +178,4 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.CCjWn1F9.js";const d
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,29)]))}const g=s(t,[["render",p]]);export{d as __pageData,g as default}; + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.359 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,29)]))}const g=s(t,[["render",p]]);export{d as __pageData,g as default}; diff --git a/dev/assets/tutorials_intermediate_1_NeuralODE.md.B1cz3w_g.js b/dev/assets/tutorials_intermediate_1_NeuralODE.md.d8rfNx5W.js similarity index 96% rename from dev/assets/tutorials_intermediate_1_NeuralODE.md.B1cz3w_g.js rename to dev/assets/tutorials_intermediate_1_NeuralODE.md.d8rfNx5W.js index 8cbd0bea0f..cc54b5347a 100644 --- a/dev/assets/tutorials_intermediate_1_NeuralODE.md.B1cz3w_g.js +++ b/dev/assets/tutorials_intermediate_1_NeuralODE.md.d8rfNx5W.js @@ -107,47 +107,47 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const E end end -train(NeuralODECompact)
    [1/9]	Time 117.1921s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.5168s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
-[3/9]	Time 0.6670s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
-[4/9]	Time 0.4899s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
-[5/9]	Time 0.5107s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
-[6/9]	Time 0.7411s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
-[7/9]	Time 0.4911s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
-[8/9]	Time 0.4896s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
-[9/9]	Time 0.7356s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 35.1691s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4975s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5030s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
-[4/9]	Time 0.4983s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
-[5/9]	Time 0.7347s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
-[6/9]	Time 0.4963s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
-[7/9]	Time 0.5021s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
-[8/9]	Time 0.7346s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
-[9/9]	Time 0.4987s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 41.5691s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4795s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
-[3/9]	Time 0.4873s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
-[4/9]	Time 0.7266s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
-[5/9]	Time 0.4854s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
-[6/9]	Time 0.4804s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
-[7/9]	Time 0.7027s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
-[8/9]	Time 0.4765s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
-[9/9]	Time 0.4773s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 102.8680s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 14.2136s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
-[3/9]	Time 19.1814s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
-[4/9]	Time 17.8224s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
-[5/9]	Time 21.2390s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
-[6/9]	Time 18.3580s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 17.0590s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
-[8/9]	Time 20.5356s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
-[9/9]	Time 26.9693s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 50.5271s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 22.8199s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
-[3/9]	Time 23.3855s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
-[4/9]	Time 30.0392s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
-[5/9]	Time 29.7560s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
-[6/9]	Time 28.8348s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 30.3525s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
-[8/9]	Time 31.6796s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
-[9/9]	Time 25.7284s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
+train(NeuralODECompact)
    [1/9]	Time 119.9114s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4894s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
+[3/9]	Time 0.5984s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
+[4/9]	Time 0.4816s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
+[5/9]	Time 0.4886s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
+[6/9]	Time 0.6129s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
+[7/9]	Time 0.4809s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
+[8/9]	Time 0.6183s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
+[9/9]	Time 0.4846s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 36.1949s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4934s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
+[3/9]	Time 0.6716s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
+[4/9]	Time 0.4907s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
+[5/9]	Time 0.5005s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
+[6/9]	Time 0.5111s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
+[7/9]	Time 0.5156s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
+[8/9]	Time 0.5218s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4958s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 42.7072s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4585s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
+[3/9]	Time 0.5124s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
+[4/9]	Time 0.6816s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
+[5/9]	Time 0.4651s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
+[6/9]	Time 0.4662s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
+[7/9]	Time 0.7050s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
+[8/9]	Time 0.4780s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
+[9/9]	Time 0.4721s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 99.0023s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 16.3657s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
+[3/9]	Time 16.9835s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
+[4/9]	Time 19.8992s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
+[5/9]	Time 19.0555s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
+[6/9]	Time 18.0183s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 19.1584s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
+[8/9]	Time 19.9653s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
+[9/9]	Time 21.4458s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 52.5981s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 28.1671s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
+[3/9]	Time 29.9169s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
+[4/9]	Time 30.6306s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
+[5/9]	Time 27.4726s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
+[6/9]	Time 27.3794s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 27.8209s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
+[8/9]	Time 21.9799s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
+[9/9]	Time 29.7374s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
        Lux.AbstractLuxWrapperLayer{:model}
     model::M
     solver::So
@@ -165,15 +165,15 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const E
     dudt(u, p, t) = st_model(u, p)
     prob = ODEProblem{false}(ODEFunction{false}(dudt), x, n.tspan, ps)
     return solve(prob, n.solver; n.kwargs...), st_model.st
-end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 38.1173s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4742s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
-[3/9]	Time 0.5017s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
-[4/9]	Time 0.7221s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
-[5/9]	Time 0.4642s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
-[6/9]	Time 0.4642s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
-[7/9]	Time 0.4763s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
-[8/9]	Time 0.7946s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
-[9/9]	Time 0.4674s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
+end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 39.1820s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4664s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
+[3/9]	Time 0.6511s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
+[4/9]	Time 0.4597s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
+[5/9]	Time 0.4650s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
+[6/9]	Time 0.4761s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
+[7/9]	Time 0.7648s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
+[8/9]	Time 0.4612s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4605s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
 
 model_stateful, ps_stateful, st_stateful = create_model(StatefulNeuralODE)
 
@@ -269,4 +269,4 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const E
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.889 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default}; + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.920 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default}; diff --git a/dev/assets/tutorials_intermediate_1_NeuralODE.md.B1cz3w_g.lean.js b/dev/assets/tutorials_intermediate_1_NeuralODE.md.d8rfNx5W.lean.js similarity index 96% rename from dev/assets/tutorials_intermediate_1_NeuralODE.md.B1cz3w_g.lean.js rename to dev/assets/tutorials_intermediate_1_NeuralODE.md.d8rfNx5W.lean.js index 8cbd0bea0f..cc54b5347a 100644 --- a/dev/assets/tutorials_intermediate_1_NeuralODE.md.B1cz3w_g.lean.js +++ b/dev/assets/tutorials_intermediate_1_NeuralODE.md.d8rfNx5W.lean.js @@ -107,47 +107,47 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const E end end -train(NeuralODECompact)
    [1/9]	Time 117.1921s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.5168s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
-[3/9]	Time 0.6670s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
-[4/9]	Time 0.4899s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
-[5/9]	Time 0.5107s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
-[6/9]	Time 0.7411s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
-[7/9]	Time 0.4911s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
-[8/9]	Time 0.4896s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
-[9/9]	Time 0.7356s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 35.1691s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4975s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5030s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
-[4/9]	Time 0.4983s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
-[5/9]	Time 0.7347s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
-[6/9]	Time 0.4963s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
-[7/9]	Time 0.5021s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
-[8/9]	Time 0.7346s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
-[9/9]	Time 0.4987s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 41.5691s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4795s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
-[3/9]	Time 0.4873s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
-[4/9]	Time 0.7266s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
-[5/9]	Time 0.4854s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
-[6/9]	Time 0.4804s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
-[7/9]	Time 0.7027s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
-[8/9]	Time 0.4765s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
-[9/9]	Time 0.4773s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 102.8680s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 14.2136s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
-[3/9]	Time 19.1814s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
-[4/9]	Time 17.8224s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
-[5/9]	Time 21.2390s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
-[6/9]	Time 18.3580s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 17.0590s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
-[8/9]	Time 20.5356s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
-[9/9]	Time 26.9693s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 50.5271s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 22.8199s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
-[3/9]	Time 23.3855s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
-[4/9]	Time 30.0392s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
-[5/9]	Time 29.7560s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
-[6/9]	Time 28.8348s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 30.3525s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
-[8/9]	Time 31.6796s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
-[9/9]	Time 25.7284s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
+train(NeuralODECompact)
    [1/9]	Time 119.9114s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4894s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
+[3/9]	Time 0.5984s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
+[4/9]	Time 0.4816s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
+[5/9]	Time 0.4886s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
+[6/9]	Time 0.6129s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
+[7/9]	Time 0.4809s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
+[8/9]	Time 0.6183s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
+[9/9]	Time 0.4846s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 36.1949s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4934s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
+[3/9]	Time 0.6716s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
+[4/9]	Time 0.4907s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
+[5/9]	Time 0.5005s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
+[6/9]	Time 0.5111s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
+[7/9]	Time 0.5156s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
+[8/9]	Time 0.5218s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4958s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 42.7072s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4585s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
+[3/9]	Time 0.5124s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
+[4/9]	Time 0.6816s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
+[5/9]	Time 0.4651s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
+[6/9]	Time 0.4662s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
+[7/9]	Time 0.7050s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
+[8/9]	Time 0.4780s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
+[9/9]	Time 0.4721s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 99.0023s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 16.3657s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
+[3/9]	Time 16.9835s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
+[4/9]	Time 19.8992s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
+[5/9]	Time 19.0555s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
+[6/9]	Time 18.0183s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 19.1584s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
+[8/9]	Time 19.9653s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
+[9/9]	Time 21.4458s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 52.5981s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 28.1671s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
+[3/9]	Time 29.9169s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
+[4/9]	Time 30.6306s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
+[5/9]	Time 27.4726s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
+[6/9]	Time 27.3794s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 27.8209s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
+[8/9]	Time 21.9799s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
+[9/9]	Time 29.7374s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
        Lux.AbstractLuxWrapperLayer{:model}
     model::M
     solver::So
@@ -165,15 +165,15 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const E
     dudt(u, p, t) = st_model(u, p)
     prob = ODEProblem{false}(ODEFunction{false}(dudt), x, n.tspan, ps)
     return solve(prob, n.solver; n.kwargs...), st_model.st
-end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 38.1173s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4742s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
-[3/9]	Time 0.5017s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
-[4/9]	Time 0.7221s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
-[5/9]	Time 0.4642s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
-[6/9]	Time 0.4642s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
-[7/9]	Time 0.4763s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
-[8/9]	Time 0.7946s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
-[9/9]	Time 0.4674s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
+end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 39.1820s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4664s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
+[3/9]	Time 0.6511s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
+[4/9]	Time 0.4597s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
+[5/9]	Time 0.4650s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
+[6/9]	Time 0.4761s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
+[7/9]	Time 0.7648s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
+[8/9]	Time 0.4612s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4605s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
 
 model_stateful, ps_stateful, st_stateful = create_model(StatefulNeuralODE)
 
@@ -269,4 +269,4 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const E
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.889 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default}; + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.920 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default}; diff --git a/dev/assets/tutorials_intermediate_2_BayesianNN.md.e4b93mHN.js b/dev/assets/tutorials_intermediate_2_BayesianNN.md.ErflZY0P.js similarity index 99% rename from dev/assets/tutorials_intermediate_2_BayesianNN.md.e4b93mHN.js rename to dev/assets/tutorials_intermediate_2_BayesianNN.md.ErflZY0P.js index 212ed7b19a..ce1cf03e1e 100644 --- a/dev/assets/tutorials_intermediate_2_BayesianNN.md.e4b93mHN.js +++ b/dev/assets/tutorials_intermediate_2_BayesianNN.md.ErflZY0P.js @@ -85,8 +85,8 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.CCjWn1F9.js"; Iterations = 1:1:5000 Number of chains = 1 Samples per chain = 5000 -Wall duration = 20.89 seconds -Compute duration = 20.89 seconds +Wall duration = 21.82 seconds +Compute duration = 21.82 seconds parameters = parameters[1], parameters[2], parameters[3], parameters[4], parameters[5], parameters[6], parameters[7], parameters[8], parameters[9], parameters[10], parameters[11], parameters[12], parameters[13], parameters[14], parameters[15], parameters[16], parameters[17], parameters[18], parameters[19], parameters[20] internals = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, numerical_error, step_size, nom_step_size @@ -94,26 +94,26 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.CCjWn1F9.js"; parameters mean std mcse ess_bulk ess_tail rhat ess_per_sec Symbol Float64 Float64 Float64 Float64 Float64 Float64 Float64 - parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.5787 - parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 3.8271 - parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.9285 - parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.6877 - parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.7213 - parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 1.0398 - parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.5957 - parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 1.2539 - parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.6770 - parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.6744 - parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.8775 - parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 1.1049 - parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.9310 - parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.6869 - parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.7805 - parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.5017 - parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.7504 - parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 2.1515 - parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.8464 - parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.4596 + parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.5541 + parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 3.6645 + parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.8890 + parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.6585 + parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.6907 + parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 0.9956 + parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.5703 + parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 1.2006 + parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.6482 + parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.6457 + parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.8402 + parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 1.0579 + parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.8915 + parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.6578 + parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.7474 + parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.4379 + parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.7186 + parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 2.0601 + parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.8104 + parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.3976 Quantiles parameters 2.5% 25.0% 50.0% 75.0% 97.5% diff --git a/dev/assets/tutorials_intermediate_2_BayesianNN.md.e4b93mHN.lean.js b/dev/assets/tutorials_intermediate_2_BayesianNN.md.ErflZY0P.lean.js similarity index 99% rename from dev/assets/tutorials_intermediate_2_BayesianNN.md.e4b93mHN.lean.js rename to dev/assets/tutorials_intermediate_2_BayesianNN.md.ErflZY0P.lean.js index 212ed7b19a..ce1cf03e1e 100644 --- a/dev/assets/tutorials_intermediate_2_BayesianNN.md.e4b93mHN.lean.js +++ b/dev/assets/tutorials_intermediate_2_BayesianNN.md.ErflZY0P.lean.js @@ -85,8 +85,8 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.CCjWn1F9.js"; Iterations = 1:1:5000 Number of chains = 1 Samples per chain = 5000 -Wall duration = 20.89 seconds -Compute duration = 20.89 seconds +Wall duration = 21.82 seconds +Compute duration = 21.82 seconds parameters = parameters[1], parameters[2], parameters[3], parameters[4], parameters[5], parameters[6], parameters[7], parameters[8], parameters[9], parameters[10], parameters[11], parameters[12], parameters[13], parameters[14], parameters[15], parameters[16], parameters[17], parameters[18], parameters[19], parameters[20] internals = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, numerical_error, step_size, nom_step_size @@ -94,26 +94,26 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.CCjWn1F9.js"; parameters mean std mcse ess_bulk ess_tail rhat ess_per_sec Symbol Float64 Float64 Float64 Float64 Float64 Float64 Float64 - parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.5787 - parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 3.8271 - parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.9285 - parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.6877 - parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.7213 - parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 1.0398 - parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.5957 - parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 1.2539 - parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.6770 - parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.6744 - parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.8775 - parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 1.1049 - parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.9310 - parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.6869 - parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.7805 - parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.5017 - parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.7504 - parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 2.1515 - parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.8464 - parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.4596 + parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.5541 + parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 3.6645 + parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.8890 + parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.6585 + parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.6907 + parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 0.9956 + parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.5703 + parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 1.2006 + parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.6482 + parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.6457 + parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.8402 + parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 1.0579 + parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.8915 + parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.6578 + parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.7474 + parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.4379 + parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.7186 + parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 2.0601 + parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.8104 + parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.3976 Quantiles parameters 2.5% 25.0% 50.0% 75.0% 97.5% diff --git a/dev/assets/tutorials_intermediate_3_HyperNet.md.DEpFQTc0.js b/dev/assets/tutorials_intermediate_3_HyperNet.md.eD8WCe_f.js similarity index 80% rename from dev/assets/tutorials_intermediate_3_HyperNet.md.DEpFQTc0.js rename to dev/assets/tutorials_intermediate_3_HyperNet.md.eD8WCe_f.js index 2da5660a04..192286b9c5 100644 --- a/dev/assets/tutorials_intermediate_3_HyperNet.md.DEpFQTc0.js +++ b/dev/assets/tutorials_intermediate_3_HyperNet.md.eD8WCe_f.js @@ -107,109 +107,109 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const d return test_acc_list end -test_acc_list = train()
    [  1/ 50]	       MNIST	Time 67.83115s	Training Accuracy: 57.71%	Test Accuracy: 59.38%
-[  1/ 50]	FashionMNIST	Time 0.02742s	Training Accuracy: 51.17%	Test Accuracy: 53.12%
-[  2/ 50]	       MNIST	Time 0.02788s	Training Accuracy: 67.48%	Test Accuracy: 65.62%
-[  2/ 50]	FashionMNIST	Time 0.02874s	Training Accuracy: 63.87%	Test Accuracy: 68.75%
-[  3/ 50]	       MNIST	Time 0.02629s	Training Accuracy: 71.48%	Test Accuracy: 65.62%
-[  3/ 50]	FashionMNIST	Time 0.02589s	Training Accuracy: 68.85%	Test Accuracy: 59.38%
-[  4/ 50]	       MNIST	Time 0.02471s	Training Accuracy: 79.98%	Test Accuracy: 62.50%
-[  4/ 50]	FashionMNIST	Time 0.02508s	Training Accuracy: 71.09%	Test Accuracy: 71.88%
-[  5/ 50]	       MNIST	Time 0.02502s	Training Accuracy: 85.84%	Test Accuracy: 65.62%
-[  5/ 50]	FashionMNIST	Time 0.03813s	Training Accuracy: 76.17%	Test Accuracy: 68.75%
-[  6/ 50]	       MNIST	Time 0.02162s	Training Accuracy: 87.70%	Test Accuracy: 59.38%
-[  6/ 50]	FashionMNIST	Time 0.02053s	Training Accuracy: 79.88%	Test Accuracy: 75.00%
-[  7/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 91.11%	Test Accuracy: 71.88%
-[  7/ 50]	FashionMNIST	Time 0.02198s	Training Accuracy: 81.74%	Test Accuracy: 68.75%
-[  8/ 50]	       MNIST	Time 0.02085s	Training Accuracy: 93.95%	Test Accuracy: 71.88%
-[  8/ 50]	FashionMNIST	Time 0.02078s	Training Accuracy: 82.62%	Test Accuracy: 75.00%
-[  9/ 50]	       MNIST	Time 0.02036s	Training Accuracy: 95.41%	Test Accuracy: 71.88%
-[  9/ 50]	FashionMNIST	Time 0.02046s	Training Accuracy: 79.98%	Test Accuracy: 68.75%
-[ 10/ 50]	       MNIST	Time 0.03479s	Training Accuracy: 96.68%	Test Accuracy: 75.00%
-[ 10/ 50]	FashionMNIST	Time 0.02228s	Training Accuracy: 84.57%	Test Accuracy: 75.00%
-[ 11/ 50]	       MNIST	Time 0.02177s	Training Accuracy: 97.27%	Test Accuracy: 75.00%
-[ 11/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 84.38%	Test Accuracy: 78.12%
-[ 12/ 50]	       MNIST	Time 0.02053s	Training Accuracy: 97.85%	Test Accuracy: 71.88%
-[ 12/ 50]	FashionMNIST	Time 0.02042s	Training Accuracy: 85.74%	Test Accuracy: 75.00%
-[ 13/ 50]	       MNIST	Time 0.02046s	Training Accuracy: 97.75%	Test Accuracy: 68.75%
-[ 13/ 50]	FashionMNIST	Time 0.03820s	Training Accuracy: 85.84%	Test Accuracy: 75.00%
-[ 14/ 50]	       MNIST	Time 0.04972s	Training Accuracy: 98.54%	Test Accuracy: 75.00%
-[ 14/ 50]	FashionMNIST	Time 0.02482s	Training Accuracy: 87.30%	Test Accuracy: 75.00%
-[ 15/ 50]	       MNIST	Time 0.06350s	Training Accuracy: 98.34%	Test Accuracy: 71.88%
-[ 15/ 50]	FashionMNIST	Time 0.04444s	Training Accuracy: 85.16%	Test Accuracy: 78.12%
-[ 16/ 50]	       MNIST	Time 0.03805s	Training Accuracy: 99.32%	Test Accuracy: 75.00%
-[ 16/ 50]	FashionMNIST	Time 0.02524s	Training Accuracy: 84.96%	Test Accuracy: 68.75%
-[ 17/ 50]	       MNIST	Time 0.02049s	Training Accuracy: 99.32%	Test Accuracy: 75.00%
-[ 17/ 50]	FashionMNIST	Time 0.02056s	Training Accuracy: 86.82%	Test Accuracy: 75.00%
-[ 18/ 50]	       MNIST	Time 0.02162s	Training Accuracy: 99.61%	Test Accuracy: 75.00%
-[ 18/ 50]	FashionMNIST	Time 0.02066s	Training Accuracy: 84.47%	Test Accuracy: 68.75%
-[ 19/ 50]	       MNIST	Time 0.02218s	Training Accuracy: 99.80%	Test Accuracy: 75.00%
-[ 19/ 50]	FashionMNIST	Time 0.02053s	Training Accuracy: 86.62%	Test Accuracy: 71.88%
-[ 20/ 50]	       MNIST	Time 0.02384s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 20/ 50]	FashionMNIST	Time 0.02117s	Training Accuracy: 86.13%	Test Accuracy: 71.88%
-[ 21/ 50]	       MNIST	Time 0.02141s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 21/ 50]	FashionMNIST	Time 0.02030s	Training Accuracy: 85.35%	Test Accuracy: 78.12%
-[ 22/ 50]	       MNIST	Time 0.02055s	Training Accuracy: 99.90%	Test Accuracy: 75.00%
-[ 22/ 50]	FashionMNIST	Time 0.02162s	Training Accuracy: 87.30%	Test Accuracy: 71.88%
-[ 23/ 50]	       MNIST	Time 0.03429s	Training Accuracy: 99.71%	Test Accuracy: 75.00%
-[ 23/ 50]	FashionMNIST	Time 0.02119s	Training Accuracy: 87.40%	Test Accuracy: 78.12%
-[ 24/ 50]	       MNIST	Time 0.02052s	Training Accuracy: 99.71%	Test Accuracy: 75.00%
-[ 24/ 50]	FashionMNIST	Time 0.02107s	Training Accuracy: 87.70%	Test Accuracy: 71.88%
-[ 25/ 50]	       MNIST	Time 0.02111s	Training Accuracy: 99.90%	Test Accuracy: 75.00%
-[ 25/ 50]	FashionMNIST	Time 0.02145s	Training Accuracy: 87.99%	Test Accuracy: 81.25%
-[ 26/ 50]	       MNIST	Time 0.02167s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 26/ 50]	FashionMNIST	Time 0.02121s	Training Accuracy: 88.67%	Test Accuracy: 78.12%
-[ 27/ 50]	       MNIST	Time 0.02237s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 27/ 50]	FashionMNIST	Time 0.03032s	Training Accuracy: 89.45%	Test Accuracy: 78.12%
-[ 28/ 50]	       MNIST	Time 0.02097s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 28/ 50]	FashionMNIST	Time 0.02132s	Training Accuracy: 90.43%	Test Accuracy: 81.25%
-[ 29/ 50]	       MNIST	Time 0.02005s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 29/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 91.50%	Test Accuracy: 81.25%
-[ 30/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 30/ 50]	FashionMNIST	Time 0.02047s	Training Accuracy: 91.99%	Test Accuracy: 78.12%
-[ 31/ 50]	       MNIST	Time 0.02050s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 31/ 50]	FashionMNIST	Time 0.02051s	Training Accuracy: 92.19%	Test Accuracy: 81.25%
-[ 32/ 50]	       MNIST	Time 0.02583s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 32/ 50]	FashionMNIST	Time 0.02268s	Training Accuracy: 93.36%	Test Accuracy: 78.12%
-[ 33/ 50]	       MNIST	Time 0.02040s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 33/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 92.77%	Test Accuracy: 75.00%
-[ 34/ 50]	       MNIST	Time 0.02076s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 34/ 50]	FashionMNIST	Time 0.02091s	Training Accuracy: 94.04%	Test Accuracy: 78.12%
-[ 35/ 50]	       MNIST	Time 0.02073s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 35/ 50]	FashionMNIST	Time 0.02027s	Training Accuracy: 94.04%	Test Accuracy: 78.12%
-[ 36/ 50]	       MNIST	Time 0.02058s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 36/ 50]	FashionMNIST	Time 0.01989s	Training Accuracy: 94.73%	Test Accuracy: 81.25%
-[ 37/ 50]	       MNIST	Time 0.02057s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 37/ 50]	FashionMNIST	Time 0.02052s	Training Accuracy: 95.02%	Test Accuracy: 81.25%
-[ 38/ 50]	       MNIST	Time 0.02024s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 38/ 50]	FashionMNIST	Time 0.02059s	Training Accuracy: 95.61%	Test Accuracy: 78.12%
-[ 39/ 50]	       MNIST	Time 0.02044s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 39/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 95.51%	Test Accuracy: 81.25%
-[ 40/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 40/ 50]	FashionMNIST	Time 0.02694s	Training Accuracy: 95.70%	Test Accuracy: 84.38%
-[ 41/ 50]	       MNIST	Time 0.02009s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 41/ 50]	FashionMNIST	Time 0.02056s	Training Accuracy: 96.19%	Test Accuracy: 84.38%
-[ 42/ 50]	       MNIST	Time 0.02053s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 42/ 50]	FashionMNIST	Time 0.02058s	Training Accuracy: 96.19%	Test Accuracy: 84.38%
-[ 43/ 50]	       MNIST	Time 0.02069s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 43/ 50]	FashionMNIST	Time 0.02049s	Training Accuracy: 96.58%	Test Accuracy: 84.38%
-[ 44/ 50]	       MNIST	Time 0.02184s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 44/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 96.39%	Test Accuracy: 84.38%
-[ 45/ 50]	       MNIST	Time 0.02675s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 45/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 96.88%	Test Accuracy: 84.38%
-[ 46/ 50]	       MNIST	Time 0.02074s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 46/ 50]	FashionMNIST	Time 0.02048s	Training Accuracy: 96.97%	Test Accuracy: 81.25%
-[ 47/ 50]	       MNIST	Time 0.02057s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 47/ 50]	FashionMNIST	Time 0.01975s	Training Accuracy: 96.88%	Test Accuracy: 84.38%
-[ 48/ 50]	       MNIST	Time 0.02605s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 48/ 50]	FashionMNIST	Time 0.02151s	Training Accuracy: 97.17%	Test Accuracy: 81.25%
-[ 49/ 50]	       MNIST	Time 0.02068s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 49/ 50]	FashionMNIST	Time 0.04558s	Training Accuracy: 96.97%	Test Accuracy: 84.38%
-[ 50/ 50]	       MNIST	Time 0.04517s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 50/ 50]	FashionMNIST	Time 0.05537s	Training Accuracy: 97.36%	Test Accuracy: 84.38%
+test_acc_list = train()
    [  1/ 50]	       MNIST	Time 69.55185s	Training Accuracy: 62.70%	Test Accuracy: 56.25%
+[  1/ 50]	FashionMNIST	Time 0.01251s	Training Accuracy: 45.90%	Test Accuracy: 37.50%
+[  2/ 50]	       MNIST	Time 0.01233s	Training Accuracy: 72.85%	Test Accuracy: 65.62%
+[  2/ 50]	FashionMNIST	Time 0.01215s	Training Accuracy: 56.84%	Test Accuracy: 50.00%
+[  3/ 50]	       MNIST	Time 0.01216s	Training Accuracy: 74.80%	Test Accuracy: 56.25%
+[  3/ 50]	FashionMNIST	Time 0.01212s	Training Accuracy: 63.18%	Test Accuracy: 62.50%
+[  4/ 50]	       MNIST	Time 0.01212s	Training Accuracy: 83.59%	Test Accuracy: 68.75%
+[  4/ 50]	FashionMNIST	Time 0.01207s	Training Accuracy: 68.26%	Test Accuracy: 59.38%
+[  5/ 50]	       MNIST	Time 0.01219s	Training Accuracy: 83.98%	Test Accuracy: 75.00%
+[  5/ 50]	FashionMNIST	Time 0.01210s	Training Accuracy: 71.39%	Test Accuracy: 53.12%
+[  6/ 50]	       MNIST	Time 0.01214s	Training Accuracy: 87.01%	Test Accuracy: 75.00%
+[  6/ 50]	FashionMNIST	Time 0.03375s	Training Accuracy: 77.73%	Test Accuracy: 65.62%
+[  7/ 50]	       MNIST	Time 0.01088s	Training Accuracy: 88.09%	Test Accuracy: 71.88%
+[  7/ 50]	FashionMNIST	Time 0.01087s	Training Accuracy: 76.86%	Test Accuracy: 68.75%
+[  8/ 50]	       MNIST	Time 0.01083s	Training Accuracy: 92.48%	Test Accuracy: 75.00%
+[  8/ 50]	FashionMNIST	Time 0.01082s	Training Accuracy: 78.61%	Test Accuracy: 71.88%
+[  9/ 50]	       MNIST	Time 0.01074s	Training Accuracy: 93.26%	Test Accuracy: 78.12%
+[  9/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 81.25%	Test Accuracy: 71.88%
+[ 10/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 95.51%	Test Accuracy: 75.00%
+[ 10/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 82.71%	Test Accuracy: 68.75%
+[ 11/ 50]	       MNIST	Time 0.01072s	Training Accuracy: 96.68%	Test Accuracy: 71.88%
+[ 11/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 84.57%	Test Accuracy: 75.00%
+[ 12/ 50]	       MNIST	Time 0.01218s	Training Accuracy: 98.05%	Test Accuracy: 75.00%
+[ 12/ 50]	FashionMNIST	Time 0.03001s	Training Accuracy: 85.35%	Test Accuracy: 78.12%
+[ 13/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 98.73%	Test Accuracy: 75.00%
+[ 13/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 85.35%	Test Accuracy: 75.00%
+[ 14/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 98.73%	Test Accuracy: 81.25%
+[ 14/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 87.79%	Test Accuracy: 78.12%
+[ 15/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 99.12%	Test Accuracy: 75.00%
+[ 15/ 50]	FashionMNIST	Time 0.01060s	Training Accuracy: 88.96%	Test Accuracy: 75.00%
+[ 16/ 50]	       MNIST	Time 0.01060s	Training Accuracy: 99.12%	Test Accuracy: 81.25%
+[ 16/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 83.11%	Test Accuracy: 62.50%
+[ 17/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 99.22%	Test Accuracy: 75.00%
+[ 17/ 50]	FashionMNIST	Time 0.01063s	Training Accuracy: 84.28%	Test Accuracy: 78.12%
+[ 18/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 99.22%	Test Accuracy: 84.38%
+[ 18/ 50]	FashionMNIST	Time 0.02054s	Training Accuracy: 85.74%	Test Accuracy: 78.12%
+[ 19/ 50]	       MNIST	Time 0.01081s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 19/ 50]	FashionMNIST	Time 0.01073s	Training Accuracy: 87.60%	Test Accuracy: 78.12%
+[ 20/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 20/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 85.64%	Test Accuracy: 71.88%
+[ 21/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
+[ 21/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 87.79%	Test Accuracy: 75.00%
+[ 22/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 22/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 89.16%	Test Accuracy: 78.12%
+[ 23/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 23/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 90.82%	Test Accuracy: 75.00%
+[ 24/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 24/ 50]	FashionMNIST	Time 0.02139s	Training Accuracy: 91.89%	Test Accuracy: 75.00%
+[ 25/ 50]	       MNIST	Time 0.01081s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 25/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 92.58%	Test Accuracy: 81.25%
+[ 26/ 50]	       MNIST	Time 0.01069s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 26/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 92.97%	Test Accuracy: 78.12%
+[ 27/ 50]	       MNIST	Time 0.01063s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 27/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 93.85%	Test Accuracy: 78.12%
+[ 28/ 50]	       MNIST	Time 0.01063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 28/ 50]	FashionMNIST	Time 0.01061s	Training Accuracy: 93.36%	Test Accuracy: 84.38%
+[ 29/ 50]	       MNIST	Time 0.01062s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 29/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 91.80%	Test Accuracy: 75.00%
+[ 30/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 30/ 50]	FashionMNIST	Time 0.01107s	Training Accuracy: 92.38%	Test Accuracy: 84.38%
+[ 31/ 50]	       MNIST	Time 0.01085s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 31/ 50]	FashionMNIST	Time 0.01073s	Training Accuracy: 94.14%	Test Accuracy: 81.25%
+[ 32/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 32/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 94.63%	Test Accuracy: 84.38%
+[ 33/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 33/ 50]	FashionMNIST	Time 0.01070s	Training Accuracy: 95.51%	Test Accuracy: 84.38%
+[ 34/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 34/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 96.19%	Test Accuracy: 81.25%
+[ 35/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 35/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 96.09%	Test Accuracy: 81.25%
+[ 36/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 36/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 97.07%	Test Accuracy: 87.50%
+[ 37/ 50]	       MNIST	Time 0.01079s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 37/ 50]	FashionMNIST	Time 0.01177s	Training Accuracy: 97.46%	Test Accuracy: 81.25%
+[ 38/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 38/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 97.17%	Test Accuracy: 84.38%
+[ 39/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 39/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.56%	Test Accuracy: 84.38%
+[ 40/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 40/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.36%	Test Accuracy: 87.50%
+[ 41/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 41/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 97.36%	Test Accuracy: 84.38%
+[ 42/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 42/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.66%	Test Accuracy: 84.38%
+[ 43/ 50]	       MNIST	Time 0.01086s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 43/ 50]	FashionMNIST	Time 0.01189s	Training Accuracy: 97.46%	Test Accuracy: 84.38%
+[ 44/ 50]	       MNIST	Time 0.01077s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 44/ 50]	FashionMNIST	Time 0.01074s	Training Accuracy: 97.85%	Test Accuracy: 84.38%
+[ 45/ 50]	       MNIST	Time 0.01072s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 45/ 50]	FashionMNIST	Time 0.01070s	Training Accuracy: 98.05%	Test Accuracy: 84.38%
+[ 46/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 46/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 97.75%	Test Accuracy: 87.50%
+[ 47/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 47/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 98.54%	Test Accuracy: 81.25%
+[ 48/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 48/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 98.34%	Test Accuracy: 84.38%
+[ 49/ 50]	       MNIST	Time 0.02206s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 49/ 50]	FashionMNIST	Time 0.01183s	Training Accuracy: 98.54%	Test Accuracy: 84.38%
+[ 50/ 50]	       MNIST	Time 0.01088s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 50/ 50]	FashionMNIST	Time 0.01084s	Training Accuracy: 98.63%	Test Accuracy: 84.38%
 
-[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[FINAL]	FashionMNIST	Training Accuracy: 97.36%	Test Accuracy: 84.38%

    Appendix ​

    julia
    using InteractiveUtils
+[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[FINAL]	FashionMNIST	Training Accuracy: 98.63%	Test Accuracy: 84.38%

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -268,5 +268,6 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const d
 Environment:
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
-1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 2.170 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,25)]))}const y=i(p,[["render",l]]);export{d as __pageData,y as default}; +2 devices: + 0: Quadro RTX 5000 (sm_75, 12.100 GiB / 16.000 GiB available) + 1: Quadro RTX 5000 (sm_75, 15.549 GiB / 16.000 GiB available)

    This page was generated using Literate.jl.

    `,25)]))}const y=i(p,[["render",l]]);export{d as __pageData,y as default}; diff --git a/dev/assets/tutorials_intermediate_3_HyperNet.md.DEpFQTc0.lean.js b/dev/assets/tutorials_intermediate_3_HyperNet.md.eD8WCe_f.lean.js similarity index 80% rename from dev/assets/tutorials_intermediate_3_HyperNet.md.DEpFQTc0.lean.js rename to dev/assets/tutorials_intermediate_3_HyperNet.md.eD8WCe_f.lean.js index 2da5660a04..192286b9c5 100644 --- a/dev/assets/tutorials_intermediate_3_HyperNet.md.DEpFQTc0.lean.js +++ b/dev/assets/tutorials_intermediate_3_HyperNet.md.eD8WCe_f.lean.js @@ -107,109 +107,109 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const d return test_acc_list end -test_acc_list = train()
    [  1/ 50]	       MNIST	Time 67.83115s	Training Accuracy: 57.71%	Test Accuracy: 59.38%
-[  1/ 50]	FashionMNIST	Time 0.02742s	Training Accuracy: 51.17%	Test Accuracy: 53.12%
-[  2/ 50]	       MNIST	Time 0.02788s	Training Accuracy: 67.48%	Test Accuracy: 65.62%
-[  2/ 50]	FashionMNIST	Time 0.02874s	Training Accuracy: 63.87%	Test Accuracy: 68.75%
-[  3/ 50]	       MNIST	Time 0.02629s	Training Accuracy: 71.48%	Test Accuracy: 65.62%
-[  3/ 50]	FashionMNIST	Time 0.02589s	Training Accuracy: 68.85%	Test Accuracy: 59.38%
-[  4/ 50]	       MNIST	Time 0.02471s	Training Accuracy: 79.98%	Test Accuracy: 62.50%
-[  4/ 50]	FashionMNIST	Time 0.02508s	Training Accuracy: 71.09%	Test Accuracy: 71.88%
-[  5/ 50]	       MNIST	Time 0.02502s	Training Accuracy: 85.84%	Test Accuracy: 65.62%
-[  5/ 50]	FashionMNIST	Time 0.03813s	Training Accuracy: 76.17%	Test Accuracy: 68.75%
-[  6/ 50]	       MNIST	Time 0.02162s	Training Accuracy: 87.70%	Test Accuracy: 59.38%
-[  6/ 50]	FashionMNIST	Time 0.02053s	Training Accuracy: 79.88%	Test Accuracy: 75.00%
-[  7/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 91.11%	Test Accuracy: 71.88%
-[  7/ 50]	FashionMNIST	Time 0.02198s	Training Accuracy: 81.74%	Test Accuracy: 68.75%
-[  8/ 50]	       MNIST	Time 0.02085s	Training Accuracy: 93.95%	Test Accuracy: 71.88%
-[  8/ 50]	FashionMNIST	Time 0.02078s	Training Accuracy: 82.62%	Test Accuracy: 75.00%
-[  9/ 50]	       MNIST	Time 0.02036s	Training Accuracy: 95.41%	Test Accuracy: 71.88%
-[  9/ 50]	FashionMNIST	Time 0.02046s	Training Accuracy: 79.98%	Test Accuracy: 68.75%
-[ 10/ 50]	       MNIST	Time 0.03479s	Training Accuracy: 96.68%	Test Accuracy: 75.00%
-[ 10/ 50]	FashionMNIST	Time 0.02228s	Training Accuracy: 84.57%	Test Accuracy: 75.00%
-[ 11/ 50]	       MNIST	Time 0.02177s	Training Accuracy: 97.27%	Test Accuracy: 75.00%
-[ 11/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 84.38%	Test Accuracy: 78.12%
-[ 12/ 50]	       MNIST	Time 0.02053s	Training Accuracy: 97.85%	Test Accuracy: 71.88%
-[ 12/ 50]	FashionMNIST	Time 0.02042s	Training Accuracy: 85.74%	Test Accuracy: 75.00%
-[ 13/ 50]	       MNIST	Time 0.02046s	Training Accuracy: 97.75%	Test Accuracy: 68.75%
-[ 13/ 50]	FashionMNIST	Time 0.03820s	Training Accuracy: 85.84%	Test Accuracy: 75.00%
-[ 14/ 50]	       MNIST	Time 0.04972s	Training Accuracy: 98.54%	Test Accuracy: 75.00%
-[ 14/ 50]	FashionMNIST	Time 0.02482s	Training Accuracy: 87.30%	Test Accuracy: 75.00%
-[ 15/ 50]	       MNIST	Time 0.06350s	Training Accuracy: 98.34%	Test Accuracy: 71.88%
-[ 15/ 50]	FashionMNIST	Time 0.04444s	Training Accuracy: 85.16%	Test Accuracy: 78.12%
-[ 16/ 50]	       MNIST	Time 0.03805s	Training Accuracy: 99.32%	Test Accuracy: 75.00%
-[ 16/ 50]	FashionMNIST	Time 0.02524s	Training Accuracy: 84.96%	Test Accuracy: 68.75%
-[ 17/ 50]	       MNIST	Time 0.02049s	Training Accuracy: 99.32%	Test Accuracy: 75.00%
-[ 17/ 50]	FashionMNIST	Time 0.02056s	Training Accuracy: 86.82%	Test Accuracy: 75.00%
-[ 18/ 50]	       MNIST	Time 0.02162s	Training Accuracy: 99.61%	Test Accuracy: 75.00%
-[ 18/ 50]	FashionMNIST	Time 0.02066s	Training Accuracy: 84.47%	Test Accuracy: 68.75%
-[ 19/ 50]	       MNIST	Time 0.02218s	Training Accuracy: 99.80%	Test Accuracy: 75.00%
-[ 19/ 50]	FashionMNIST	Time 0.02053s	Training Accuracy: 86.62%	Test Accuracy: 71.88%
-[ 20/ 50]	       MNIST	Time 0.02384s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 20/ 50]	FashionMNIST	Time 0.02117s	Training Accuracy: 86.13%	Test Accuracy: 71.88%
-[ 21/ 50]	       MNIST	Time 0.02141s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 21/ 50]	FashionMNIST	Time 0.02030s	Training Accuracy: 85.35%	Test Accuracy: 78.12%
-[ 22/ 50]	       MNIST	Time 0.02055s	Training Accuracy: 99.90%	Test Accuracy: 75.00%
-[ 22/ 50]	FashionMNIST	Time 0.02162s	Training Accuracy: 87.30%	Test Accuracy: 71.88%
-[ 23/ 50]	       MNIST	Time 0.03429s	Training Accuracy: 99.71%	Test Accuracy: 75.00%
-[ 23/ 50]	FashionMNIST	Time 0.02119s	Training Accuracy: 87.40%	Test Accuracy: 78.12%
-[ 24/ 50]	       MNIST	Time 0.02052s	Training Accuracy: 99.71%	Test Accuracy: 75.00%
-[ 24/ 50]	FashionMNIST	Time 0.02107s	Training Accuracy: 87.70%	Test Accuracy: 71.88%
-[ 25/ 50]	       MNIST	Time 0.02111s	Training Accuracy: 99.90%	Test Accuracy: 75.00%
-[ 25/ 50]	FashionMNIST	Time 0.02145s	Training Accuracy: 87.99%	Test Accuracy: 81.25%
-[ 26/ 50]	       MNIST	Time 0.02167s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 26/ 50]	FashionMNIST	Time 0.02121s	Training Accuracy: 88.67%	Test Accuracy: 78.12%
-[ 27/ 50]	       MNIST	Time 0.02237s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 27/ 50]	FashionMNIST	Time 0.03032s	Training Accuracy: 89.45%	Test Accuracy: 78.12%
-[ 28/ 50]	       MNIST	Time 0.02097s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 28/ 50]	FashionMNIST	Time 0.02132s	Training Accuracy: 90.43%	Test Accuracy: 81.25%
-[ 29/ 50]	       MNIST	Time 0.02005s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 29/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 91.50%	Test Accuracy: 81.25%
-[ 30/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 30/ 50]	FashionMNIST	Time 0.02047s	Training Accuracy: 91.99%	Test Accuracy: 78.12%
-[ 31/ 50]	       MNIST	Time 0.02050s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 31/ 50]	FashionMNIST	Time 0.02051s	Training Accuracy: 92.19%	Test Accuracy: 81.25%
-[ 32/ 50]	       MNIST	Time 0.02583s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 32/ 50]	FashionMNIST	Time 0.02268s	Training Accuracy: 93.36%	Test Accuracy: 78.12%
-[ 33/ 50]	       MNIST	Time 0.02040s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 33/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 92.77%	Test Accuracy: 75.00%
-[ 34/ 50]	       MNIST	Time 0.02076s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 34/ 50]	FashionMNIST	Time 0.02091s	Training Accuracy: 94.04%	Test Accuracy: 78.12%
-[ 35/ 50]	       MNIST	Time 0.02073s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 35/ 50]	FashionMNIST	Time 0.02027s	Training Accuracy: 94.04%	Test Accuracy: 78.12%
-[ 36/ 50]	       MNIST	Time 0.02058s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 36/ 50]	FashionMNIST	Time 0.01989s	Training Accuracy: 94.73%	Test Accuracy: 81.25%
-[ 37/ 50]	       MNIST	Time 0.02057s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 37/ 50]	FashionMNIST	Time 0.02052s	Training Accuracy: 95.02%	Test Accuracy: 81.25%
-[ 38/ 50]	       MNIST	Time 0.02024s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 38/ 50]	FashionMNIST	Time 0.02059s	Training Accuracy: 95.61%	Test Accuracy: 78.12%
-[ 39/ 50]	       MNIST	Time 0.02044s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 39/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 95.51%	Test Accuracy: 81.25%
-[ 40/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 40/ 50]	FashionMNIST	Time 0.02694s	Training Accuracy: 95.70%	Test Accuracy: 84.38%
-[ 41/ 50]	       MNIST	Time 0.02009s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 41/ 50]	FashionMNIST	Time 0.02056s	Training Accuracy: 96.19%	Test Accuracy: 84.38%
-[ 42/ 50]	       MNIST	Time 0.02053s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 42/ 50]	FashionMNIST	Time 0.02058s	Training Accuracy: 96.19%	Test Accuracy: 84.38%
-[ 43/ 50]	       MNIST	Time 0.02069s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 43/ 50]	FashionMNIST	Time 0.02049s	Training Accuracy: 96.58%	Test Accuracy: 84.38%
-[ 44/ 50]	       MNIST	Time 0.02184s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 44/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 96.39%	Test Accuracy: 84.38%
-[ 45/ 50]	       MNIST	Time 0.02675s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 45/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 96.88%	Test Accuracy: 84.38%
-[ 46/ 50]	       MNIST	Time 0.02074s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 46/ 50]	FashionMNIST	Time 0.02048s	Training Accuracy: 96.97%	Test Accuracy: 81.25%
-[ 47/ 50]	       MNIST	Time 0.02057s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 47/ 50]	FashionMNIST	Time 0.01975s	Training Accuracy: 96.88%	Test Accuracy: 84.38%
-[ 48/ 50]	       MNIST	Time 0.02605s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 48/ 50]	FashionMNIST	Time 0.02151s	Training Accuracy: 97.17%	Test Accuracy: 81.25%
-[ 49/ 50]	       MNIST	Time 0.02068s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 49/ 50]	FashionMNIST	Time 0.04558s	Training Accuracy: 96.97%	Test Accuracy: 84.38%
-[ 50/ 50]	       MNIST	Time 0.04517s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 50/ 50]	FashionMNIST	Time 0.05537s	Training Accuracy: 97.36%	Test Accuracy: 84.38%
+test_acc_list = train()
    [  1/ 50]	       MNIST	Time 69.55185s	Training Accuracy: 62.70%	Test Accuracy: 56.25%
+[  1/ 50]	FashionMNIST	Time 0.01251s	Training Accuracy: 45.90%	Test Accuracy: 37.50%
+[  2/ 50]	       MNIST	Time 0.01233s	Training Accuracy: 72.85%	Test Accuracy: 65.62%
+[  2/ 50]	FashionMNIST	Time 0.01215s	Training Accuracy: 56.84%	Test Accuracy: 50.00%
+[  3/ 50]	       MNIST	Time 0.01216s	Training Accuracy: 74.80%	Test Accuracy: 56.25%
+[  3/ 50]	FashionMNIST	Time 0.01212s	Training Accuracy: 63.18%	Test Accuracy: 62.50%
+[  4/ 50]	       MNIST	Time 0.01212s	Training Accuracy: 83.59%	Test Accuracy: 68.75%
+[  4/ 50]	FashionMNIST	Time 0.01207s	Training Accuracy: 68.26%	Test Accuracy: 59.38%
+[  5/ 50]	       MNIST	Time 0.01219s	Training Accuracy: 83.98%	Test Accuracy: 75.00%
+[  5/ 50]	FashionMNIST	Time 0.01210s	Training Accuracy: 71.39%	Test Accuracy: 53.12%
+[  6/ 50]	       MNIST	Time 0.01214s	Training Accuracy: 87.01%	Test Accuracy: 75.00%
+[  6/ 50]	FashionMNIST	Time 0.03375s	Training Accuracy: 77.73%	Test Accuracy: 65.62%
+[  7/ 50]	       MNIST	Time 0.01088s	Training Accuracy: 88.09%	Test Accuracy: 71.88%
+[  7/ 50]	FashionMNIST	Time 0.01087s	Training Accuracy: 76.86%	Test Accuracy: 68.75%
+[  8/ 50]	       MNIST	Time 0.01083s	Training Accuracy: 92.48%	Test Accuracy: 75.00%
+[  8/ 50]	FashionMNIST	Time 0.01082s	Training Accuracy: 78.61%	Test Accuracy: 71.88%
+[  9/ 50]	       MNIST	Time 0.01074s	Training Accuracy: 93.26%	Test Accuracy: 78.12%
+[  9/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 81.25%	Test Accuracy: 71.88%
+[ 10/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 95.51%	Test Accuracy: 75.00%
+[ 10/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 82.71%	Test Accuracy: 68.75%
+[ 11/ 50]	       MNIST	Time 0.01072s	Training Accuracy: 96.68%	Test Accuracy: 71.88%
+[ 11/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 84.57%	Test Accuracy: 75.00%
+[ 12/ 50]	       MNIST	Time 0.01218s	Training Accuracy: 98.05%	Test Accuracy: 75.00%
+[ 12/ 50]	FashionMNIST	Time 0.03001s	Training Accuracy: 85.35%	Test Accuracy: 78.12%
+[ 13/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 98.73%	Test Accuracy: 75.00%
+[ 13/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 85.35%	Test Accuracy: 75.00%
+[ 14/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 98.73%	Test Accuracy: 81.25%
+[ 14/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 87.79%	Test Accuracy: 78.12%
+[ 15/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 99.12%	Test Accuracy: 75.00%
+[ 15/ 50]	FashionMNIST	Time 0.01060s	Training Accuracy: 88.96%	Test Accuracy: 75.00%
+[ 16/ 50]	       MNIST	Time 0.01060s	Training Accuracy: 99.12%	Test Accuracy: 81.25%
+[ 16/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 83.11%	Test Accuracy: 62.50%
+[ 17/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 99.22%	Test Accuracy: 75.00%
+[ 17/ 50]	FashionMNIST	Time 0.01063s	Training Accuracy: 84.28%	Test Accuracy: 78.12%
+[ 18/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 99.22%	Test Accuracy: 84.38%
+[ 18/ 50]	FashionMNIST	Time 0.02054s	Training Accuracy: 85.74%	Test Accuracy: 78.12%
+[ 19/ 50]	       MNIST	Time 0.01081s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 19/ 50]	FashionMNIST	Time 0.01073s	Training Accuracy: 87.60%	Test Accuracy: 78.12%
+[ 20/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 20/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 85.64%	Test Accuracy: 71.88%
+[ 21/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
+[ 21/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 87.79%	Test Accuracy: 75.00%
+[ 22/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 22/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 89.16%	Test Accuracy: 78.12%
+[ 23/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 23/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 90.82%	Test Accuracy: 75.00%
+[ 24/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 24/ 50]	FashionMNIST	Time 0.02139s	Training Accuracy: 91.89%	Test Accuracy: 75.00%
+[ 25/ 50]	       MNIST	Time 0.01081s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 25/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 92.58%	Test Accuracy: 81.25%
+[ 26/ 50]	       MNIST	Time 0.01069s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 26/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 92.97%	Test Accuracy: 78.12%
+[ 27/ 50]	       MNIST	Time 0.01063s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 27/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 93.85%	Test Accuracy: 78.12%
+[ 28/ 50]	       MNIST	Time 0.01063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 28/ 50]	FashionMNIST	Time 0.01061s	Training Accuracy: 93.36%	Test Accuracy: 84.38%
+[ 29/ 50]	       MNIST	Time 0.01062s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 29/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 91.80%	Test Accuracy: 75.00%
+[ 30/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 30/ 50]	FashionMNIST	Time 0.01107s	Training Accuracy: 92.38%	Test Accuracy: 84.38%
+[ 31/ 50]	       MNIST	Time 0.01085s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 31/ 50]	FashionMNIST	Time 0.01073s	Training Accuracy: 94.14%	Test Accuracy: 81.25%
+[ 32/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 32/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 94.63%	Test Accuracy: 84.38%
+[ 33/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 33/ 50]	FashionMNIST	Time 0.01070s	Training Accuracy: 95.51%	Test Accuracy: 84.38%
+[ 34/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 34/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 96.19%	Test Accuracy: 81.25%
+[ 35/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 35/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 96.09%	Test Accuracy: 81.25%
+[ 36/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 36/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 97.07%	Test Accuracy: 87.50%
+[ 37/ 50]	       MNIST	Time 0.01079s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 37/ 50]	FashionMNIST	Time 0.01177s	Training Accuracy: 97.46%	Test Accuracy: 81.25%
+[ 38/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 38/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 97.17%	Test Accuracy: 84.38%
+[ 39/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 39/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.56%	Test Accuracy: 84.38%
+[ 40/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 40/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.36%	Test Accuracy: 87.50%
+[ 41/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 41/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 97.36%	Test Accuracy: 84.38%
+[ 42/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 42/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.66%	Test Accuracy: 84.38%
+[ 43/ 50]	       MNIST	Time 0.01086s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 43/ 50]	FashionMNIST	Time 0.01189s	Training Accuracy: 97.46%	Test Accuracy: 84.38%
+[ 44/ 50]	       MNIST	Time 0.01077s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 44/ 50]	FashionMNIST	Time 0.01074s	Training Accuracy: 97.85%	Test Accuracy: 84.38%
+[ 45/ 50]	       MNIST	Time 0.01072s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 45/ 50]	FashionMNIST	Time 0.01070s	Training Accuracy: 98.05%	Test Accuracy: 84.38%
+[ 46/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 46/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 97.75%	Test Accuracy: 87.50%
+[ 47/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 47/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 98.54%	Test Accuracy: 81.25%
+[ 48/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 48/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 98.34%	Test Accuracy: 84.38%
+[ 49/ 50]	       MNIST	Time 0.02206s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 49/ 50]	FashionMNIST	Time 0.01183s	Training Accuracy: 98.54%	Test Accuracy: 84.38%
+[ 50/ 50]	       MNIST	Time 0.01088s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 50/ 50]	FashionMNIST	Time 0.01084s	Training Accuracy: 98.63%	Test Accuracy: 84.38%
 
-[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[FINAL]	FashionMNIST	Training Accuracy: 97.36%	Test Accuracy: 84.38%

    Appendix ​

    julia
    using InteractiveUtils
+[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[FINAL]	FashionMNIST	Training Accuracy: 98.63%	Test Accuracy: 84.38%

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -268,5 +268,6 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.CCjWn1F9.js";const d
 Environment:
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
-1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 2.170 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    `,25)]))}const y=i(p,[["render",l]]);export{d as __pageData,y as default}; +2 devices: + 0: Quadro RTX 5000 (sm_75, 12.100 GiB / 16.000 GiB available) + 1: Quadro RTX 5000 (sm_75, 15.549 GiB / 16.000 GiB available)

    This page was generated using Literate.jl.

    `,25)]))}const y=i(p,[["render",l]]);export{d as __pageData,y as default}; diff --git a/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.C3tygY0O.js b/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.DSWj7UgP.js similarity index 84% rename from dev/assets/tutorials_intermediate_4_PINN2DPDE.md.C3tygY0O.js rename to dev/assets/tutorials_intermediate_4_PINN2DPDE.md.DSWj7UgP.js index 9732c2c1f8..417875e8a4 100644 --- a/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.C3tygY0O.js +++ b/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.DSWj7UgP.js @@ -1,4 +1,4 @@ -import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1F9.js";const p="/dev/assets/pinn_nested_ad.B__JnolW.gif",m=JSON.parse('{"title":"Training a PINN on 2D PDE","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/intermediate/4_PINN2DPDE.md","filePath":"tutorials/intermediate/4_PINN2DPDE.md","lastUpdated":null}'),k={name:"tutorials/intermediate/4_PINN2DPDE.md"},e={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.586ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3795.2 1000","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.401ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3713.2 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.109ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3584.2 1000","aria-hidden":"true"};function o(c,i,F,C,D,L){return l(),n("div",null,[i[10]||(i[10]=a(`

    Training a PINN on 2D PDE ​

    In this tutorial we will go over using a PINN to solve 2D PDEs. We will be using the system from NeuralPDE Tutorials. However, we will be using our custom loss function and use nested AD capabilities of Lux.jl.

    This is a demonstration of Lux.jl. For serious usecases of PINNs, please refer to the package: NeuralPDE.jl.

    Package Imports ​

    julia
    using ADTypes, Lux, Optimisers, Zygote, Random, Printf, Statistics, MLUtils, OnlineStats,
+import{_ as p,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1F9.js";const h="/dev/assets/pinn_nested_ad.Bwdus6h-.gif",m=JSON.parse('{"title":"Training a PINN on 2D PDE","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/intermediate/4_PINN2DPDE.md","filePath":"tutorials/intermediate/4_PINN2DPDE.md","lastUpdated":null}'),k={name:"tutorials/intermediate/4_PINN2DPDE.md"},e={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.586ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3795.2 1000","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.401ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3713.2 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.109ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3584.2 1000","aria-hidden":"true"};function o(c,i,F,C,L,D){return l(),n("div",null,[i[10]||(i[10]=a(`
    Training a PINN on 2D PDE ​
    In this tutorial we will go over using a PINN to solve 2D PDEs. We will be using the system from NeuralPDE Tutorials. However, we will be using our custom loss function and use nested AD capabilities of Lux.jl.
    This is a demonstration of Lux.jl. For serious usecases of PINNs, please refer to the package: NeuralPDE.jl.
    Package Imports ​
    julia
    using ADTypes, Lux, Optimisers, Zygote, Random, Printf, Statistics, MLUtils, OnlineStats,
       CairoMakie
 using LuxCUDA
 
@@ -147,116 +147,116 @@ import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1
 trained_u = Lux.testmode(StatefulLuxLayer{true}(
     trained_model.model.u, trained_model.ps.u, trained_model.st.u))
    ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
-Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162476 (1.982162476) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505080 (0.598505080)
-Iteration: [  501 / 50000] 	 Loss: 0.040918160 (0.025583776) 	 Physics Loss: 0.000391877 (0.000269295) 	 Data Loss: 0.014243508 (0.009196416) 	 BC Loss: 0.026282774 (0.016118063)
-Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281426) 	 Physics Loss: 0.000071670 (0.000163182) 	 Data Loss: 0.007905648 (0.010876314) 	 BC Loss: 0.007363341 (0.014241929)
-Iteration: [ 1501 / 50000] 	 Loss: 0.019567011 (0.026170366) 	 Physics Loss: 0.001279382 (0.001009728) 	 Data Loss: 0.003071612 (0.010452257) 	 BC Loss: 0.015216017 (0.014708381)
-Iteration: [ 2001 / 50000] 	 Loss: 0.035556547 (0.027273500) 	 Physics Loss: 0.004061943 (0.001871931) 	 Data Loss: 0.013011228 (0.010586374) 	 BC Loss: 0.018483378 (0.014815190)
-Iteration: [ 2501 / 50000] 	 Loss: 0.011505678 (0.022150228) 	 Physics Loss: 0.002304791 (0.001940841) 	 Data Loss: 0.005615299 (0.007885863) 	 BC Loss: 0.003585588 (0.012323526)
-Iteration: [ 3001 / 50000] 	 Loss: 0.031768262 (0.029164422) 	 Physics Loss: 0.008404830 (0.003920683) 	 Data Loss: 0.008808360 (0.010936455) 	 BC Loss: 0.014555071 (0.014307282)
-Iteration: [ 3501 / 50000] 	 Loss: 0.017645847 (0.042499166) 	 Physics Loss: 0.001848093 (0.001690981) 	 Data Loss: 0.005461216 (0.018073166) 	 BC Loss: 0.010336538 (0.022735020)
-Iteration: [ 4001 / 50000] 	 Loss: 0.028128654 (0.027620461) 	 Physics Loss: 0.005112350 (0.002448601) 	 Data Loss: 0.013499700 (0.011073254) 	 BC Loss: 0.009516605 (0.014098606)
-Iteration: [ 4501 / 50000] 	 Loss: 0.014340003 (0.033320315) 	 Physics Loss: 0.001292084 (0.004329988) 	 Data Loss: 0.008556721 (0.012002973) 	 BC Loss: 0.004491198 (0.016987354)
-Iteration: [ 5001 / 50000] 	 Loss: 0.030331207 (0.041541956) 	 Physics Loss: 0.000723805 (0.002695386) 	 Data Loss: 0.004466736 (0.016228525) 	 BC Loss: 0.025140665 (0.022618050)
-Iteration: [ 5501 / 50000] 	 Loss: 0.022293953 (0.021166507) 	 Physics Loss: 0.000554349 (0.000707158) 	 Data Loss: 0.004542338 (0.007950513) 	 BC Loss: 0.017197266 (0.012508835)
-Iteration: [ 6001 / 50000] 	 Loss: 0.018723227 (0.020389127) 	 Physics Loss: 0.000442930 (0.000975201) 	 Data Loss: 0.006906096 (0.007273634) 	 BC Loss: 0.011374202 (0.012140292)
-Iteration: [ 6501 / 50000] 	 Loss: 0.028305896 (0.020493284) 	 Physics Loss: 0.000753467 (0.001849154) 	 Data Loss: 0.011062279 (0.008014920) 	 BC Loss: 0.016490150 (0.010629211)
-Iteration: [ 7001 / 50000] 	 Loss: 0.015494239 (0.020624701) 	 Physics Loss: 0.007333768 (0.001517879) 	 Data Loss: 0.004043682 (0.008185850) 	 BC Loss: 0.004116789 (0.010920972)
-Iteration: [ 7501 / 50000] 	 Loss: 0.019155467 (0.018008159) 	 Physics Loss: 0.002511769 (0.001448493) 	 Data Loss: 0.009514628 (0.006572613) 	 BC Loss: 0.007129070 (0.009987052)
-Iteration: [ 8001 / 50000] 	 Loss: 0.018706074 (0.017215939) 	 Physics Loss: 0.001792779 (0.001894138) 	 Data Loss: 0.005628760 (0.005433898) 	 BC Loss: 0.011284535 (0.009887908)
-Iteration: [ 8501 / 50000] 	 Loss: 0.017605182 (0.019859709) 	 Physics Loss: 0.002696904 (0.003125851) 	 Data Loss: 0.007372384 (0.005978104) 	 BC Loss: 0.007535893 (0.010755754)
-Iteration: [ 9001 / 50000] 	 Loss: 0.017614847 (0.016386209) 	 Physics Loss: 0.003162773 (0.002853032) 	 Data Loss: 0.003553676 (0.004607514) 	 BC Loss: 0.010898398 (0.008925664)
-Iteration: [ 9501 / 50000] 	 Loss: 0.008732248 (0.014530762) 	 Physics Loss: 0.001913586 (0.002123826) 	 Data Loss: 0.003329928 (0.004253434) 	 BC Loss: 0.003488734 (0.008153505)
-Iteration: [10001 / 50000] 	 Loss: 0.017750096 (0.017799046) 	 Physics Loss: 0.003630795 (0.003061282) 	 Data Loss: 0.003031955 (0.005044522) 	 BC Loss: 0.011087346 (0.009693242)
-Iteration: [10501 / 50000] 	 Loss: 0.007509941 (0.011342090) 	 Physics Loss: 0.001571818 (0.001135202) 	 Data Loss: 0.001611185 (0.002613829) 	 BC Loss: 0.004326937 (0.007593059)
-Iteration: [11001 / 50000] 	 Loss: 0.022739200 (0.011915382) 	 Physics Loss: 0.000807529 (0.001103305) 	 Data Loss: 0.002606506 (0.003132161) 	 BC Loss: 0.019325165 (0.007679918)
-Iteration: [11501 / 50000] 	 Loss: 0.019484218 (0.011457845) 	 Physics Loss: 0.001845157 (0.001325290) 	 Data Loss: 0.002358936 (0.002806861) 	 BC Loss: 0.015280124 (0.007325693)
-Iteration: [12001 / 50000] 	 Loss: 0.019427970 (0.011598296) 	 Physics Loss: 0.004434146 (0.001386037) 	 Data Loss: 0.008772802 (0.003136263) 	 BC Loss: 0.006221022 (0.007075997)
-Iteration: [12501 / 50000] 	 Loss: 0.012775686 (0.011906523) 	 Physics Loss: 0.001197650 (0.001402911) 	 Data Loss: 0.001059842 (0.002556076) 	 BC Loss: 0.010518193 (0.007947534)
-Iteration: [13001 / 50000] 	 Loss: 0.006105572 (0.011037273) 	 Physics Loss: 0.000987124 (0.001454655) 	 Data Loss: 0.001305370 (0.002395016) 	 BC Loss: 0.003813077 (0.007187600)
-Iteration: [13501 / 50000] 	 Loss: 0.010004668 (0.011247103) 	 Physics Loss: 0.001224264 (0.001615586) 	 Data Loss: 0.002474443 (0.002668936) 	 BC Loss: 0.006305961 (0.006962581)
-Iteration: [14001 / 50000] 	 Loss: 0.009895653 (0.009313912) 	 Physics Loss: 0.001215764 (0.001694928) 	 Data Loss: 0.002037087 (0.002155375) 	 BC Loss: 0.006642802 (0.005463609)
-Iteration: [14501 / 50000] 	 Loss: 0.014400685 (0.008724037) 	 Physics Loss: 0.001658659 (0.001564218) 	 Data Loss: 0.003658900 (0.002201537) 	 BC Loss: 0.009083126 (0.004958282)
-Iteration: [15001 / 50000] 	 Loss: 0.007676640 (0.008405063) 	 Physics Loss: 0.001566568 (0.001608545) 	 Data Loss: 0.002262217 (0.001837625) 	 BC Loss: 0.003847855 (0.004958895)
-Iteration: [15501 / 50000] 	 Loss: 0.004365115 (0.009256126) 	 Physics Loss: 0.001047856 (0.002247394) 	 Data Loss: 0.000648518 (0.002218451) 	 BC Loss: 0.002668741 (0.004790282)
-Iteration: [16001 / 50000] 	 Loss: 0.004880759 (0.007501942) 	 Physics Loss: 0.001649067 (0.001671217) 	 Data Loss: 0.000296087 (0.001492234) 	 BC Loss: 0.002935605 (0.004338491)
-Iteration: [16501 / 50000] 	 Loss: 0.008074892 (0.007220342) 	 Physics Loss: 0.001825325 (0.001970405) 	 Data Loss: 0.001302087 (0.001730468) 	 BC Loss: 0.004947479 (0.003519468)
-Iteration: [17001 / 50000] 	 Loss: 0.005824474 (0.005835793) 	 Physics Loss: 0.002019164 (0.001728887) 	 Data Loss: 0.000624455 (0.001011056) 	 BC Loss: 0.003180855 (0.003095849)
-Iteration: [17501 / 50000] 	 Loss: 0.006616294 (0.005807751) 	 Physics Loss: 0.002489866 (0.001884541) 	 Data Loss: 0.001154157 (0.001245214) 	 BC Loss: 0.002972272 (0.002677995)
-Iteration: [18001 / 50000] 	 Loss: 0.004335414 (0.005200472) 	 Physics Loss: 0.001764537 (0.002015869) 	 Data Loss: 0.000705982 (0.001058994) 	 BC Loss: 0.001864895 (0.002125610)
-Iteration: [18501 / 50000] 	 Loss: 0.004978007 (0.005351806) 	 Physics Loss: 0.002553018 (0.002129085) 	 Data Loss: 0.000752965 (0.001336156) 	 BC Loss: 0.001672024 (0.001886566)
-Iteration: [19001 / 50000] 	 Loss: 0.004518208 (0.004657542) 	 Physics Loss: 0.001975382 (0.001959185) 	 Data Loss: 0.000239237 (0.000983244) 	 BC Loss: 0.002303589 (0.001715113)
-Iteration: [19501 / 50000] 	 Loss: 0.006942283 (0.004119421) 	 Physics Loss: 0.004590358 (0.001714717) 	 Data Loss: 0.001148389 (0.000894285) 	 BC Loss: 0.001203537 (0.001510418)
-Iteration: [20001 / 50000] 	 Loss: 0.008026988 (0.003463188) 	 Physics Loss: 0.005315070 (0.001485110) 	 Data Loss: 0.001919697 (0.000688489) 	 BC Loss: 0.000792221 (0.001289588)
-Iteration: [20501 / 50000] 	 Loss: 0.004855067 (0.003504427) 	 Physics Loss: 0.001560361 (0.001747428) 	 Data Loss: 0.000460401 (0.000618138) 	 BC Loss: 0.002834306 (0.001138862)
-Iteration: [21001 / 50000] 	 Loss: 0.002417301 (0.003299790) 	 Physics Loss: 0.000924424 (0.001513747) 	 Data Loss: 0.000300984 (0.000659882) 	 BC Loss: 0.001191893 (0.001126162)
-Iteration: [21501 / 50000] 	 Loss: 0.003911218 (0.002670718) 	 Physics Loss: 0.002752088 (0.001218763) 	 Data Loss: 0.000835131 (0.000637475) 	 BC Loss: 0.000323999 (0.000814481)
-Iteration: [22001 / 50000] 	 Loss: 0.002318957 (0.002511334) 	 Physics Loss: 0.001409405 (0.001309370) 	 Data Loss: 0.000344849 (0.000535839) 	 BC Loss: 0.000564703 (0.000666125)
-Iteration: [22501 / 50000] 	 Loss: 0.001994215 (0.002343247) 	 Physics Loss: 0.000746255 (0.001113559) 	 Data Loss: 0.000431564 (0.000577084) 	 BC Loss: 0.000816396 (0.000652603)
-Iteration: [23001 / 50000] 	 Loss: 0.002989073 (0.002301548) 	 Physics Loss: 0.001930058 (0.001243673) 	 Data Loss: 0.000805100 (0.000536315) 	 BC Loss: 0.000253915 (0.000521560)
-Iteration: [23501 / 50000] 	 Loss: 0.002210422 (0.002371583) 	 Physics Loss: 0.001325711 (0.001179150) 	 Data Loss: 0.000160725 (0.000516864) 	 BC Loss: 0.000723986 (0.000675569)
-Iteration: [24001 / 50000] 	 Loss: 0.002023818 (0.002276814) 	 Physics Loss: 0.000857553 (0.001135292) 	 Data Loss: 0.000525994 (0.000580056) 	 BC Loss: 0.000640270 (0.000561465)
-Iteration: [24501 / 50000] 	 Loss: 0.002234935 (0.002236427) 	 Physics Loss: 0.001041825 (0.001241080) 	 Data Loss: 0.000286281 (0.000510494) 	 BC Loss: 0.000906830 (0.000484853)
-Iteration: [25001 / 50000] 	 Loss: 0.001972227 (0.001982885) 	 Physics Loss: 0.000893347 (0.000971555) 	 Data Loss: 0.000666251 (0.000537486) 	 BC Loss: 0.000412629 (0.000473843)
-Iteration: [25501 / 50000] 	 Loss: 0.001659834 (0.002081697) 	 Physics Loss: 0.001085884 (0.001258897) 	 Data Loss: 0.000359855 (0.000474958) 	 BC Loss: 0.000214095 (0.000347842)
-Iteration: [26001 / 50000] 	 Loss: 0.002059042 (0.001770784) 	 Physics Loss: 0.001171167 (0.000890729) 	 Data Loss: 0.000426463 (0.000447986) 	 BC Loss: 0.000461412 (0.000432069)
-Iteration: [26501 / 50000] 	 Loss: 0.002089650 (0.002068657) 	 Physics Loss: 0.001594031 (0.001356683) 	 Data Loss: 0.000330550 (0.000420685) 	 BC Loss: 0.000165069 (0.000291289)
-Iteration: [27001 / 50000] 	 Loss: 0.001346401 (0.001879478) 	 Physics Loss: 0.000798861 (0.001050717) 	 Data Loss: 0.000333685 (0.000522715) 	 BC Loss: 0.000213855 (0.000306047)
-Iteration: [27501 / 50000] 	 Loss: 0.001717428 (0.001494603) 	 Physics Loss: 0.000754971 (0.000796866) 	 Data Loss: 0.000408040 (0.000351864) 	 BC Loss: 0.000554417 (0.000345872)
-Iteration: [28001 / 50000] 	 Loss: 0.001498525 (0.001789054) 	 Physics Loss: 0.000881248 (0.001008533) 	 Data Loss: 0.000310122 (0.000482373) 	 BC Loss: 0.000307154 (0.000298148)
-Iteration: [28501 / 50000] 	 Loss: 0.001300987 (0.001541014) 	 Physics Loss: 0.000685782 (0.000873007) 	 Data Loss: 0.000372513 (0.000378884) 	 BC Loss: 0.000242692 (0.000289123)
-Iteration: [29001 / 50000] 	 Loss: 0.001302390 (0.001531350) 	 Physics Loss: 0.000764329 (0.000809421) 	 Data Loss: 0.000326938 (0.000448536) 	 BC Loss: 0.000211123 (0.000273393)
-Iteration: [29501 / 50000] 	 Loss: 0.001228882 (0.001639404) 	 Physics Loss: 0.000864926 (0.000892691) 	 Data Loss: 0.000257633 (0.000503808) 	 BC Loss: 0.000106323 (0.000242905)
-Iteration: [30001 / 50000] 	 Loss: 0.001362466 (0.001636085) 	 Physics Loss: 0.000635906 (0.000994050) 	 Data Loss: 0.000649345 (0.000412151) 	 BC Loss: 0.000077215 (0.000229884)
-Iteration: [30501 / 50000] 	 Loss: 0.000972152 (0.001479216) 	 Physics Loss: 0.000619220 (0.000883626) 	 Data Loss: 0.000230181 (0.000379169) 	 BC Loss: 0.000122752 (0.000216420)
-Iteration: [31001 / 50000] 	 Loss: 0.005065940 (0.001380907) 	 Physics Loss: 0.004397592 (0.000831873) 	 Data Loss: 0.000477057 (0.000344165) 	 BC Loss: 0.000191290 (0.000204869)
-Iteration: [31501 / 50000] 	 Loss: 0.001720798 (0.001439294) 	 Physics Loss: 0.001153235 (0.000778960) 	 Data Loss: 0.000366380 (0.000444564) 	 BC Loss: 0.000201183 (0.000215771)
-Iteration: [32001 / 50000] 	 Loss: 0.001272172 (0.001182971) 	 Physics Loss: 0.000547293 (0.000645338) 	 Data Loss: 0.000612554 (0.000330688) 	 BC Loss: 0.000112325 (0.000206944)
-Iteration: [32501 / 50000] 	 Loss: 0.001592739 (0.001303701) 	 Physics Loss: 0.001089040 (0.000732894) 	 Data Loss: 0.000368751 (0.000377450) 	 BC Loss: 0.000134948 (0.000193357)
-Iteration: [33001 / 50000] 	 Loss: 0.001257007 (0.001731674) 	 Physics Loss: 0.000964463 (0.001143823) 	 Data Loss: 0.000217459 (0.000395994) 	 BC Loss: 0.000075086 (0.000191858)
-Iteration: [33501 / 50000] 	 Loss: 0.001404967 (0.001307026) 	 Physics Loss: 0.000708074 (0.000685339) 	 Data Loss: 0.000530431 (0.000435325) 	 BC Loss: 0.000166462 (0.000186362)
-Iteration: [34001 / 50000] 	 Loss: 0.000798481 (0.001060399) 	 Physics Loss: 0.000366831 (0.000566683) 	 Data Loss: 0.000218087 (0.000317301) 	 BC Loss: 0.000213564 (0.000176415)
-Iteration: [34501 / 50000] 	 Loss: 0.001798573 (0.001364744) 	 Physics Loss: 0.001362987 (0.000847362) 	 Data Loss: 0.000308547 (0.000354412) 	 BC Loss: 0.000127040 (0.000162970)
-Iteration: [35001 / 50000] 	 Loss: 0.000781053 (0.001014936) 	 Physics Loss: 0.000422994 (0.000539345) 	 Data Loss: 0.000270824 (0.000311768) 	 BC Loss: 0.000087234 (0.000163823)
-Iteration: [35501 / 50000] 	 Loss: 0.001209691 (0.001329915) 	 Physics Loss: 0.000784697 (0.000694873) 	 Data Loss: 0.000264404 (0.000425117) 	 BC Loss: 0.000160591 (0.000209925)
-Iteration: [36001 / 50000] 	 Loss: 0.001398915 (0.001136703) 	 Physics Loss: 0.000490567 (0.000583094) 	 Data Loss: 0.000744279 (0.000401171) 	 BC Loss: 0.000164069 (0.000152438)
-Iteration: [36501 / 50000] 	 Loss: 0.000835373 (0.001367094) 	 Physics Loss: 0.000595709 (0.000792289) 	 Data Loss: 0.000132676 (0.000426714) 	 BC Loss: 0.000106987 (0.000148092)
-Iteration: [37001 / 50000] 	 Loss: 0.000952036 (0.001044580) 	 Physics Loss: 0.000709375 (0.000580873) 	 Data Loss: 0.000161545 (0.000334467) 	 BC Loss: 0.000081115 (0.000129240)
-Iteration: [37501 / 50000] 	 Loss: 0.000579479 (0.001085730) 	 Physics Loss: 0.000270049 (0.000599332) 	 Data Loss: 0.000242183 (0.000348117) 	 BC Loss: 0.000067247 (0.000138281)
-Iteration: [38001 / 50000] 	 Loss: 0.001053359 (0.001110448) 	 Physics Loss: 0.000445312 (0.000595116) 	 Data Loss: 0.000482833 (0.000353724) 	 BC Loss: 0.000125214 (0.000161607)
-Iteration: [38501 / 50000] 	 Loss: 0.000667115 (0.001049175) 	 Physics Loss: 0.000326662 (0.000543516) 	 Data Loss: 0.000173648 (0.000387762) 	 BC Loss: 0.000166804 (0.000117897)
-Iteration: [39001 / 50000] 	 Loss: 0.000687853 (0.001085206) 	 Physics Loss: 0.000313576 (0.000590442) 	 Data Loss: 0.000321921 (0.000367264) 	 BC Loss: 0.000052356 (0.000127500)
-Iteration: [39501 / 50000] 	 Loss: 0.000673423 (0.001160439) 	 Physics Loss: 0.000421422 (0.000714949) 	 Data Loss: 0.000192454 (0.000300986) 	 BC Loss: 0.000059547 (0.000144504)
-Iteration: [40001 / 50000] 	 Loss: 0.000938448 (0.001243982) 	 Physics Loss: 0.000547043 (0.000710499) 	 Data Loss: 0.000287677 (0.000374305) 	 BC Loss: 0.000103729 (0.000159178)
-Iteration: [40501 / 50000] 	 Loss: 0.001798752 (0.001093546) 	 Physics Loss: 0.000866074 (0.000650020) 	 Data Loss: 0.000780730 (0.000305183) 	 BC Loss: 0.000151947 (0.000138344)
-Iteration: [41001 / 50000] 	 Loss: 0.001101400 (0.001448896) 	 Physics Loss: 0.000600817 (0.000901216) 	 Data Loss: 0.000405329 (0.000389668) 	 BC Loss: 0.000095254 (0.000158011)
-Iteration: [41501 / 50000] 	 Loss: 0.000776893 (0.000824789) 	 Physics Loss: 0.000455096 (0.000410463) 	 Data Loss: 0.000206055 (0.000317277) 	 BC Loss: 0.000115742 (0.000097049)
-Iteration: [42001 / 50000] 	 Loss: 0.001060384 (0.001240770) 	 Physics Loss: 0.000583868 (0.000778712) 	 Data Loss: 0.000378937 (0.000335865) 	 BC Loss: 0.000097578 (0.000126193)
-Iteration: [42501 / 50000] 	 Loss: 0.000691815 (0.000944194) 	 Physics Loss: 0.000381213 (0.000516831) 	 Data Loss: 0.000162085 (0.000307154) 	 BC Loss: 0.000148517 (0.000120209)
-Iteration: [43001 / 50000] 	 Loss: 0.000545245 (0.000868335) 	 Physics Loss: 0.000363892 (0.000449657) 	 Data Loss: 0.000124517 (0.000324281) 	 BC Loss: 0.000056836 (0.000094398)
-Iteration: [43501 / 50000] 	 Loss: 0.001068081 (0.001141232) 	 Physics Loss: 0.000613936 (0.000622923) 	 Data Loss: 0.000369106 (0.000392804) 	 BC Loss: 0.000085040 (0.000125504)
-Iteration: [44001 / 50000] 	 Loss: 0.001168622 (0.000963809) 	 Physics Loss: 0.000486189 (0.000540564) 	 Data Loss: 0.000549527 (0.000306839) 	 BC Loss: 0.000132906 (0.000116407)
-Iteration: [44501 / 50000] 	 Loss: 0.000939374 (0.000756769) 	 Physics Loss: 0.000379668 (0.000386392) 	 Data Loss: 0.000454265 (0.000277635) 	 BC Loss: 0.000105440 (0.000092742)
-Iteration: [45001 / 50000] 	 Loss: 0.001202180 (0.000892342) 	 Physics Loss: 0.000726597 (0.000506370) 	 Data Loss: 0.000367492 (0.000280093) 	 BC Loss: 0.000108091 (0.000105879)
-Iteration: [45501 / 50000] 	 Loss: 0.000887047 (0.000908168) 	 Physics Loss: 0.000641108 (0.000560744) 	 Data Loss: 0.000148591 (0.000263541) 	 BC Loss: 0.000097348 (0.000083883)
-Iteration: [46001 / 50000] 	 Loss: 0.000525872 (0.000734937) 	 Physics Loss: 0.000322635 (0.000362759) 	 Data Loss: 0.000158824 (0.000288360) 	 BC Loss: 0.000044412 (0.000083818)
-Iteration: [46501 / 50000] 	 Loss: 0.000545968 (0.000907963) 	 Physics Loss: 0.000350560 (0.000489585) 	 Data Loss: 0.000136768 (0.000312205) 	 BC Loss: 0.000058640 (0.000106173)
-Iteration: [47001 / 50000] 	 Loss: 0.000826687 (0.000924496) 	 Physics Loss: 0.000575209 (0.000511645) 	 Data Loss: 0.000205284 (0.000314716) 	 BC Loss: 0.000046194 (0.000098134)
-Iteration: [47501 / 50000] 	 Loss: 0.000620959 (0.001185708) 	 Physics Loss: 0.000382126 (0.000788119) 	 Data Loss: 0.000148993 (0.000271556) 	 BC Loss: 0.000089840 (0.000126034)
-Iteration: [48001 / 50000] 	 Loss: 0.000664698 (0.000798627) 	 Physics Loss: 0.000412207 (0.000461797) 	 Data Loss: 0.000182339 (0.000250679) 	 BC Loss: 0.000070152 (0.000086152)
-Iteration: [48501 / 50000] 	 Loss: 0.000826713 (0.000925605) 	 Physics Loss: 0.000485339 (0.000512687) 	 Data Loss: 0.000170909 (0.000314013) 	 BC Loss: 0.000170466 (0.000098905)
-Iteration: [49001 / 50000] 	 Loss: 0.000559556 (0.000846881) 	 Physics Loss: 0.000350503 (0.000421408) 	 Data Loss: 0.000165743 (0.000346362) 	 BC Loss: 0.000043311 (0.000079111)
-Iteration: [49501 / 50000] 	 Loss: 0.000652327 (0.000740137) 	 Physics Loss: 0.000301691 (0.000376061) 	 Data Loss: 0.000195276 (0.000275985) 	 BC Loss: 0.000155360 (0.000088090)
    Visualizing the Results ​
    julia
    ts, xs, ys = 0.0f0:0.05f0:2.0f0, 0.0f0:0.02f0:2.0f0, 0.0f0:0.02f0:2.0f0
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162595 (1.982162595) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505020 (0.598505020)
+Iteration: [  501 / 50000] 	 Loss: 0.040918142 (0.025583766) 	 Physics Loss: 0.000391871 (0.000269293) 	 Data Loss: 0.014243502 (0.009196415) 	 BC Loss: 0.026282769 (0.016118057)
+Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281416) 	 Physics Loss: 0.000071671 (0.000163183) 	 Data Loss: 0.007905646 (0.010876307) 	 BC Loss: 0.007363342 (0.014241925)
+Iteration: [ 1501 / 50000] 	 Loss: 0.019566942 (0.026170360) 	 Physics Loss: 0.001279285 (0.001009776) 	 Data Loss: 0.003071634 (0.010452234) 	 BC Loss: 0.015216023 (0.014708348)
+Iteration: [ 2001 / 50000] 	 Loss: 0.035557814 (0.027273539) 	 Physics Loss: 0.004062099 (0.001872000) 	 Data Loss: 0.013011657 (0.010586375) 	 BC Loss: 0.018484056 (0.014815168)
+Iteration: [ 2501 / 50000] 	 Loss: 0.011523969 (0.022175739) 	 Physics Loss: 0.002366644 (0.001909967) 	 Data Loss: 0.005568597 (0.007926281) 	 BC Loss: 0.003588729 (0.012339489)
+Iteration: [ 3001 / 50000] 	 Loss: 0.026762877 (0.031860039) 	 Physics Loss: 0.003192307 (0.006136029) 	 Data Loss: 0.009067146 (0.010883353) 	 BC Loss: 0.014503423 (0.014840655)
+Iteration: [ 3501 / 50000] 	 Loss: 0.025398642 (0.045591548) 	 Physics Loss: 0.003200589 (0.001476229) 	 Data Loss: 0.007934441 (0.019488331) 	 BC Loss: 0.014263613 (0.024626993)
+Iteration: [ 4001 / 50000] 	 Loss: 0.024822138 (0.029854555) 	 Physics Loss: 0.001797515 (0.003666145) 	 Data Loss: 0.012387147 (0.011416912) 	 BC Loss: 0.010637476 (0.014771502)
+Iteration: [ 4501 / 50000] 	 Loss: 0.029747024 (0.031501673) 	 Physics Loss: 0.005927891 (0.004604227) 	 Data Loss: 0.018316794 (0.011011195) 	 BC Loss: 0.005502339 (0.015886255)
+Iteration: [ 5001 / 50000] 	 Loss: 0.025349757 (0.043140102) 	 Physics Loss: 0.000728280 (0.002826607) 	 Data Loss: 0.003646539 (0.016781844) 	 BC Loss: 0.020974938 (0.023531647)
+Iteration: [ 5501 / 50000] 	 Loss: 0.026193798 (0.022408014) 	 Physics Loss: 0.000325338 (0.000346806) 	 Data Loss: 0.004838886 (0.008093885) 	 BC Loss: 0.021029573 (0.013967324)
+Iteration: [ 6001 / 50000] 	 Loss: 0.021179065 (0.022343241) 	 Physics Loss: 0.000527887 (0.000702699) 	 Data Loss: 0.007729640 (0.007971080) 	 BC Loss: 0.012921538 (0.013669461)
+Iteration: [ 6501 / 50000] 	 Loss: 0.030424416 (0.021639718) 	 Physics Loss: 0.000403142 (0.000825630) 	 Data Loss: 0.013241904 (0.009147485) 	 BC Loss: 0.016779371 (0.011666608)
+Iteration: [ 7001 / 50000] 	 Loss: 0.012885604 (0.022515845) 	 Physics Loss: 0.000715418 (0.000757549) 	 Data Loss: 0.006018532 (0.009810397) 	 BC Loss: 0.006151655 (0.011947901)
+Iteration: [ 7501 / 50000] 	 Loss: 0.021707822 (0.020194370) 	 Physics Loss: 0.000589236 (0.000854950) 	 Data Loss: 0.011797804 (0.008223927) 	 BC Loss: 0.009320782 (0.011115490)
+Iteration: [ 8001 / 50000] 	 Loss: 0.021062549 (0.019913949) 	 Physics Loss: 0.000528065 (0.000852256) 	 Data Loss: 0.008882683 (0.007675199) 	 BC Loss: 0.011651803 (0.011386497)
+Iteration: [ 8501 / 50000] 	 Loss: 0.022159226 (0.022652669) 	 Physics Loss: 0.001293960 (0.001621564) 	 Data Loss: 0.009795113 (0.008712809) 	 BC Loss: 0.011070153 (0.012318298)
+Iteration: [ 9001 / 50000] 	 Loss: 0.022167893 (0.021835698) 	 Physics Loss: 0.001893832 (0.001330249) 	 Data Loss: 0.006999766 (0.008149015) 	 BC Loss: 0.013274295 (0.012356437)
+Iteration: [ 9501 / 50000] 	 Loss: 0.016295999 (0.020470668) 	 Physics Loss: 0.002220259 (0.000820911) 	 Data Loss: 0.006721330 (0.008737788) 	 BC Loss: 0.007354409 (0.010911969)
+Iteration: [10001 / 50000] 	 Loss: 0.020919006 (0.022420531) 	 Physics Loss: 0.001985265 (0.000919413) 	 Data Loss: 0.003050850 (0.008777075) 	 BC Loss: 0.015882893 (0.012724041)
+Iteration: [10501 / 50000] 	 Loss: 0.015559800 (0.017435743) 	 Physics Loss: 0.000461346 (0.000464835) 	 Data Loss: 0.005280917 (0.006347454) 	 BC Loss: 0.009817538 (0.010623454)
+Iteration: [11001 / 50000] 	 Loss: 0.022515260 (0.018052436) 	 Physics Loss: 0.000395792 (0.000500864) 	 Data Loss: 0.004308904 (0.006823588) 	 BC Loss: 0.017810564 (0.010727987)
+Iteration: [11501 / 50000] 	 Loss: 0.021128871 (0.017566014) 	 Physics Loss: 0.000394875 (0.000523516) 	 Data Loss: 0.005251979 (0.006506131) 	 BC Loss: 0.015482016 (0.010536365)
+Iteration: [12001 / 50000] 	 Loss: 0.030411797 (0.018228574) 	 Physics Loss: 0.000672969 (0.000554123) 	 Data Loss: 0.019357491 (0.007100587) 	 BC Loss: 0.010381337 (0.010573865)
+Iteration: [12501 / 50000] 	 Loss: 0.018811548 (0.018401809) 	 Physics Loss: 0.000697236 (0.000691837) 	 Data Loss: 0.002697282 (0.006085245) 	 BC Loss: 0.015417031 (0.011624727)
+Iteration: [13001 / 50000] 	 Loss: 0.011868341 (0.017906483) 	 Physics Loss: 0.000776913 (0.000821749) 	 Data Loss: 0.003124564 (0.006143386) 	 BC Loss: 0.007966863 (0.010941349)
+Iteration: [13501 / 50000] 	 Loss: 0.014784457 (0.018100940) 	 Physics Loss: 0.000598925 (0.000848341) 	 Data Loss: 0.006280850 (0.006679880) 	 BC Loss: 0.007904682 (0.010572717)
+Iteration: [14001 / 50000] 	 Loss: 0.015592407 (0.016142424) 	 Physics Loss: 0.000597963 (0.000956681) 	 Data Loss: 0.006269322 (0.005398072) 	 BC Loss: 0.008725122 (0.009787671)
+Iteration: [14501 / 50000] 	 Loss: 0.025311429 (0.015725624) 	 Physics Loss: 0.001073368 (0.001058255) 	 Data Loss: 0.009163789 (0.005325366) 	 BC Loss: 0.015074271 (0.009342002)
+Iteration: [15001 / 50000] 	 Loss: 0.016914256 (0.015927596) 	 Physics Loss: 0.000966280 (0.001092387) 	 Data Loss: 0.005907979 (0.004847473) 	 BC Loss: 0.010039996 (0.009987737)
+Iteration: [15501 / 50000] 	 Loss: 0.010531792 (0.017309623) 	 Physics Loss: 0.001679290 (0.001229975) 	 Data Loss: 0.002361344 (0.005612177) 	 BC Loss: 0.006491158 (0.010467471)
+Iteration: [16001 / 50000] 	 Loss: 0.012041669 (0.015774786) 	 Physics Loss: 0.000827484 (0.001132011) 	 Data Loss: 0.001785598 (0.004045264) 	 BC Loss: 0.009428588 (0.010597511)
+Iteration: [16501 / 50000] 	 Loss: 0.017424010 (0.015305851) 	 Physics Loss: 0.001297660 (0.001166611) 	 Data Loss: 0.004497696 (0.004839269) 	 BC Loss: 0.011628653 (0.009299970)
+Iteration: [17001 / 50000] 	 Loss: 0.012141772 (0.013891050) 	 Physics Loss: 0.001339099 (0.001353682) 	 Data Loss: 0.001933840 (0.003089970) 	 BC Loss: 0.008868832 (0.009447398)
+Iteration: [17501 / 50000] 	 Loss: 0.015801162 (0.014529660) 	 Physics Loss: 0.001557242 (0.001193460) 	 Data Loss: 0.004102683 (0.004099627) 	 BC Loss: 0.010141237 (0.009236576)
+Iteration: [18001 / 50000] 	 Loss: 0.011515658 (0.013194922) 	 Physics Loss: 0.001411696 (0.001229187) 	 Data Loss: 0.003250526 (0.003880639) 	 BC Loss: 0.006853437 (0.008085096)
+Iteration: [18501 / 50000] 	 Loss: 0.014405347 (0.014845728) 	 Physics Loss: 0.000946846 (0.001311332) 	 Data Loss: 0.005034053 (0.004776356) 	 BC Loss: 0.008424449 (0.008758042)
+Iteration: [19001 / 50000] 	 Loss: 0.015056589 (0.013655247) 	 Physics Loss: 0.001184456 (0.001292421) 	 Data Loss: 0.003681621 (0.003807114) 	 BC Loss: 0.010190512 (0.008555712)
+Iteration: [19501 / 50000] 	 Loss: 0.013299890 (0.013724705) 	 Physics Loss: 0.001885105 (0.001415296) 	 Data Loss: 0.005226072 (0.003555858) 	 BC Loss: 0.006188713 (0.008753551)
+Iteration: [20001 / 50000] 	 Loss: 0.017522790 (0.012518023) 	 Physics Loss: 0.001983398 (0.001058186) 	 Data Loss: 0.009327068 (0.002995419) 	 BC Loss: 0.006212323 (0.008464416)
+Iteration: [20501 / 50000] 	 Loss: 0.021622123 (0.012733568) 	 Physics Loss: 0.001156326 (0.001213073) 	 Data Loss: 0.002187656 (0.003051734) 	 BC Loss: 0.018278142 (0.008468761)
+Iteration: [21001 / 50000] 	 Loss: 0.012820827 (0.013354947) 	 Physics Loss: 0.001192615 (0.001322284) 	 Data Loss: 0.000899577 (0.003003312) 	 BC Loss: 0.010728635 (0.009029351)
+Iteration: [21501 / 50000] 	 Loss: 0.016243555 (0.012213146) 	 Physics Loss: 0.002533023 (0.001458879) 	 Data Loss: 0.008658717 (0.003036225) 	 BC Loss: 0.005051813 (0.007718043)
+Iteration: [22001 / 50000] 	 Loss: 0.009802144 (0.011197914) 	 Physics Loss: 0.000975115 (0.001180587) 	 Data Loss: 0.002004704 (0.002875937) 	 BC Loss: 0.006822325 (0.007141389)
+Iteration: [22501 / 50000] 	 Loss: 0.010460960 (0.011299610) 	 Physics Loss: 0.001939239 (0.001350272) 	 Data Loss: 0.003044792 (0.002758041) 	 BC Loss: 0.005476930 (0.007191297)
+Iteration: [23001 / 50000] 	 Loss: 0.007303775 (0.010617163) 	 Physics Loss: 0.001568149 (0.001255589) 	 Data Loss: 0.002208042 (0.002901576) 	 BC Loss: 0.003527584 (0.006459999)
+Iteration: [23501 / 50000] 	 Loss: 0.013054269 (0.012929341) 	 Physics Loss: 0.001233365 (0.001229342) 	 Data Loss: 0.002502190 (0.002853266) 	 BC Loss: 0.009318714 (0.008846736)
+Iteration: [24001 / 50000] 	 Loss: 0.015850905 (0.011500452) 	 Physics Loss: 0.001933446 (0.001243522) 	 Data Loss: 0.003105919 (0.002514284) 	 BC Loss: 0.010811541 (0.007742645)
+Iteration: [24501 / 50000] 	 Loss: 0.014612451 (0.011235875) 	 Physics Loss: 0.000830096 (0.001126977) 	 Data Loss: 0.001665982 (0.002516578) 	 BC Loss: 0.012116373 (0.007592318)
+Iteration: [25001 / 50000] 	 Loss: 0.012446284 (0.012136834) 	 Physics Loss: 0.001868680 (0.001571323) 	 Data Loss: 0.002885855 (0.002738171) 	 BC Loss: 0.007691749 (0.007827340)
+Iteration: [25501 / 50000] 	 Loss: 0.006147437 (0.010866777) 	 Physics Loss: 0.001199473 (0.001432847) 	 Data Loss: 0.000741202 (0.003181246) 	 BC Loss: 0.004206762 (0.006252684)
+Iteration: [26001 / 50000] 	 Loss: 0.009382382 (0.012447415) 	 Physics Loss: 0.002055192 (0.001882480) 	 Data Loss: 0.001872860 (0.002723655) 	 BC Loss: 0.005454330 (0.007841279)
+Iteration: [26501 / 50000] 	 Loss: 0.005613507 (0.009200627) 	 Physics Loss: 0.001204579 (0.001562393) 	 Data Loss: 0.001970439 (0.002349454) 	 BC Loss: 0.002438489 (0.005288779)
+Iteration: [27001 / 50000] 	 Loss: 0.008868203 (0.010649174) 	 Physics Loss: 0.001465238 (0.001703513) 	 Data Loss: 0.002235665 (0.002514324) 	 BC Loss: 0.005167301 (0.006431336)
+Iteration: [27501 / 50000] 	 Loss: 0.016564976 (0.011443286) 	 Physics Loss: 0.001060150 (0.001400374) 	 Data Loss: 0.003114783 (0.002061526) 	 BC Loss: 0.012390045 (0.007981384)
+Iteration: [28001 / 50000] 	 Loss: 0.008029928 (0.011309550) 	 Physics Loss: 0.001292446 (0.001715263) 	 Data Loss: 0.001651582 (0.003145006) 	 BC Loss: 0.005085900 (0.006449282)
+Iteration: [28501 / 50000] 	 Loss: 0.009412916 (0.010573968) 	 Physics Loss: 0.002310942 (0.001589268) 	 Data Loss: 0.000935075 (0.002185026) 	 BC Loss: 0.006166900 (0.006799675)
+Iteration: [29001 / 50000] 	 Loss: 0.008956734 (0.010948162) 	 Physics Loss: 0.001730394 (0.001973851) 	 Data Loss: 0.002910380 (0.002606902) 	 BC Loss: 0.004315960 (0.006367408)
+Iteration: [29501 / 50000] 	 Loss: 0.004513775 (0.010220452) 	 Physics Loss: 0.000894471 (0.001687118) 	 Data Loss: 0.001609863 (0.002537781) 	 BC Loss: 0.002009441 (0.005995554)
+Iteration: [30001 / 50000] 	 Loss: 0.004987885 (0.009731424) 	 Physics Loss: 0.001069190 (0.001562893) 	 Data Loss: 0.001401214 (0.002276014) 	 BC Loss: 0.002517481 (0.005892518)
+Iteration: [30501 / 50000] 	 Loss: 0.002595518 (0.008276105) 	 Physics Loss: 0.000917391 (0.001500407) 	 Data Loss: 0.000658372 (0.001976204) 	 BC Loss: 0.001019756 (0.004799493)
+Iteration: [31001 / 50000] 	 Loss: 0.011266185 (0.010293764) 	 Physics Loss: 0.001391491 (0.001473510) 	 Data Loss: 0.003635545 (0.002334290) 	 BC Loss: 0.006239149 (0.006485965)
+Iteration: [31501 / 50000] 	 Loss: 0.013991628 (0.010204735) 	 Physics Loss: 0.002423544 (0.001848977) 	 Data Loss: 0.004265617 (0.002343330) 	 BC Loss: 0.007302467 (0.006012429)
+Iteration: [32001 / 50000] 	 Loss: 0.007164373 (0.009639673) 	 Physics Loss: 0.001379599 (0.001698582) 	 Data Loss: 0.001701293 (0.001939833) 	 BC Loss: 0.004083481 (0.006001258)
+Iteration: [32501 / 50000] 	 Loss: 0.005502027 (0.009115184) 	 Physics Loss: 0.001436674 (0.001596168) 	 Data Loss: 0.001871673 (0.001895187) 	 BC Loss: 0.002193680 (0.005623828)
+Iteration: [33001 / 50000] 	 Loss: 0.005276884 (0.009546057) 	 Physics Loss: 0.002079992 (0.001929016) 	 Data Loss: 0.001436205 (0.001975889) 	 BC Loss: 0.001760687 (0.005641153)
+Iteration: [33501 / 50000] 	 Loss: 0.009125081 (0.008468037) 	 Physics Loss: 0.002191632 (0.001853661) 	 Data Loss: 0.002888216 (0.001945538) 	 BC Loss: 0.004045234 (0.004668837)
+Iteration: [34001 / 50000] 	 Loss: 0.008431979 (0.009071054) 	 Physics Loss: 0.001222603 (0.002027335) 	 Data Loss: 0.001151307 (0.001467187) 	 BC Loss: 0.006058069 (0.005576532)
+Iteration: [34501 / 50000] 	 Loss: 0.009580643 (0.008130345) 	 Physics Loss: 0.002242853 (0.001757617) 	 Data Loss: 0.005897780 (0.002249273) 	 BC Loss: 0.001440010 (0.004123454)
+Iteration: [35001 / 50000] 	 Loss: 0.005147196 (0.007530828) 	 Physics Loss: 0.001256183 (0.001518356) 	 Data Loss: 0.001415687 (0.001431392) 	 BC Loss: 0.002475326 (0.004581080)
+Iteration: [35501 / 50000] 	 Loss: 0.006185614 (0.008019534) 	 Physics Loss: 0.001828601 (0.001882976) 	 Data Loss: 0.001599119 (0.001704919) 	 BC Loss: 0.002757893 (0.004431641)
+Iteration: [36001 / 50000] 	 Loss: 0.009109061 (0.007414200) 	 Physics Loss: 0.001948264 (0.001928137) 	 Data Loss: 0.001738493 (0.001558672) 	 BC Loss: 0.005422304 (0.003927392)
+Iteration: [36501 / 50000] 	 Loss: 0.007278774 (0.007622246) 	 Physics Loss: 0.002025729 (0.001955235) 	 Data Loss: 0.002425219 (0.001680354) 	 BC Loss: 0.002827825 (0.003986657)
+Iteration: [37001 / 50000] 	 Loss: 0.003918901 (0.005813450) 	 Physics Loss: 0.000957004 (0.001656469) 	 Data Loss: 0.001278843 (0.001385545) 	 BC Loss: 0.001683053 (0.002771435)
+Iteration: [37501 / 50000] 	 Loss: 0.004814619 (0.006178515) 	 Physics Loss: 0.001686519 (0.001814278) 	 Data Loss: 0.001063806 (0.001266953) 	 BC Loss: 0.002064294 (0.003097285)
+Iteration: [38001 / 50000] 	 Loss: 0.004643182 (0.004791833) 	 Physics Loss: 0.001702445 (0.001350533) 	 Data Loss: 0.000782973 (0.001124672) 	 BC Loss: 0.002157764 (0.002316626)
+Iteration: [38501 / 50000] 	 Loss: 0.003810477 (0.004910624) 	 Physics Loss: 0.001390965 (0.001866840) 	 Data Loss: 0.000597971 (0.000985127) 	 BC Loss: 0.001821541 (0.002058655)
+Iteration: [39001 / 50000] 	 Loss: 0.003878577 (0.005460986) 	 Physics Loss: 0.002414002 (0.002075948) 	 Data Loss: 0.000412366 (0.001095256) 	 BC Loss: 0.001052209 (0.002289784)
+Iteration: [39501 / 50000] 	 Loss: 0.004065084 (0.004690929) 	 Physics Loss: 0.002393598 (0.001760620) 	 Data Loss: 0.000800843 (0.000796935) 	 BC Loss: 0.000870643 (0.002133374)
+Iteration: [40001 / 50000] 	 Loss: 0.004619529 (0.004839203) 	 Physics Loss: 0.001658047 (0.001965609) 	 Data Loss: 0.001588614 (0.000951834) 	 BC Loss: 0.001372868 (0.001921760)
+Iteration: [40501 / 50000] 	 Loss: 0.004326439 (0.004265270) 	 Physics Loss: 0.001820831 (0.001626283) 	 Data Loss: 0.001034181 (0.000920903) 	 BC Loss: 0.001471427 (0.001718084)
+Iteration: [41001 / 50000] 	 Loss: 0.002906828 (0.003618096) 	 Physics Loss: 0.001272737 (0.001407278) 	 Data Loss: 0.000809125 (0.000837591) 	 BC Loss: 0.000824967 (0.001373227)
+Iteration: [41501 / 50000] 	 Loss: 0.004206825 (0.003857500) 	 Physics Loss: 0.001842899 (0.001706076) 	 Data Loss: 0.000595867 (0.000833410) 	 BC Loss: 0.001768059 (0.001318014)
+Iteration: [42001 / 50000] 	 Loss: 0.003873884 (0.003361913) 	 Physics Loss: 0.002234953 (0.001453047) 	 Data Loss: 0.000641468 (0.000736817) 	 BC Loss: 0.000997462 (0.001172049)
+Iteration: [42501 / 50000] 	 Loss: 0.003796777 (0.003109351) 	 Physics Loss: 0.001149042 (0.001176956) 	 Data Loss: 0.000481490 (0.000653678) 	 BC Loss: 0.002166245 (0.001278718)
+Iteration: [43001 / 50000] 	 Loss: 0.001652764 (0.003025806) 	 Physics Loss: 0.000779939 (0.001280978) 	 Data Loss: 0.000271947 (0.000640416) 	 BC Loss: 0.000600878 (0.001104411)
+Iteration: [43501 / 50000] 	 Loss: 0.002625272 (0.003282005) 	 Physics Loss: 0.001812232 (0.001467459) 	 Data Loss: 0.000510654 (0.000728400) 	 BC Loss: 0.000302386 (0.001086145)
+Iteration: [44001 / 50000] 	 Loss: 0.002716297 (0.002971325) 	 Physics Loss: 0.001654499 (0.001524955) 	 Data Loss: 0.000789657 (0.000616715) 	 BC Loss: 0.000272142 (0.000829655)
+Iteration: [44501 / 50000] 	 Loss: 0.003345200 (0.002695851) 	 Physics Loss: 0.001435974 (0.001263972) 	 Data Loss: 0.001142784 (0.000586666) 	 BC Loss: 0.000766442 (0.000845213)
+Iteration: [45001 / 50000] 	 Loss: 0.002160525 (0.002570425) 	 Physics Loss: 0.001014665 (0.001206274) 	 Data Loss: 0.000531116 (0.000582542) 	 BC Loss: 0.000614744 (0.000781609)
+Iteration: [45501 / 50000] 	 Loss: 0.002704483 (0.002375467) 	 Physics Loss: 0.001256422 (0.001126713) 	 Data Loss: 0.000432294 (0.000537340) 	 BC Loss: 0.001015767 (0.000711415)
+Iteration: [46001 / 50000] 	 Loss: 0.002083311 (0.002229830) 	 Physics Loss: 0.001028772 (0.001128200) 	 Data Loss: 0.000365198 (0.000481077) 	 BC Loss: 0.000689341 (0.000620552)
+Iteration: [46501 / 50000] 	 Loss: 0.001869006 (0.002387334) 	 Physics Loss: 0.001049295 (0.001213058) 	 Data Loss: 0.000542785 (0.000525414) 	 BC Loss: 0.000276926 (0.000648862)
+Iteration: [47001 / 50000] 	 Loss: 0.002161729 (0.002562924) 	 Physics Loss: 0.001238901 (0.001363050) 	 Data Loss: 0.000660004 (0.000578991) 	 BC Loss: 0.000262824 (0.000620883)
+Iteration: [47501 / 50000] 	 Loss: 0.003218761 (0.002436990) 	 Physics Loss: 0.001678471 (0.001291234) 	 Data Loss: 0.000470345 (0.000499937) 	 BC Loss: 0.001069946 (0.000645818)
+Iteration: [48001 / 50000] 	 Loss: 0.001741661 (0.001888746) 	 Physics Loss: 0.000994589 (0.000895934) 	 Data Loss: 0.000492043 (0.000409009) 	 BC Loss: 0.000255029 (0.000583803)
+Iteration: [48501 / 50000] 	 Loss: 0.001460486 (0.002372291) 	 Physics Loss: 0.000619964 (0.001287611) 	 Data Loss: 0.000285411 (0.000508584) 	 BC Loss: 0.000555111 (0.000576096)
+Iteration: [49001 / 50000] 	 Loss: 0.001497735 (0.002357919) 	 Physics Loss: 0.000912286 (0.001282274) 	 Data Loss: 0.000287049 (0.000532408) 	 BC Loss: 0.000298400 (0.000543235)
+Iteration: [49501 / 50000] 	 Loss: 0.003226456 (0.001985326) 	 Physics Loss: 0.001385620 (0.000979689) 	 Data Loss: 0.000554629 (0.000451327) 	 BC Loss: 0.001286207 (0.000554309)
    Visualizing the Results ​
    julia
    ts, xs, ys = 0.0f0:0.05f0:2.0f0, 0.0f0:0.02f0:2.0f0, 0.0f0:0.02f0:2.0f0
 grid = stack([[elem...] for elem in vec(collect(Iterators.product(xs, ys, ts)))])
 
 u_real = reshape(analytical_solution(grid), length(xs), length(ys), length(ts))
@@ -280,7 +280,7 @@ import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1
     end
 
     fig
-end
    Appendix ​
    julia
    using InteractiveUtils
+end
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -339,5 +339,6 @@ import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1
 Environment:
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
-1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,12))])}const B=h(k,[["render",o]]);export{m as __pageData,B as default};
+2 devices:
+  0: Quadro RTX 5000 (sm_75, 15.258 GiB / 16.000 GiB available)
+  1: Quadro RTX 5000 (sm_75, 15.549 GiB / 16.000 GiB available)

    This page was generated using Literate.jl.

    `,12))])}const B=p(k,[["render",o]]);export{m as __pageData,B as default}; diff --git a/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.C3tygY0O.lean.js b/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.DSWj7UgP.lean.js similarity index 84% rename from dev/assets/tutorials_intermediate_4_PINN2DPDE.md.C3tygY0O.lean.js rename to dev/assets/tutorials_intermediate_4_PINN2DPDE.md.DSWj7UgP.lean.js index 9732c2c1f8..417875e8a4 100644 --- a/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.C3tygY0O.lean.js +++ b/dev/assets/tutorials_intermediate_4_PINN2DPDE.md.DSWj7UgP.lean.js @@ -1,4 +1,4 @@ -import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1F9.js";const p="/dev/assets/pinn_nested_ad.B__JnolW.gif",m=JSON.parse('{"title":"Training a PINN on 2D PDE","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/intermediate/4_PINN2DPDE.md","filePath":"tutorials/intermediate/4_PINN2DPDE.md","lastUpdated":null}'),k={name:"tutorials/intermediate/4_PINN2DPDE.md"},e={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.586ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3795.2 1000","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.401ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3713.2 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.109ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3584.2 1000","aria-hidden":"true"};function o(c,i,F,C,D,L){return l(),n("div",null,[i[10]||(i[10]=a(`

    Training a PINN on 2D PDE ​

    In this tutorial we will go over using a PINN to solve 2D PDEs. We will be using the system from NeuralPDE Tutorials. However, we will be using our custom loss function and use nested AD capabilities of Lux.jl.

    This is a demonstration of Lux.jl. For serious usecases of PINNs, please refer to the package: NeuralPDE.jl.

    Package Imports ​

    julia
    using ADTypes, Lux, Optimisers, Zygote, Random, Printf, Statistics, MLUtils, OnlineStats,
+import{_ as p,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1F9.js";const h="/dev/assets/pinn_nested_ad.Bwdus6h-.gif",m=JSON.parse('{"title":"Training a PINN on 2D PDE","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/intermediate/4_PINN2DPDE.md","filePath":"tutorials/intermediate/4_PINN2DPDE.md","lastUpdated":null}'),k={name:"tutorials/intermediate/4_PINN2DPDE.md"},e={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.586ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3795.2 1000","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.401ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3713.2 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},y={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.109ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3584.2 1000","aria-hidden":"true"};function o(c,i,F,C,L,D){return l(),n("div",null,[i[10]||(i[10]=a(`
    Training a PINN on 2D PDE ​
    In this tutorial we will go over using a PINN to solve 2D PDEs. We will be using the system from NeuralPDE Tutorials. However, we will be using our custom loss function and use nested AD capabilities of Lux.jl.
    This is a demonstration of Lux.jl. For serious usecases of PINNs, please refer to the package: NeuralPDE.jl.
    Package Imports ​
    julia
    using ADTypes, Lux, Optimisers, Zygote, Random, Printf, Statistics, MLUtils, OnlineStats,
       CairoMakie
 using LuxCUDA
 
@@ -147,116 +147,116 @@ import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1
 trained_u = Lux.testmode(StatefulLuxLayer{true}(
     trained_model.model.u, trained_model.ps.u, trained_model.st.u))
    ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
-Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162476 (1.982162476) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505080 (0.598505080)
-Iteration: [  501 / 50000] 	 Loss: 0.040918160 (0.025583776) 	 Physics Loss: 0.000391877 (0.000269295) 	 Data Loss: 0.014243508 (0.009196416) 	 BC Loss: 0.026282774 (0.016118063)
-Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281426) 	 Physics Loss: 0.000071670 (0.000163182) 	 Data Loss: 0.007905648 (0.010876314) 	 BC Loss: 0.007363341 (0.014241929)
-Iteration: [ 1501 / 50000] 	 Loss: 0.019567011 (0.026170366) 	 Physics Loss: 0.001279382 (0.001009728) 	 Data Loss: 0.003071612 (0.010452257) 	 BC Loss: 0.015216017 (0.014708381)
-Iteration: [ 2001 / 50000] 	 Loss: 0.035556547 (0.027273500) 	 Physics Loss: 0.004061943 (0.001871931) 	 Data Loss: 0.013011228 (0.010586374) 	 BC Loss: 0.018483378 (0.014815190)
-Iteration: [ 2501 / 50000] 	 Loss: 0.011505678 (0.022150228) 	 Physics Loss: 0.002304791 (0.001940841) 	 Data Loss: 0.005615299 (0.007885863) 	 BC Loss: 0.003585588 (0.012323526)
-Iteration: [ 3001 / 50000] 	 Loss: 0.031768262 (0.029164422) 	 Physics Loss: 0.008404830 (0.003920683) 	 Data Loss: 0.008808360 (0.010936455) 	 BC Loss: 0.014555071 (0.014307282)
-Iteration: [ 3501 / 50000] 	 Loss: 0.017645847 (0.042499166) 	 Physics Loss: 0.001848093 (0.001690981) 	 Data Loss: 0.005461216 (0.018073166) 	 BC Loss: 0.010336538 (0.022735020)
-Iteration: [ 4001 / 50000] 	 Loss: 0.028128654 (0.027620461) 	 Physics Loss: 0.005112350 (0.002448601) 	 Data Loss: 0.013499700 (0.011073254) 	 BC Loss: 0.009516605 (0.014098606)
-Iteration: [ 4501 / 50000] 	 Loss: 0.014340003 (0.033320315) 	 Physics Loss: 0.001292084 (0.004329988) 	 Data Loss: 0.008556721 (0.012002973) 	 BC Loss: 0.004491198 (0.016987354)
-Iteration: [ 5001 / 50000] 	 Loss: 0.030331207 (0.041541956) 	 Physics Loss: 0.000723805 (0.002695386) 	 Data Loss: 0.004466736 (0.016228525) 	 BC Loss: 0.025140665 (0.022618050)
-Iteration: [ 5501 / 50000] 	 Loss: 0.022293953 (0.021166507) 	 Physics Loss: 0.000554349 (0.000707158) 	 Data Loss: 0.004542338 (0.007950513) 	 BC Loss: 0.017197266 (0.012508835)
-Iteration: [ 6001 / 50000] 	 Loss: 0.018723227 (0.020389127) 	 Physics Loss: 0.000442930 (0.000975201) 	 Data Loss: 0.006906096 (0.007273634) 	 BC Loss: 0.011374202 (0.012140292)
-Iteration: [ 6501 / 50000] 	 Loss: 0.028305896 (0.020493284) 	 Physics Loss: 0.000753467 (0.001849154) 	 Data Loss: 0.011062279 (0.008014920) 	 BC Loss: 0.016490150 (0.010629211)
-Iteration: [ 7001 / 50000] 	 Loss: 0.015494239 (0.020624701) 	 Physics Loss: 0.007333768 (0.001517879) 	 Data Loss: 0.004043682 (0.008185850) 	 BC Loss: 0.004116789 (0.010920972)
-Iteration: [ 7501 / 50000] 	 Loss: 0.019155467 (0.018008159) 	 Physics Loss: 0.002511769 (0.001448493) 	 Data Loss: 0.009514628 (0.006572613) 	 BC Loss: 0.007129070 (0.009987052)
-Iteration: [ 8001 / 50000] 	 Loss: 0.018706074 (0.017215939) 	 Physics Loss: 0.001792779 (0.001894138) 	 Data Loss: 0.005628760 (0.005433898) 	 BC Loss: 0.011284535 (0.009887908)
-Iteration: [ 8501 / 50000] 	 Loss: 0.017605182 (0.019859709) 	 Physics Loss: 0.002696904 (0.003125851) 	 Data Loss: 0.007372384 (0.005978104) 	 BC Loss: 0.007535893 (0.010755754)
-Iteration: [ 9001 / 50000] 	 Loss: 0.017614847 (0.016386209) 	 Physics Loss: 0.003162773 (0.002853032) 	 Data Loss: 0.003553676 (0.004607514) 	 BC Loss: 0.010898398 (0.008925664)
-Iteration: [ 9501 / 50000] 	 Loss: 0.008732248 (0.014530762) 	 Physics Loss: 0.001913586 (0.002123826) 	 Data Loss: 0.003329928 (0.004253434) 	 BC Loss: 0.003488734 (0.008153505)
-Iteration: [10001 / 50000] 	 Loss: 0.017750096 (0.017799046) 	 Physics Loss: 0.003630795 (0.003061282) 	 Data Loss: 0.003031955 (0.005044522) 	 BC Loss: 0.011087346 (0.009693242)
-Iteration: [10501 / 50000] 	 Loss: 0.007509941 (0.011342090) 	 Physics Loss: 0.001571818 (0.001135202) 	 Data Loss: 0.001611185 (0.002613829) 	 BC Loss: 0.004326937 (0.007593059)
-Iteration: [11001 / 50000] 	 Loss: 0.022739200 (0.011915382) 	 Physics Loss: 0.000807529 (0.001103305) 	 Data Loss: 0.002606506 (0.003132161) 	 BC Loss: 0.019325165 (0.007679918)
-Iteration: [11501 / 50000] 	 Loss: 0.019484218 (0.011457845) 	 Physics Loss: 0.001845157 (0.001325290) 	 Data Loss: 0.002358936 (0.002806861) 	 BC Loss: 0.015280124 (0.007325693)
-Iteration: [12001 / 50000] 	 Loss: 0.019427970 (0.011598296) 	 Physics Loss: 0.004434146 (0.001386037) 	 Data Loss: 0.008772802 (0.003136263) 	 BC Loss: 0.006221022 (0.007075997)
-Iteration: [12501 / 50000] 	 Loss: 0.012775686 (0.011906523) 	 Physics Loss: 0.001197650 (0.001402911) 	 Data Loss: 0.001059842 (0.002556076) 	 BC Loss: 0.010518193 (0.007947534)
-Iteration: [13001 / 50000] 	 Loss: 0.006105572 (0.011037273) 	 Physics Loss: 0.000987124 (0.001454655) 	 Data Loss: 0.001305370 (0.002395016) 	 BC Loss: 0.003813077 (0.007187600)
-Iteration: [13501 / 50000] 	 Loss: 0.010004668 (0.011247103) 	 Physics Loss: 0.001224264 (0.001615586) 	 Data Loss: 0.002474443 (0.002668936) 	 BC Loss: 0.006305961 (0.006962581)
-Iteration: [14001 / 50000] 	 Loss: 0.009895653 (0.009313912) 	 Physics Loss: 0.001215764 (0.001694928) 	 Data Loss: 0.002037087 (0.002155375) 	 BC Loss: 0.006642802 (0.005463609)
-Iteration: [14501 / 50000] 	 Loss: 0.014400685 (0.008724037) 	 Physics Loss: 0.001658659 (0.001564218) 	 Data Loss: 0.003658900 (0.002201537) 	 BC Loss: 0.009083126 (0.004958282)
-Iteration: [15001 / 50000] 	 Loss: 0.007676640 (0.008405063) 	 Physics Loss: 0.001566568 (0.001608545) 	 Data Loss: 0.002262217 (0.001837625) 	 BC Loss: 0.003847855 (0.004958895)
-Iteration: [15501 / 50000] 	 Loss: 0.004365115 (0.009256126) 	 Physics Loss: 0.001047856 (0.002247394) 	 Data Loss: 0.000648518 (0.002218451) 	 BC Loss: 0.002668741 (0.004790282)
-Iteration: [16001 / 50000] 	 Loss: 0.004880759 (0.007501942) 	 Physics Loss: 0.001649067 (0.001671217) 	 Data Loss: 0.000296087 (0.001492234) 	 BC Loss: 0.002935605 (0.004338491)
-Iteration: [16501 / 50000] 	 Loss: 0.008074892 (0.007220342) 	 Physics Loss: 0.001825325 (0.001970405) 	 Data Loss: 0.001302087 (0.001730468) 	 BC Loss: 0.004947479 (0.003519468)
-Iteration: [17001 / 50000] 	 Loss: 0.005824474 (0.005835793) 	 Physics Loss: 0.002019164 (0.001728887) 	 Data Loss: 0.000624455 (0.001011056) 	 BC Loss: 0.003180855 (0.003095849)
-Iteration: [17501 / 50000] 	 Loss: 0.006616294 (0.005807751) 	 Physics Loss: 0.002489866 (0.001884541) 	 Data Loss: 0.001154157 (0.001245214) 	 BC Loss: 0.002972272 (0.002677995)
-Iteration: [18001 / 50000] 	 Loss: 0.004335414 (0.005200472) 	 Physics Loss: 0.001764537 (0.002015869) 	 Data Loss: 0.000705982 (0.001058994) 	 BC Loss: 0.001864895 (0.002125610)
-Iteration: [18501 / 50000] 	 Loss: 0.004978007 (0.005351806) 	 Physics Loss: 0.002553018 (0.002129085) 	 Data Loss: 0.000752965 (0.001336156) 	 BC Loss: 0.001672024 (0.001886566)
-Iteration: [19001 / 50000] 	 Loss: 0.004518208 (0.004657542) 	 Physics Loss: 0.001975382 (0.001959185) 	 Data Loss: 0.000239237 (0.000983244) 	 BC Loss: 0.002303589 (0.001715113)
-Iteration: [19501 / 50000] 	 Loss: 0.006942283 (0.004119421) 	 Physics Loss: 0.004590358 (0.001714717) 	 Data Loss: 0.001148389 (0.000894285) 	 BC Loss: 0.001203537 (0.001510418)
-Iteration: [20001 / 50000] 	 Loss: 0.008026988 (0.003463188) 	 Physics Loss: 0.005315070 (0.001485110) 	 Data Loss: 0.001919697 (0.000688489) 	 BC Loss: 0.000792221 (0.001289588)
-Iteration: [20501 / 50000] 	 Loss: 0.004855067 (0.003504427) 	 Physics Loss: 0.001560361 (0.001747428) 	 Data Loss: 0.000460401 (0.000618138) 	 BC Loss: 0.002834306 (0.001138862)
-Iteration: [21001 / 50000] 	 Loss: 0.002417301 (0.003299790) 	 Physics Loss: 0.000924424 (0.001513747) 	 Data Loss: 0.000300984 (0.000659882) 	 BC Loss: 0.001191893 (0.001126162)
-Iteration: [21501 / 50000] 	 Loss: 0.003911218 (0.002670718) 	 Physics Loss: 0.002752088 (0.001218763) 	 Data Loss: 0.000835131 (0.000637475) 	 BC Loss: 0.000323999 (0.000814481)
-Iteration: [22001 / 50000] 	 Loss: 0.002318957 (0.002511334) 	 Physics Loss: 0.001409405 (0.001309370) 	 Data Loss: 0.000344849 (0.000535839) 	 BC Loss: 0.000564703 (0.000666125)
-Iteration: [22501 / 50000] 	 Loss: 0.001994215 (0.002343247) 	 Physics Loss: 0.000746255 (0.001113559) 	 Data Loss: 0.000431564 (0.000577084) 	 BC Loss: 0.000816396 (0.000652603)
-Iteration: [23001 / 50000] 	 Loss: 0.002989073 (0.002301548) 	 Physics Loss: 0.001930058 (0.001243673) 	 Data Loss: 0.000805100 (0.000536315) 	 BC Loss: 0.000253915 (0.000521560)
-Iteration: [23501 / 50000] 	 Loss: 0.002210422 (0.002371583) 	 Physics Loss: 0.001325711 (0.001179150) 	 Data Loss: 0.000160725 (0.000516864) 	 BC Loss: 0.000723986 (0.000675569)
-Iteration: [24001 / 50000] 	 Loss: 0.002023818 (0.002276814) 	 Physics Loss: 0.000857553 (0.001135292) 	 Data Loss: 0.000525994 (0.000580056) 	 BC Loss: 0.000640270 (0.000561465)
-Iteration: [24501 / 50000] 	 Loss: 0.002234935 (0.002236427) 	 Physics Loss: 0.001041825 (0.001241080) 	 Data Loss: 0.000286281 (0.000510494) 	 BC Loss: 0.000906830 (0.000484853)
-Iteration: [25001 / 50000] 	 Loss: 0.001972227 (0.001982885) 	 Physics Loss: 0.000893347 (0.000971555) 	 Data Loss: 0.000666251 (0.000537486) 	 BC Loss: 0.000412629 (0.000473843)
-Iteration: [25501 / 50000] 	 Loss: 0.001659834 (0.002081697) 	 Physics Loss: 0.001085884 (0.001258897) 	 Data Loss: 0.000359855 (0.000474958) 	 BC Loss: 0.000214095 (0.000347842)
-Iteration: [26001 / 50000] 	 Loss: 0.002059042 (0.001770784) 	 Physics Loss: 0.001171167 (0.000890729) 	 Data Loss: 0.000426463 (0.000447986) 	 BC Loss: 0.000461412 (0.000432069)
-Iteration: [26501 / 50000] 	 Loss: 0.002089650 (0.002068657) 	 Physics Loss: 0.001594031 (0.001356683) 	 Data Loss: 0.000330550 (0.000420685) 	 BC Loss: 0.000165069 (0.000291289)
-Iteration: [27001 / 50000] 	 Loss: 0.001346401 (0.001879478) 	 Physics Loss: 0.000798861 (0.001050717) 	 Data Loss: 0.000333685 (0.000522715) 	 BC Loss: 0.000213855 (0.000306047)
-Iteration: [27501 / 50000] 	 Loss: 0.001717428 (0.001494603) 	 Physics Loss: 0.000754971 (0.000796866) 	 Data Loss: 0.000408040 (0.000351864) 	 BC Loss: 0.000554417 (0.000345872)
-Iteration: [28001 / 50000] 	 Loss: 0.001498525 (0.001789054) 	 Physics Loss: 0.000881248 (0.001008533) 	 Data Loss: 0.000310122 (0.000482373) 	 BC Loss: 0.000307154 (0.000298148)
-Iteration: [28501 / 50000] 	 Loss: 0.001300987 (0.001541014) 	 Physics Loss: 0.000685782 (0.000873007) 	 Data Loss: 0.000372513 (0.000378884) 	 BC Loss: 0.000242692 (0.000289123)
-Iteration: [29001 / 50000] 	 Loss: 0.001302390 (0.001531350) 	 Physics Loss: 0.000764329 (0.000809421) 	 Data Loss: 0.000326938 (0.000448536) 	 BC Loss: 0.000211123 (0.000273393)
-Iteration: [29501 / 50000] 	 Loss: 0.001228882 (0.001639404) 	 Physics Loss: 0.000864926 (0.000892691) 	 Data Loss: 0.000257633 (0.000503808) 	 BC Loss: 0.000106323 (0.000242905)
-Iteration: [30001 / 50000] 	 Loss: 0.001362466 (0.001636085) 	 Physics Loss: 0.000635906 (0.000994050) 	 Data Loss: 0.000649345 (0.000412151) 	 BC Loss: 0.000077215 (0.000229884)
-Iteration: [30501 / 50000] 	 Loss: 0.000972152 (0.001479216) 	 Physics Loss: 0.000619220 (0.000883626) 	 Data Loss: 0.000230181 (0.000379169) 	 BC Loss: 0.000122752 (0.000216420)
-Iteration: [31001 / 50000] 	 Loss: 0.005065940 (0.001380907) 	 Physics Loss: 0.004397592 (0.000831873) 	 Data Loss: 0.000477057 (0.000344165) 	 BC Loss: 0.000191290 (0.000204869)
-Iteration: [31501 / 50000] 	 Loss: 0.001720798 (0.001439294) 	 Physics Loss: 0.001153235 (0.000778960) 	 Data Loss: 0.000366380 (0.000444564) 	 BC Loss: 0.000201183 (0.000215771)
-Iteration: [32001 / 50000] 	 Loss: 0.001272172 (0.001182971) 	 Physics Loss: 0.000547293 (0.000645338) 	 Data Loss: 0.000612554 (0.000330688) 	 BC Loss: 0.000112325 (0.000206944)
-Iteration: [32501 / 50000] 	 Loss: 0.001592739 (0.001303701) 	 Physics Loss: 0.001089040 (0.000732894) 	 Data Loss: 0.000368751 (0.000377450) 	 BC Loss: 0.000134948 (0.000193357)
-Iteration: [33001 / 50000] 	 Loss: 0.001257007 (0.001731674) 	 Physics Loss: 0.000964463 (0.001143823) 	 Data Loss: 0.000217459 (0.000395994) 	 BC Loss: 0.000075086 (0.000191858)
-Iteration: [33501 / 50000] 	 Loss: 0.001404967 (0.001307026) 	 Physics Loss: 0.000708074 (0.000685339) 	 Data Loss: 0.000530431 (0.000435325) 	 BC Loss: 0.000166462 (0.000186362)
-Iteration: [34001 / 50000] 	 Loss: 0.000798481 (0.001060399) 	 Physics Loss: 0.000366831 (0.000566683) 	 Data Loss: 0.000218087 (0.000317301) 	 BC Loss: 0.000213564 (0.000176415)
-Iteration: [34501 / 50000] 	 Loss: 0.001798573 (0.001364744) 	 Physics Loss: 0.001362987 (0.000847362) 	 Data Loss: 0.000308547 (0.000354412) 	 BC Loss: 0.000127040 (0.000162970)
-Iteration: [35001 / 50000] 	 Loss: 0.000781053 (0.001014936) 	 Physics Loss: 0.000422994 (0.000539345) 	 Data Loss: 0.000270824 (0.000311768) 	 BC Loss: 0.000087234 (0.000163823)
-Iteration: [35501 / 50000] 	 Loss: 0.001209691 (0.001329915) 	 Physics Loss: 0.000784697 (0.000694873) 	 Data Loss: 0.000264404 (0.000425117) 	 BC Loss: 0.000160591 (0.000209925)
-Iteration: [36001 / 50000] 	 Loss: 0.001398915 (0.001136703) 	 Physics Loss: 0.000490567 (0.000583094) 	 Data Loss: 0.000744279 (0.000401171) 	 BC Loss: 0.000164069 (0.000152438)
-Iteration: [36501 / 50000] 	 Loss: 0.000835373 (0.001367094) 	 Physics Loss: 0.000595709 (0.000792289) 	 Data Loss: 0.000132676 (0.000426714) 	 BC Loss: 0.000106987 (0.000148092)
-Iteration: [37001 / 50000] 	 Loss: 0.000952036 (0.001044580) 	 Physics Loss: 0.000709375 (0.000580873) 	 Data Loss: 0.000161545 (0.000334467) 	 BC Loss: 0.000081115 (0.000129240)
-Iteration: [37501 / 50000] 	 Loss: 0.000579479 (0.001085730) 	 Physics Loss: 0.000270049 (0.000599332) 	 Data Loss: 0.000242183 (0.000348117) 	 BC Loss: 0.000067247 (0.000138281)
-Iteration: [38001 / 50000] 	 Loss: 0.001053359 (0.001110448) 	 Physics Loss: 0.000445312 (0.000595116) 	 Data Loss: 0.000482833 (0.000353724) 	 BC Loss: 0.000125214 (0.000161607)
-Iteration: [38501 / 50000] 	 Loss: 0.000667115 (0.001049175) 	 Physics Loss: 0.000326662 (0.000543516) 	 Data Loss: 0.000173648 (0.000387762) 	 BC Loss: 0.000166804 (0.000117897)
-Iteration: [39001 / 50000] 	 Loss: 0.000687853 (0.001085206) 	 Physics Loss: 0.000313576 (0.000590442) 	 Data Loss: 0.000321921 (0.000367264) 	 BC Loss: 0.000052356 (0.000127500)
-Iteration: [39501 / 50000] 	 Loss: 0.000673423 (0.001160439) 	 Physics Loss: 0.000421422 (0.000714949) 	 Data Loss: 0.000192454 (0.000300986) 	 BC Loss: 0.000059547 (0.000144504)
-Iteration: [40001 / 50000] 	 Loss: 0.000938448 (0.001243982) 	 Physics Loss: 0.000547043 (0.000710499) 	 Data Loss: 0.000287677 (0.000374305) 	 BC Loss: 0.000103729 (0.000159178)
-Iteration: [40501 / 50000] 	 Loss: 0.001798752 (0.001093546) 	 Physics Loss: 0.000866074 (0.000650020) 	 Data Loss: 0.000780730 (0.000305183) 	 BC Loss: 0.000151947 (0.000138344)
-Iteration: [41001 / 50000] 	 Loss: 0.001101400 (0.001448896) 	 Physics Loss: 0.000600817 (0.000901216) 	 Data Loss: 0.000405329 (0.000389668) 	 BC Loss: 0.000095254 (0.000158011)
-Iteration: [41501 / 50000] 	 Loss: 0.000776893 (0.000824789) 	 Physics Loss: 0.000455096 (0.000410463) 	 Data Loss: 0.000206055 (0.000317277) 	 BC Loss: 0.000115742 (0.000097049)
-Iteration: [42001 / 50000] 	 Loss: 0.001060384 (0.001240770) 	 Physics Loss: 0.000583868 (0.000778712) 	 Data Loss: 0.000378937 (0.000335865) 	 BC Loss: 0.000097578 (0.000126193)
-Iteration: [42501 / 50000] 	 Loss: 0.000691815 (0.000944194) 	 Physics Loss: 0.000381213 (0.000516831) 	 Data Loss: 0.000162085 (0.000307154) 	 BC Loss: 0.000148517 (0.000120209)
-Iteration: [43001 / 50000] 	 Loss: 0.000545245 (0.000868335) 	 Physics Loss: 0.000363892 (0.000449657) 	 Data Loss: 0.000124517 (0.000324281) 	 BC Loss: 0.000056836 (0.000094398)
-Iteration: [43501 / 50000] 	 Loss: 0.001068081 (0.001141232) 	 Physics Loss: 0.000613936 (0.000622923) 	 Data Loss: 0.000369106 (0.000392804) 	 BC Loss: 0.000085040 (0.000125504)
-Iteration: [44001 / 50000] 	 Loss: 0.001168622 (0.000963809) 	 Physics Loss: 0.000486189 (0.000540564) 	 Data Loss: 0.000549527 (0.000306839) 	 BC Loss: 0.000132906 (0.000116407)
-Iteration: [44501 / 50000] 	 Loss: 0.000939374 (0.000756769) 	 Physics Loss: 0.000379668 (0.000386392) 	 Data Loss: 0.000454265 (0.000277635) 	 BC Loss: 0.000105440 (0.000092742)
-Iteration: [45001 / 50000] 	 Loss: 0.001202180 (0.000892342) 	 Physics Loss: 0.000726597 (0.000506370) 	 Data Loss: 0.000367492 (0.000280093) 	 BC Loss: 0.000108091 (0.000105879)
-Iteration: [45501 / 50000] 	 Loss: 0.000887047 (0.000908168) 	 Physics Loss: 0.000641108 (0.000560744) 	 Data Loss: 0.000148591 (0.000263541) 	 BC Loss: 0.000097348 (0.000083883)
-Iteration: [46001 / 50000] 	 Loss: 0.000525872 (0.000734937) 	 Physics Loss: 0.000322635 (0.000362759) 	 Data Loss: 0.000158824 (0.000288360) 	 BC Loss: 0.000044412 (0.000083818)
-Iteration: [46501 / 50000] 	 Loss: 0.000545968 (0.000907963) 	 Physics Loss: 0.000350560 (0.000489585) 	 Data Loss: 0.000136768 (0.000312205) 	 BC Loss: 0.000058640 (0.000106173)
-Iteration: [47001 / 50000] 	 Loss: 0.000826687 (0.000924496) 	 Physics Loss: 0.000575209 (0.000511645) 	 Data Loss: 0.000205284 (0.000314716) 	 BC Loss: 0.000046194 (0.000098134)
-Iteration: [47501 / 50000] 	 Loss: 0.000620959 (0.001185708) 	 Physics Loss: 0.000382126 (0.000788119) 	 Data Loss: 0.000148993 (0.000271556) 	 BC Loss: 0.000089840 (0.000126034)
-Iteration: [48001 / 50000] 	 Loss: 0.000664698 (0.000798627) 	 Physics Loss: 0.000412207 (0.000461797) 	 Data Loss: 0.000182339 (0.000250679) 	 BC Loss: 0.000070152 (0.000086152)
-Iteration: [48501 / 50000] 	 Loss: 0.000826713 (0.000925605) 	 Physics Loss: 0.000485339 (0.000512687) 	 Data Loss: 0.000170909 (0.000314013) 	 BC Loss: 0.000170466 (0.000098905)
-Iteration: [49001 / 50000] 	 Loss: 0.000559556 (0.000846881) 	 Physics Loss: 0.000350503 (0.000421408) 	 Data Loss: 0.000165743 (0.000346362) 	 BC Loss: 0.000043311 (0.000079111)
-Iteration: [49501 / 50000] 	 Loss: 0.000652327 (0.000740137) 	 Physics Loss: 0.000301691 (0.000376061) 	 Data Loss: 0.000195276 (0.000275985) 	 BC Loss: 0.000155360 (0.000088090)
    Visualizing the Results ​
    julia
    ts, xs, ys = 0.0f0:0.05f0:2.0f0, 0.0f0:0.02f0:2.0f0, 0.0f0:0.02f0:2.0f0
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162595 (1.982162595) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505020 (0.598505020)
+Iteration: [  501 / 50000] 	 Loss: 0.040918142 (0.025583766) 	 Physics Loss: 0.000391871 (0.000269293) 	 Data Loss: 0.014243502 (0.009196415) 	 BC Loss: 0.026282769 (0.016118057)
+Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281416) 	 Physics Loss: 0.000071671 (0.000163183) 	 Data Loss: 0.007905646 (0.010876307) 	 BC Loss: 0.007363342 (0.014241925)
+Iteration: [ 1501 / 50000] 	 Loss: 0.019566942 (0.026170360) 	 Physics Loss: 0.001279285 (0.001009776) 	 Data Loss: 0.003071634 (0.010452234) 	 BC Loss: 0.015216023 (0.014708348)
+Iteration: [ 2001 / 50000] 	 Loss: 0.035557814 (0.027273539) 	 Physics Loss: 0.004062099 (0.001872000) 	 Data Loss: 0.013011657 (0.010586375) 	 BC Loss: 0.018484056 (0.014815168)
+Iteration: [ 2501 / 50000] 	 Loss: 0.011523969 (0.022175739) 	 Physics Loss: 0.002366644 (0.001909967) 	 Data Loss: 0.005568597 (0.007926281) 	 BC Loss: 0.003588729 (0.012339489)
+Iteration: [ 3001 / 50000] 	 Loss: 0.026762877 (0.031860039) 	 Physics Loss: 0.003192307 (0.006136029) 	 Data Loss: 0.009067146 (0.010883353) 	 BC Loss: 0.014503423 (0.014840655)
+Iteration: [ 3501 / 50000] 	 Loss: 0.025398642 (0.045591548) 	 Physics Loss: 0.003200589 (0.001476229) 	 Data Loss: 0.007934441 (0.019488331) 	 BC Loss: 0.014263613 (0.024626993)
+Iteration: [ 4001 / 50000] 	 Loss: 0.024822138 (0.029854555) 	 Physics Loss: 0.001797515 (0.003666145) 	 Data Loss: 0.012387147 (0.011416912) 	 BC Loss: 0.010637476 (0.014771502)
+Iteration: [ 4501 / 50000] 	 Loss: 0.029747024 (0.031501673) 	 Physics Loss: 0.005927891 (0.004604227) 	 Data Loss: 0.018316794 (0.011011195) 	 BC Loss: 0.005502339 (0.015886255)
+Iteration: [ 5001 / 50000] 	 Loss: 0.025349757 (0.043140102) 	 Physics Loss: 0.000728280 (0.002826607) 	 Data Loss: 0.003646539 (0.016781844) 	 BC Loss: 0.020974938 (0.023531647)
+Iteration: [ 5501 / 50000] 	 Loss: 0.026193798 (0.022408014) 	 Physics Loss: 0.000325338 (0.000346806) 	 Data Loss: 0.004838886 (0.008093885) 	 BC Loss: 0.021029573 (0.013967324)
+Iteration: [ 6001 / 50000] 	 Loss: 0.021179065 (0.022343241) 	 Physics Loss: 0.000527887 (0.000702699) 	 Data Loss: 0.007729640 (0.007971080) 	 BC Loss: 0.012921538 (0.013669461)
+Iteration: [ 6501 / 50000] 	 Loss: 0.030424416 (0.021639718) 	 Physics Loss: 0.000403142 (0.000825630) 	 Data Loss: 0.013241904 (0.009147485) 	 BC Loss: 0.016779371 (0.011666608)
+Iteration: [ 7001 / 50000] 	 Loss: 0.012885604 (0.022515845) 	 Physics Loss: 0.000715418 (0.000757549) 	 Data Loss: 0.006018532 (0.009810397) 	 BC Loss: 0.006151655 (0.011947901)
+Iteration: [ 7501 / 50000] 	 Loss: 0.021707822 (0.020194370) 	 Physics Loss: 0.000589236 (0.000854950) 	 Data Loss: 0.011797804 (0.008223927) 	 BC Loss: 0.009320782 (0.011115490)
+Iteration: [ 8001 / 50000] 	 Loss: 0.021062549 (0.019913949) 	 Physics Loss: 0.000528065 (0.000852256) 	 Data Loss: 0.008882683 (0.007675199) 	 BC Loss: 0.011651803 (0.011386497)
+Iteration: [ 8501 / 50000] 	 Loss: 0.022159226 (0.022652669) 	 Physics Loss: 0.001293960 (0.001621564) 	 Data Loss: 0.009795113 (0.008712809) 	 BC Loss: 0.011070153 (0.012318298)
+Iteration: [ 9001 / 50000] 	 Loss: 0.022167893 (0.021835698) 	 Physics Loss: 0.001893832 (0.001330249) 	 Data Loss: 0.006999766 (0.008149015) 	 BC Loss: 0.013274295 (0.012356437)
+Iteration: [ 9501 / 50000] 	 Loss: 0.016295999 (0.020470668) 	 Physics Loss: 0.002220259 (0.000820911) 	 Data Loss: 0.006721330 (0.008737788) 	 BC Loss: 0.007354409 (0.010911969)
+Iteration: [10001 / 50000] 	 Loss: 0.020919006 (0.022420531) 	 Physics Loss: 0.001985265 (0.000919413) 	 Data Loss: 0.003050850 (0.008777075) 	 BC Loss: 0.015882893 (0.012724041)
+Iteration: [10501 / 50000] 	 Loss: 0.015559800 (0.017435743) 	 Physics Loss: 0.000461346 (0.000464835) 	 Data Loss: 0.005280917 (0.006347454) 	 BC Loss: 0.009817538 (0.010623454)
+Iteration: [11001 / 50000] 	 Loss: 0.022515260 (0.018052436) 	 Physics Loss: 0.000395792 (0.000500864) 	 Data Loss: 0.004308904 (0.006823588) 	 BC Loss: 0.017810564 (0.010727987)
+Iteration: [11501 / 50000] 	 Loss: 0.021128871 (0.017566014) 	 Physics Loss: 0.000394875 (0.000523516) 	 Data Loss: 0.005251979 (0.006506131) 	 BC Loss: 0.015482016 (0.010536365)
+Iteration: [12001 / 50000] 	 Loss: 0.030411797 (0.018228574) 	 Physics Loss: 0.000672969 (0.000554123) 	 Data Loss: 0.019357491 (0.007100587) 	 BC Loss: 0.010381337 (0.010573865)
+Iteration: [12501 / 50000] 	 Loss: 0.018811548 (0.018401809) 	 Physics Loss: 0.000697236 (0.000691837) 	 Data Loss: 0.002697282 (0.006085245) 	 BC Loss: 0.015417031 (0.011624727)
+Iteration: [13001 / 50000] 	 Loss: 0.011868341 (0.017906483) 	 Physics Loss: 0.000776913 (0.000821749) 	 Data Loss: 0.003124564 (0.006143386) 	 BC Loss: 0.007966863 (0.010941349)
+Iteration: [13501 / 50000] 	 Loss: 0.014784457 (0.018100940) 	 Physics Loss: 0.000598925 (0.000848341) 	 Data Loss: 0.006280850 (0.006679880) 	 BC Loss: 0.007904682 (0.010572717)
+Iteration: [14001 / 50000] 	 Loss: 0.015592407 (0.016142424) 	 Physics Loss: 0.000597963 (0.000956681) 	 Data Loss: 0.006269322 (0.005398072) 	 BC Loss: 0.008725122 (0.009787671)
+Iteration: [14501 / 50000] 	 Loss: 0.025311429 (0.015725624) 	 Physics Loss: 0.001073368 (0.001058255) 	 Data Loss: 0.009163789 (0.005325366) 	 BC Loss: 0.015074271 (0.009342002)
+Iteration: [15001 / 50000] 	 Loss: 0.016914256 (0.015927596) 	 Physics Loss: 0.000966280 (0.001092387) 	 Data Loss: 0.005907979 (0.004847473) 	 BC Loss: 0.010039996 (0.009987737)
+Iteration: [15501 / 50000] 	 Loss: 0.010531792 (0.017309623) 	 Physics Loss: 0.001679290 (0.001229975) 	 Data Loss: 0.002361344 (0.005612177) 	 BC Loss: 0.006491158 (0.010467471)
+Iteration: [16001 / 50000] 	 Loss: 0.012041669 (0.015774786) 	 Physics Loss: 0.000827484 (0.001132011) 	 Data Loss: 0.001785598 (0.004045264) 	 BC Loss: 0.009428588 (0.010597511)
+Iteration: [16501 / 50000] 	 Loss: 0.017424010 (0.015305851) 	 Physics Loss: 0.001297660 (0.001166611) 	 Data Loss: 0.004497696 (0.004839269) 	 BC Loss: 0.011628653 (0.009299970)
+Iteration: [17001 / 50000] 	 Loss: 0.012141772 (0.013891050) 	 Physics Loss: 0.001339099 (0.001353682) 	 Data Loss: 0.001933840 (0.003089970) 	 BC Loss: 0.008868832 (0.009447398)
+Iteration: [17501 / 50000] 	 Loss: 0.015801162 (0.014529660) 	 Physics Loss: 0.001557242 (0.001193460) 	 Data Loss: 0.004102683 (0.004099627) 	 BC Loss: 0.010141237 (0.009236576)
+Iteration: [18001 / 50000] 	 Loss: 0.011515658 (0.013194922) 	 Physics Loss: 0.001411696 (0.001229187) 	 Data Loss: 0.003250526 (0.003880639) 	 BC Loss: 0.006853437 (0.008085096)
+Iteration: [18501 / 50000] 	 Loss: 0.014405347 (0.014845728) 	 Physics Loss: 0.000946846 (0.001311332) 	 Data Loss: 0.005034053 (0.004776356) 	 BC Loss: 0.008424449 (0.008758042)
+Iteration: [19001 / 50000] 	 Loss: 0.015056589 (0.013655247) 	 Physics Loss: 0.001184456 (0.001292421) 	 Data Loss: 0.003681621 (0.003807114) 	 BC Loss: 0.010190512 (0.008555712)
+Iteration: [19501 / 50000] 	 Loss: 0.013299890 (0.013724705) 	 Physics Loss: 0.001885105 (0.001415296) 	 Data Loss: 0.005226072 (0.003555858) 	 BC Loss: 0.006188713 (0.008753551)
+Iteration: [20001 / 50000] 	 Loss: 0.017522790 (0.012518023) 	 Physics Loss: 0.001983398 (0.001058186) 	 Data Loss: 0.009327068 (0.002995419) 	 BC Loss: 0.006212323 (0.008464416)
+Iteration: [20501 / 50000] 	 Loss: 0.021622123 (0.012733568) 	 Physics Loss: 0.001156326 (0.001213073) 	 Data Loss: 0.002187656 (0.003051734) 	 BC Loss: 0.018278142 (0.008468761)
+Iteration: [21001 / 50000] 	 Loss: 0.012820827 (0.013354947) 	 Physics Loss: 0.001192615 (0.001322284) 	 Data Loss: 0.000899577 (0.003003312) 	 BC Loss: 0.010728635 (0.009029351)
+Iteration: [21501 / 50000] 	 Loss: 0.016243555 (0.012213146) 	 Physics Loss: 0.002533023 (0.001458879) 	 Data Loss: 0.008658717 (0.003036225) 	 BC Loss: 0.005051813 (0.007718043)
+Iteration: [22001 / 50000] 	 Loss: 0.009802144 (0.011197914) 	 Physics Loss: 0.000975115 (0.001180587) 	 Data Loss: 0.002004704 (0.002875937) 	 BC Loss: 0.006822325 (0.007141389)
+Iteration: [22501 / 50000] 	 Loss: 0.010460960 (0.011299610) 	 Physics Loss: 0.001939239 (0.001350272) 	 Data Loss: 0.003044792 (0.002758041) 	 BC Loss: 0.005476930 (0.007191297)
+Iteration: [23001 / 50000] 	 Loss: 0.007303775 (0.010617163) 	 Physics Loss: 0.001568149 (0.001255589) 	 Data Loss: 0.002208042 (0.002901576) 	 BC Loss: 0.003527584 (0.006459999)
+Iteration: [23501 / 50000] 	 Loss: 0.013054269 (0.012929341) 	 Physics Loss: 0.001233365 (0.001229342) 	 Data Loss: 0.002502190 (0.002853266) 	 BC Loss: 0.009318714 (0.008846736)
+Iteration: [24001 / 50000] 	 Loss: 0.015850905 (0.011500452) 	 Physics Loss: 0.001933446 (0.001243522) 	 Data Loss: 0.003105919 (0.002514284) 	 BC Loss: 0.010811541 (0.007742645)
+Iteration: [24501 / 50000] 	 Loss: 0.014612451 (0.011235875) 	 Physics Loss: 0.000830096 (0.001126977) 	 Data Loss: 0.001665982 (0.002516578) 	 BC Loss: 0.012116373 (0.007592318)
+Iteration: [25001 / 50000] 	 Loss: 0.012446284 (0.012136834) 	 Physics Loss: 0.001868680 (0.001571323) 	 Data Loss: 0.002885855 (0.002738171) 	 BC Loss: 0.007691749 (0.007827340)
+Iteration: [25501 / 50000] 	 Loss: 0.006147437 (0.010866777) 	 Physics Loss: 0.001199473 (0.001432847) 	 Data Loss: 0.000741202 (0.003181246) 	 BC Loss: 0.004206762 (0.006252684)
+Iteration: [26001 / 50000] 	 Loss: 0.009382382 (0.012447415) 	 Physics Loss: 0.002055192 (0.001882480) 	 Data Loss: 0.001872860 (0.002723655) 	 BC Loss: 0.005454330 (0.007841279)
+Iteration: [26501 / 50000] 	 Loss: 0.005613507 (0.009200627) 	 Physics Loss: 0.001204579 (0.001562393) 	 Data Loss: 0.001970439 (0.002349454) 	 BC Loss: 0.002438489 (0.005288779)
+Iteration: [27001 / 50000] 	 Loss: 0.008868203 (0.010649174) 	 Physics Loss: 0.001465238 (0.001703513) 	 Data Loss: 0.002235665 (0.002514324) 	 BC Loss: 0.005167301 (0.006431336)
+Iteration: [27501 / 50000] 	 Loss: 0.016564976 (0.011443286) 	 Physics Loss: 0.001060150 (0.001400374) 	 Data Loss: 0.003114783 (0.002061526) 	 BC Loss: 0.012390045 (0.007981384)
+Iteration: [28001 / 50000] 	 Loss: 0.008029928 (0.011309550) 	 Physics Loss: 0.001292446 (0.001715263) 	 Data Loss: 0.001651582 (0.003145006) 	 BC Loss: 0.005085900 (0.006449282)
+Iteration: [28501 / 50000] 	 Loss: 0.009412916 (0.010573968) 	 Physics Loss: 0.002310942 (0.001589268) 	 Data Loss: 0.000935075 (0.002185026) 	 BC Loss: 0.006166900 (0.006799675)
+Iteration: [29001 / 50000] 	 Loss: 0.008956734 (0.010948162) 	 Physics Loss: 0.001730394 (0.001973851) 	 Data Loss: 0.002910380 (0.002606902) 	 BC Loss: 0.004315960 (0.006367408)
+Iteration: [29501 / 50000] 	 Loss: 0.004513775 (0.010220452) 	 Physics Loss: 0.000894471 (0.001687118) 	 Data Loss: 0.001609863 (0.002537781) 	 BC Loss: 0.002009441 (0.005995554)
+Iteration: [30001 / 50000] 	 Loss: 0.004987885 (0.009731424) 	 Physics Loss: 0.001069190 (0.001562893) 	 Data Loss: 0.001401214 (0.002276014) 	 BC Loss: 0.002517481 (0.005892518)
+Iteration: [30501 / 50000] 	 Loss: 0.002595518 (0.008276105) 	 Physics Loss: 0.000917391 (0.001500407) 	 Data Loss: 0.000658372 (0.001976204) 	 BC Loss: 0.001019756 (0.004799493)
+Iteration: [31001 / 50000] 	 Loss: 0.011266185 (0.010293764) 	 Physics Loss: 0.001391491 (0.001473510) 	 Data Loss: 0.003635545 (0.002334290) 	 BC Loss: 0.006239149 (0.006485965)
+Iteration: [31501 / 50000] 	 Loss: 0.013991628 (0.010204735) 	 Physics Loss: 0.002423544 (0.001848977) 	 Data Loss: 0.004265617 (0.002343330) 	 BC Loss: 0.007302467 (0.006012429)
+Iteration: [32001 / 50000] 	 Loss: 0.007164373 (0.009639673) 	 Physics Loss: 0.001379599 (0.001698582) 	 Data Loss: 0.001701293 (0.001939833) 	 BC Loss: 0.004083481 (0.006001258)
+Iteration: [32501 / 50000] 	 Loss: 0.005502027 (0.009115184) 	 Physics Loss: 0.001436674 (0.001596168) 	 Data Loss: 0.001871673 (0.001895187) 	 BC Loss: 0.002193680 (0.005623828)
+Iteration: [33001 / 50000] 	 Loss: 0.005276884 (0.009546057) 	 Physics Loss: 0.002079992 (0.001929016) 	 Data Loss: 0.001436205 (0.001975889) 	 BC Loss: 0.001760687 (0.005641153)
+Iteration: [33501 / 50000] 	 Loss: 0.009125081 (0.008468037) 	 Physics Loss: 0.002191632 (0.001853661) 	 Data Loss: 0.002888216 (0.001945538) 	 BC Loss: 0.004045234 (0.004668837)
+Iteration: [34001 / 50000] 	 Loss: 0.008431979 (0.009071054) 	 Physics Loss: 0.001222603 (0.002027335) 	 Data Loss: 0.001151307 (0.001467187) 	 BC Loss: 0.006058069 (0.005576532)
+Iteration: [34501 / 50000] 	 Loss: 0.009580643 (0.008130345) 	 Physics Loss: 0.002242853 (0.001757617) 	 Data Loss: 0.005897780 (0.002249273) 	 BC Loss: 0.001440010 (0.004123454)
+Iteration: [35001 / 50000] 	 Loss: 0.005147196 (0.007530828) 	 Physics Loss: 0.001256183 (0.001518356) 	 Data Loss: 0.001415687 (0.001431392) 	 BC Loss: 0.002475326 (0.004581080)
+Iteration: [35501 / 50000] 	 Loss: 0.006185614 (0.008019534) 	 Physics Loss: 0.001828601 (0.001882976) 	 Data Loss: 0.001599119 (0.001704919) 	 BC Loss: 0.002757893 (0.004431641)
+Iteration: [36001 / 50000] 	 Loss: 0.009109061 (0.007414200) 	 Physics Loss: 0.001948264 (0.001928137) 	 Data Loss: 0.001738493 (0.001558672) 	 BC Loss: 0.005422304 (0.003927392)
+Iteration: [36501 / 50000] 	 Loss: 0.007278774 (0.007622246) 	 Physics Loss: 0.002025729 (0.001955235) 	 Data Loss: 0.002425219 (0.001680354) 	 BC Loss: 0.002827825 (0.003986657)
+Iteration: [37001 / 50000] 	 Loss: 0.003918901 (0.005813450) 	 Physics Loss: 0.000957004 (0.001656469) 	 Data Loss: 0.001278843 (0.001385545) 	 BC Loss: 0.001683053 (0.002771435)
+Iteration: [37501 / 50000] 	 Loss: 0.004814619 (0.006178515) 	 Physics Loss: 0.001686519 (0.001814278) 	 Data Loss: 0.001063806 (0.001266953) 	 BC Loss: 0.002064294 (0.003097285)
+Iteration: [38001 / 50000] 	 Loss: 0.004643182 (0.004791833) 	 Physics Loss: 0.001702445 (0.001350533) 	 Data Loss: 0.000782973 (0.001124672) 	 BC Loss: 0.002157764 (0.002316626)
+Iteration: [38501 / 50000] 	 Loss: 0.003810477 (0.004910624) 	 Physics Loss: 0.001390965 (0.001866840) 	 Data Loss: 0.000597971 (0.000985127) 	 BC Loss: 0.001821541 (0.002058655)
+Iteration: [39001 / 50000] 	 Loss: 0.003878577 (0.005460986) 	 Physics Loss: 0.002414002 (0.002075948) 	 Data Loss: 0.000412366 (0.001095256) 	 BC Loss: 0.001052209 (0.002289784)
+Iteration: [39501 / 50000] 	 Loss: 0.004065084 (0.004690929) 	 Physics Loss: 0.002393598 (0.001760620) 	 Data Loss: 0.000800843 (0.000796935) 	 BC Loss: 0.000870643 (0.002133374)
+Iteration: [40001 / 50000] 	 Loss: 0.004619529 (0.004839203) 	 Physics Loss: 0.001658047 (0.001965609) 	 Data Loss: 0.001588614 (0.000951834) 	 BC Loss: 0.001372868 (0.001921760)
+Iteration: [40501 / 50000] 	 Loss: 0.004326439 (0.004265270) 	 Physics Loss: 0.001820831 (0.001626283) 	 Data Loss: 0.001034181 (0.000920903) 	 BC Loss: 0.001471427 (0.001718084)
+Iteration: [41001 / 50000] 	 Loss: 0.002906828 (0.003618096) 	 Physics Loss: 0.001272737 (0.001407278) 	 Data Loss: 0.000809125 (0.000837591) 	 BC Loss: 0.000824967 (0.001373227)
+Iteration: [41501 / 50000] 	 Loss: 0.004206825 (0.003857500) 	 Physics Loss: 0.001842899 (0.001706076) 	 Data Loss: 0.000595867 (0.000833410) 	 BC Loss: 0.001768059 (0.001318014)
+Iteration: [42001 / 50000] 	 Loss: 0.003873884 (0.003361913) 	 Physics Loss: 0.002234953 (0.001453047) 	 Data Loss: 0.000641468 (0.000736817) 	 BC Loss: 0.000997462 (0.001172049)
+Iteration: [42501 / 50000] 	 Loss: 0.003796777 (0.003109351) 	 Physics Loss: 0.001149042 (0.001176956) 	 Data Loss: 0.000481490 (0.000653678) 	 BC Loss: 0.002166245 (0.001278718)
+Iteration: [43001 / 50000] 	 Loss: 0.001652764 (0.003025806) 	 Physics Loss: 0.000779939 (0.001280978) 	 Data Loss: 0.000271947 (0.000640416) 	 BC Loss: 0.000600878 (0.001104411)
+Iteration: [43501 / 50000] 	 Loss: 0.002625272 (0.003282005) 	 Physics Loss: 0.001812232 (0.001467459) 	 Data Loss: 0.000510654 (0.000728400) 	 BC Loss: 0.000302386 (0.001086145)
+Iteration: [44001 / 50000] 	 Loss: 0.002716297 (0.002971325) 	 Physics Loss: 0.001654499 (0.001524955) 	 Data Loss: 0.000789657 (0.000616715) 	 BC Loss: 0.000272142 (0.000829655)
+Iteration: [44501 / 50000] 	 Loss: 0.003345200 (0.002695851) 	 Physics Loss: 0.001435974 (0.001263972) 	 Data Loss: 0.001142784 (0.000586666) 	 BC Loss: 0.000766442 (0.000845213)
+Iteration: [45001 / 50000] 	 Loss: 0.002160525 (0.002570425) 	 Physics Loss: 0.001014665 (0.001206274) 	 Data Loss: 0.000531116 (0.000582542) 	 BC Loss: 0.000614744 (0.000781609)
+Iteration: [45501 / 50000] 	 Loss: 0.002704483 (0.002375467) 	 Physics Loss: 0.001256422 (0.001126713) 	 Data Loss: 0.000432294 (0.000537340) 	 BC Loss: 0.001015767 (0.000711415)
+Iteration: [46001 / 50000] 	 Loss: 0.002083311 (0.002229830) 	 Physics Loss: 0.001028772 (0.001128200) 	 Data Loss: 0.000365198 (0.000481077) 	 BC Loss: 0.000689341 (0.000620552)
+Iteration: [46501 / 50000] 	 Loss: 0.001869006 (0.002387334) 	 Physics Loss: 0.001049295 (0.001213058) 	 Data Loss: 0.000542785 (0.000525414) 	 BC Loss: 0.000276926 (0.000648862)
+Iteration: [47001 / 50000] 	 Loss: 0.002161729 (0.002562924) 	 Physics Loss: 0.001238901 (0.001363050) 	 Data Loss: 0.000660004 (0.000578991) 	 BC Loss: 0.000262824 (0.000620883)
+Iteration: [47501 / 50000] 	 Loss: 0.003218761 (0.002436990) 	 Physics Loss: 0.001678471 (0.001291234) 	 Data Loss: 0.000470345 (0.000499937) 	 BC Loss: 0.001069946 (0.000645818)
+Iteration: [48001 / 50000] 	 Loss: 0.001741661 (0.001888746) 	 Physics Loss: 0.000994589 (0.000895934) 	 Data Loss: 0.000492043 (0.000409009) 	 BC Loss: 0.000255029 (0.000583803)
+Iteration: [48501 / 50000] 	 Loss: 0.001460486 (0.002372291) 	 Physics Loss: 0.000619964 (0.001287611) 	 Data Loss: 0.000285411 (0.000508584) 	 BC Loss: 0.000555111 (0.000576096)
+Iteration: [49001 / 50000] 	 Loss: 0.001497735 (0.002357919) 	 Physics Loss: 0.000912286 (0.001282274) 	 Data Loss: 0.000287049 (0.000532408) 	 BC Loss: 0.000298400 (0.000543235)
+Iteration: [49501 / 50000] 	 Loss: 0.003226456 (0.001985326) 	 Physics Loss: 0.001385620 (0.000979689) 	 Data Loss: 0.000554629 (0.000451327) 	 BC Loss: 0.001286207 (0.000554309)
    Visualizing the Results ​
    julia
    ts, xs, ys = 0.0f0:0.05f0:2.0f0, 0.0f0:0.02f0:2.0f0, 0.0f0:0.02f0:2.0f0
 grid = stack([[elem...] for elem in vec(collect(Iterators.product(xs, ys, ts)))])
 
 u_real = reshape(analytical_solution(grid), length(xs), length(ys), length(ts))
@@ -280,7 +280,7 @@ import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1
     end
 
     fig
-end
    Appendix ​
    julia
    using InteractiveUtils
+end
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -339,5 +339,6 @@ import{_ as h,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.CCjWn1
 Environment:
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
-1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,12))])}const B=h(k,[["render",o]]);export{m as __pageData,B as default};
+2 devices:
+  0: Quadro RTX 5000 (sm_75, 15.258 GiB / 16.000 GiB available)
+  1: Quadro RTX 5000 (sm_75, 15.549 GiB / 16.000 GiB available)

    This page was generated using Literate.jl.

    `,12))])}const B=p(k,[["render",o]]);export{m as __pageData,B as default}; diff --git a/dev/ecosystem.html b/dev/ecosystem.html index f9d46b428b..f813401b95 100644 --- a/dev/ecosystem.html +++ b/dev/ecosystem.html @@ -5,14 +5,14 @@ Lux.jl Docs - - + + - + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance

    Ecosystem

    Frameworks Extending Lux.jl

    DiffEqFlux.jl

    DiffEqFlux.jl

    Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods

    SciMLSensitivity.jl

    SciMLSensitivity.jl

    A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.

    NeuralPDE.jl

    NeuralPDE.jl

    Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation

    NeuralLyapunov.jl

    NeuralLyapunov.jl

    A library for searching for neural Lyapunov functions in Julia

    DeepEquilibriumNetworks.jl

    DeepEquilibriumNetworks.jl

    Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence

    AbstractCosmologicalEmulators.jl

    AbstractCosmologicalEmulators.jl

    Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization

    ContinuousNormalizingFlows.jl

    ContinuousNormalizingFlows.jl

    Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia

    Sophon.jl

    Sophon.jl

    Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks

    DataDrivenDiffEq.jl

    DataDrivenDiffEq.jl

    Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization

    NeuralGraphPDE.jl

    NeuralGraphPDE.jl

    Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks

    Solaris.jl

    Solaris.jl

    Lightweight module for fusing physical and neural models

    Boltz.jl

    Boltz.jl

    Accelerate your ML research using pre-built Deep Learning Models with Lux

    GeometricMachineLearning.jl

    GeometricMachineLearning.jl

    Structure Preserving Machine Learning Models in Julia

    Automatic Differentiation

    Zygote.jl

    Zygote.jl

    Lux.jl default choice for AD

    Tracker.jl

    Tracker.jl

    Well tested and robust AD library (might fail on edge cases)

    ForwardDiff.jl

    ForwardDiff.jl

    For forward mode AD support

    ReverseDiff.jl

    ReverseDiff.jl

    Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)

    Enzyme.jl

    Enzyme.jl

    Experimental Support but will become the Future Default

    Data Manipulation, Data Loading & Datasets

    MLUtils.jl

    MLUtils.jl

    Utilities and abstractions for Machine Learning tasks

    MLDatasets.jl

    MLDatasets.jl

    Utility package for accessing common Machine Learning datasets in Julia

    Images.jl

    Images.jl

    An image library for Julia

    DataAugmentation.jl

    DataAugmentation.jl

    Flexible data augmentation library for machine and deep learning

    Neural Network Primitives

    NNlib.jl

    NNlib.jl

    Neural Network primitives with multiple backends

    LuxLib.jl

    LuxLib.jl

    Backend for Lux.jl

    Optimization

    Optimization.jl

    Optimization.jl

    Unified API for Optimization in Julia

    Optimisers.jl

    Optimisers.jl

    Optimisers.jl defines many standard optimisers and utilities for learning loops

    ParameterSchedulers.jl

    ParameterSchedulers.jl

    Common hyperparameter scheduling for ML

    Parameter Manipulation

    Functors.jl

    Functors.jl

    Parameterise all the things

    ComponentArrays.jl

    ComponentArrays.jl

    Arrays with arbitrarily nested named components

    Serialization

    Serialization.jl

    Serialization.jl

    Provides serialization of Julia objects

    JLD2.jl

    JLD2.jl

    HDF5-compatible file format in pure Julia

    Testing Utilities

    FiniteDiff.jl

    FiniteDiff.jl

    Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support

    FiniteDifferences.jl

    FiniteDifferences.jl

    High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)

    JET.jl

    JET.jl

    JET employs Julia's type inference system to detect potential bugs and type instabilities

    LuxTestUtils.jl

    LuxTestUtils.jl

    Collection of Functions useful for testing various packages in the Lux Ecosystem

    Training Visualization & Logging

    MLFlowClient.jl

    MLFlowClient.jl

    Julia client for MLFlow

    TensorBoardLogger.jl

    TensorBoardLogger.jl

    Easy peasy logging to TensorBoard with Julia

    Wandb.jl

    Wandb.jl

    Unofficial Julia bindings for logging experiments to wandb.ai

    - + \ No newline at end of file diff --git a/dev/hashmap.json b/dev/hashmap.json index 50e10a1414..f5d37d84ee 100644 --- a/dev/hashmap.json +++ b/dev/hashmap.json @@ -1 +1 @@ -{"api_accelerator_support_mldatadevices.md":"S5wrHgXm","api_building_blocks_luxcore.md":"DMM0tZYm","api_building_blocks_luxlib.md":"BKdoAGCk","api_building_blocks_weightinitializers.md":"BQtmXzUN","api_lux_autodiff.md":"C9onSrGW","api_lux_contrib.md":"BRGt7rCL","api_lux_distributed_utils.md":"DGZwfoze","api_lux_interop.md":"CseYUMhv","api_lux_layers.md":"DxMsNQ36","api_lux_utilities.md":"BJScatXv","api_testing_functionality_luxtestutils.md":"gICZ2tlR","ecosystem.md":"BywRar4V","index.md":"B5rDYBAv","introduction_citation.md":"D-b8AVke","introduction_index.md":"C7fSJQdT","introduction_overview.md":"CcMCcXqn","introduction_resources.md":"BY58ogcQ","introduction_updating_to_v1.md":"uRcSgSbZ","manual_autodiff.md":"bpTWk2c2","manual_compiling_lux_models.md":"DPjSTTUI","manual_debugging.md":"DrOcMYR1","manual_dispatch_custom_input.md":"Btlkn6MF","manual_distributed_utils.md":"DMnBWmL6","manual_freezing_model_parameters.md":"DzvhQn_f","manual_gpu_management.md":"Pg50fvgW","manual_interface.md":"ClNKLb1M","manual_migrate_from_flux.md":"8SF0xKdw","manual_nested_autodiff.md":"D282O-PO","manual_nn_inside_gpu_kernels.md":"DwtKkctC","manual_performance_pitfalls.md":"D7l0a5P1","manual_preferences.md":"D5tw-ZNe","manual_weight_initializers.md":"CwPu40d0","tutorials_advanced_1_gravitationalwaveform.md":"BfLFXCWx","tutorials_beginner_1_basics.md":"DaAbGE3H","tutorials_beginner_2_polynomialfitting.md":"fx0JNWUd","tutorials_beginner_3_simplernn.md":"CSIho-Mj","tutorials_beginner_4_simplechains.md":"UnuoVtSp","tutorials_beginner_5_optimizationintegration.md":"DkpjLori","tutorials_index.md":"DndwQCq-","tutorials_intermediate_1_neuralode.md":"B1cz3w_g","tutorials_intermediate_2_bayesiannn.md":"e4b93mHN","tutorials_intermediate_3_hypernet.md":"DEpFQTc0","tutorials_intermediate_4_pinn2dpde.md":"C3tygY0O"} +{"api_accelerator_support_mldatadevices.md":"S5wrHgXm","api_building_blocks_luxcore.md":"DMM0tZYm","api_building_blocks_luxlib.md":"BKdoAGCk","api_building_blocks_weightinitializers.md":"BQtmXzUN","api_lux_autodiff.md":"CcfC5vAK","api_lux_contrib.md":"BHxLoP1M","api_lux_distributed_utils.md":"BqeyCLT9","api_lux_interop.md":"DN6M7ehc","api_lux_layers.md":"DEbtIDj-","api_lux_utilities.md":"BeOts_Il","api_testing_functionality_luxtestutils.md":"gICZ2tlR","ecosystem.md":"BQKjQ0KY","index.md":"kmP_n3ia","introduction_citation.md":"D-b8AVke","introduction_index.md":"C7fSJQdT","introduction_overview.md":"CcMCcXqn","introduction_resources.md":"BY58ogcQ","introduction_updating_to_v1.md":"uRcSgSbZ","manual_autodiff.md":"bpTWk2c2","manual_compiling_lux_models.md":"DPjSTTUI","manual_debugging.md":"C1VsyBGc","manual_dispatch_custom_input.md":"Btlkn6MF","manual_distributed_utils.md":"DMnBWmL6","manual_freezing_model_parameters.md":"v9KBWx0y","manual_gpu_management.md":"Pg50fvgW","manual_interface.md":"ClNKLb1M","manual_migrate_from_flux.md":"Dgfpn4GK","manual_nested_autodiff.md":"D282O-PO","manual_nn_inside_gpu_kernels.md":"DwtKkctC","manual_performance_pitfalls.md":"D7l0a5P1","manual_preferences.md":"D5tw-ZNe","manual_weight_initializers.md":"BDecRyMi","tutorials_advanced_1_gravitationalwaveform.md":"D_HX1t4_","tutorials_beginner_1_basics.md":"DaAbGE3H","tutorials_beginner_2_polynomialfitting.md":"fx0JNWUd","tutorials_beginner_3_simplernn.md":"BM5rnJGw","tutorials_beginner_4_simplechains.md":"BHQtKOk4","tutorials_beginner_5_optimizationintegration.md":"Cpu7s8HO","tutorials_index.md":"DndwQCq-","tutorials_intermediate_1_neuralode.md":"d8rfNx5W","tutorials_intermediate_2_bayesiannn.md":"ErflZY0P","tutorials_intermediate_3_hypernet.md":"eD8WCe_f","tutorials_intermediate_4_pinn2dpde.md":"DSWj7UgP"} diff --git a/dev/index.html b/dev/index.html index bc1c624cb8..fcfcf5e049 100644 --- a/dev/index.html +++ b/dev/index.html @@ -5,14 +5,14 @@ Lux.jl Docs - - + + - + - + - + @@ -29,13 +29,13 @@
    Lux.jl Docs

    Appearance

    LuxDL Docs

    Elegant & Performant Scientific Machine Learning in JuliaLang

    A Pure Julia Deep Learning Framework designed for Scientific Machine Learning

    Tutorials
    Ecosystem
    API Reference 📚
    View on GitHub
    Lux.jl
    🚀

    Fast & Extendible

    Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.

    🧑‍🔬

    SciML ❤️ Lux

    Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.

    🧩

    Uniquely Composable

    Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).

    🧪

    Well Tested

    Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.

    How to Install Lux.jl? ​

    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:

    julia
    julia> using Pkg
 julia> Pkg.add("Lux")

    If you want to use the latest unreleased version of Lux.jl, you can run the following command: (in most cases the released version will be same as the version on github)

    julia
    julia> using Pkg
-julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")

    Want GPU Support? ​

    Install the following package(s):

    julia
    using Pkg
+julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")

    Want GPU Support? ​

    Install the following package(s):

    julia
    using Pkg
 Pkg.add("LuxCUDA")
 # or
 Pkg.add(["CUDA", "cuDNN"])
    julia
    using Pkg
 Pkg.add("AMDGPU")
    julia
    using Pkg
 Pkg.add("Metal")
    julia
    using Pkg
-Pkg.add("oneAPI")

    Run the following to access a device:

    julia
    using Lux, LuxCUDA
+Pkg.add("oneAPI")

    Run the following to access a device:

    julia
    using Lux, LuxCUDA
 
 const dev = gpu_device()
    julia
    using Lux, AMDGPU
 
@@ -44,7 +44,7 @@
 const dev = gpu_device()
    julia
    using Lux, oneAPI
 
 const dev = gpu_device()

    Want XLA Support? ​

    Install the following package:

    julia
    using Pkg;
-Pkg.add("Reactant")

    Run the following to access a device:

    julia
    using Reactant, Lux
+Pkg.add("Reactant")

    Run the following to access a device:

    julia
    using Reactant, Lux
 Reactant.set_default_backend("cpu") # default
 
 const dev = xla_device()
    julia
    using Reactant, Lux
@@ -54,7 +54,7 @@
 Reactant.set_default_backend("tpu")
 
 const dev = xla_device()
    - + \ No newline at end of file diff --git a/dev/introduction/citation.html b/dev/introduction/citation.html index fce3bbce72..d19a13f60b 100644 --- a/dev/introduction/citation.html +++ b/dev/introduction/citation.html @@ -5,12 +5,12 @@ Citation | Lux.jl Docs - - + + - + - + @@ -43,7 +43,7 @@ year = {2023}, school = {Massachusetts Institute of Technology} }
    - + \ No newline at end of file diff --git a/dev/introduction/index.html b/dev/introduction/index.html index 9e8e417e1a..5d5bd44d5f 100644 --- a/dev/introduction/index.html +++ b/dev/introduction/index.html @@ -5,12 +5,12 @@ Getting Started | Lux.jl Docs - - + + - + - + @@ -138,7 +138,7 @@ Iteration: 0801 Loss: 0.000239130808 Iteration: 0901 Loss: 0.00140763051 Iteration: 1000 Loss: 0.000214067404

    Training with Optimization.jl

    If you are coming from the SciML ecosystem and want to use Optimization.jl, please refer to the Optimization.jl Tutorial.

    Additional Packages ​

    LuxDL hosts various packages that provide additional functionality for Lux.jl. All packages mentioned in this documentation are available via the Julia General Registry.

    You can install all those packages via import Pkg; Pkg.add(<package name>).

    GPU Support ​

    GPU Support for Lux.jl requires loading additional packages:

    - + \ No newline at end of file diff --git a/dev/introduction/overview.html b/dev/introduction/overview.html index e7e4badf32..0663b4c1fb 100644 --- a/dev/introduction/overview.html +++ b/dev/introduction/overview.html @@ -5,12 +5,12 @@ Why we wrote Lux? | Lux.jl Docs - - + + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Why we wrote Lux? ​

    Julia already has quite a few well established Neural Network Frameworks – Flux & KNet. However, certain design elements – Coupled Model and Parameters & Internal Mutations – associated with these frameworks make them less compiler and user friendly. Making changes to address these problems in the respective frameworks would be too disruptive for users. Here comes in Lux: a neural network framework built completely using pure functions to make it both compiler and autodiff friendly.

    Design Principles ​

    • Layers must be immutable – cannot store any parameter/state but rather store the information to construct them

    • Layers are pure functions

    • Layers return a Tuple containing the result and the updated state

    • Given same inputs the outputs must be same – yes this must hold true even for stochastic functions. Randomness must be controlled using rngs passed in the state.

    • Easily extensible

    • Extensive Testing – All layers and features are tested across all supported AD backends across all supported hardware backends.

    Why use Lux over Flux? ​

    • Neural Networks for SciML: For SciML Applications (Neural ODEs, Deep Equilibrium Models) solvers typically expect a monolithic parameter vector. Flux enables this via its destructure mechanism, but destructure comes with various edge cases and limitations. Lux forces users to make an explicit distinction between state variables and parameter variables to avoid these issues. Also, it comes battery-included for distributed training.

    • Sensible display of Custom Layers – Ever wanted to see Pytorch like Network printouts or wondered how to extend the pretty printing of Flux's layers? Lux handles all of that by default.

    • Truly immutable models - No unexpected internal mutations since all layers are implemented as pure functions. All layers are also deterministic given the parameters and state: if a layer is supposed to be stochastic (say Dropout), the state must contain a seed which is then updated after the function call.

    • Easy Parameter Manipulation – By separating parameter data and layer structures, Lux makes implementing WeightNorm, SpectralNorm, etc. downright trivial. Without this separation, it is much harder to pass such parameters around without mutations which AD systems don't like.

    • Wider AD Support – Lux has extensive support for most AD systems in julia, while Flux is mostly tied to Zygote (with some initial support for Enzyme).

    • Small Neural Networks on CPU – Lux is developed for training large neural networks. For smaller architectures, we recommend using SimpleChains.jl or even better use it in conjunction with Lux via ToSimpleChainsAdaptor.

    • Reliability – We have learned from the mistakes of the past with Flux and everything in our core framework is extensively tested, along with downstream CI to ensure that everything works as expected.

    Revising Previous Recommendation about Large Models

    Previously we recommended not using Lux for very large models. But we have been making a lot of head-way with Reactant.jl and it would be worthwhile to test larger models with Lux. See compiling Lux models for more information.

    - + \ No newline at end of file diff --git a/dev/introduction/resources.html b/dev/introduction/resources.html index fc2bb2312f..dac564a63b 100644 --- a/dev/introduction/resources.html +++ b/dev/introduction/resources.html @@ -5,12 +5,12 @@ Resources to Get Started | Lux.jl Docs - - + + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Resources to Get Started ​

    • Go through the Quickstart Example.

    • Read the introductory tutorials on Julia and Lux.

    • Go through the examples sorted based on their complexity in the documentation.

    Have More Questions?

    For usage related questions, please use Github Discussions which allows questions and answers to be indexed. To report bugs use Github Issues or even better send in a Pull Request.

    - + \ No newline at end of file diff --git a/dev/introduction/updating_to_v1.html b/dev/introduction/updating_to_v1.html index fd0d27d017..7a19a8998b 100644 --- a/dev/introduction/updating_to_v1.html +++ b/dev/introduction/updating_to_v1.html @@ -5,12 +5,12 @@ Updating to Lux v1 | Lux.jl Docs - - + + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Updating to Lux v1 ​

    Lux v1 is a Major Release, mostly to signify the stability of the API. In this page, we list out a concrete set of changes that need to be made to your code to update to Lux v1. We also list out some new exciting features that were added as part of this release.

    LuxLib.jl ​

    Breaking Changes ​

    • Old deprecated API with keyword arguments has been removed. See the new docs in LuxLib API for more details.

    • Default for layernorm dims has been changed to exclude the batch dimension.

    New Major Features ​

    • Dense layers now support CUDA backend for Enzyme (starting v1.1). Wider support for other operations with Enzyme + CUDA is being actively worked on.

    LuxCore.jl ​

    Breaking Changes ​

    • AbstractExplicitLayer has been renamed to AbstractLuxLayer.

    • AbstractExplicitContainerLayer behaviour

      • This has been renamed to AbstractLuxContainerLayer.

      • Previously, AbstractExplicitContainerLayer{(:a,)} (i.e. singleton containers) would produce default initial parameters and states without wrapping them in a NamedTuple{(:a,)}. This was inconsistent with non-singleton containers, and was a source of confusion. With v we return (; a = <parameters>) and (; a = <states>) by default. See AbstractLuxWrapperLayer for a replacement of this functionality.

    • inputsize has been removed since it was ambiguous and not used anywhere.

    • Changes to outputsize:

      • Single argument version has been removed. See LuxCore.jl Pull Request 43 for more details on the rationale behind this change.

      • Fallback implementation has been moved to Lux.jl. (i.e. users using Lux shouldn't see a difference, but if Lux.jl isn't loaded, this function has error.)

        • Internally this uses a NilArray that is able to compute sizes without actually running the computation.

    • Functors and Setfield have been made into optional dependencies. Certain LuxCore functionality that rely on these functions, will throw an error if these packages are not loaded.

    New Major Features ​

    • Introduction of AbstractLuxWrapperLayer. This behaves exactly like the old singleton container. For example, the old AbstractExplicitContainerLayer{(:a,)} is equivalent to AbstractLuxWrapperLayer{:a}.

    WeightInitializers.jl ​

    This was a major release to signify the stability of the API. There were no breaking changes. We do support a wider range of RNG types, see Supported RNG Types for more details.

    MLDataDevices.jl ​

    This is the most aggressive change that was made. We renamed the LuxDeviceUtils.jl package to MLDataDevices.jl, to allow for non-Lux packages to use this shared device management abstraction.

    Deprecation of LuxDeviceUtils.jl

    This also marks the deprecation of the LuxDeviceUtils.jl package. We won't be making any updates to that package, including fixing any bugs. All users should switch to MLDataDevices.jl instead.

    Breaking Changes ​

    • Lux(___)Device objects have been renamed to (___)Device. For example, LuxCUDADevice has been renamed to CUDADevice.

    • Lux(___)Adaptor objects have been removed. The corresponding Device objects should be used directly instead.

    New Major Features ​

    • DeviceIterator provides a generalization of CUDA.CuIterator and works for all backends and more data types (using Functors.jl). MLUtils.DataLoader |> gdev now returns a DeviceIterator instead of being a no-op.

    Lux.jl ​

    Breaking Changes (Removed Functionality) ​

    • Direct reexport of NNlib has been removed. We reexport selected functionality from NNlib. Direactly load NNlib if you need to use the other functions.

    • Flattening of Chain layers has been removed, and the corresponding disable_optimizations kwarg has been removed.

    • Some layers overloaded Base.keys, these have been removed. These were mostly un-documented and weren't supposed to be used outside of the Lux.jl package.

    • Training.TrainState construction with rng has been removed.

    • Older versions of Preferences have been removed.

    • disable_stacktrace_truncation! has been removed. From Julia 1.9 onwards, stacktrace truncation is enabled by default.

    • Certain Experimental features were present outside the Lux.Experimental module. These have been removed, use them via Lux.Experimental instead. Run Julia with with depwarn as error and Lux v0.5 to see the deprecations.

    • Lux.Experimental.@layer_map is not longer needed and has been removed. The name of the variable prevents writing generic functions and is no longer pre-pended to the KeyPath. See the docstring of Lux.Experimental.layer_map for more details.

    • allow_fast_activation kwarg has been removed completely. Pass an anonymous function as the activation to prevent internal modivations to the activation function.

    Breaking Changes (Moved Functionality) ​

    • Lux.Experimental.Training has been moved to Lux.Training. We guarantee SemVar on this new module.

    • Lux.cpu and Lux.gpu have been removed. Use cpu_device and gpu_device instead.

    • Experimental.@compact can be directly used via @compact now.

    • Experimental.StatefulLuxLayer has been moved to Lux.StatefulLuxLayer.

    • st_fixed_path kwarg has been removed from Lux.StatefulLuxLayer, instead use it as StatefulLuxLayer{st_fixed_path}(...).

    • Strings as inputs to Lux.Experimental.layer_map and Lux.Experimental.@debug_mode are removed, use Functors.KeyPath instead.

    • CrossCor has been removed. Use Conv(args...; kwargs..., cross_correlation=true) instead.

    Breaking Changes (Changes in Defaults) ​

    • Conv and ConvTranspose use an initialization based on the activation function, taken from Pytorch. Pytorch assumes the activation function is leakyrelu to compute the gain, however, we compute the gain based on the activation function passed in to the layer.

    • Upsample now has an align_corners keyword argument, which defaults to false. Previously this was always true.

    • Dense and Bilinear have updated default initializations to align with the defaults from Pytorch. See the documentation for more details.

    • InstanceNorm now defaults to affine=false instead of affine=true.

    • Embedding now defaults to init_weight=rand32 instead of init_weight=randn32.

    • Recurrent Cells - RNNCell, LSTMCell, and GRUCell now have different default initializations. See the documentation for more details.

    New Features ​

    • InstanceNorm now supports tracking statistics.

    • RNNCell and LSTMCell add bias_ih and bias_hh to the parameters to align with Pytorch. Both are controlled using init_bias and use_bias.

    • ConvTranspose allows flipkernel=true via cross_correlation=true. This makes it efficient for MIOpen.

    • ConvTranspose now has an outpad keyword argument, which is used to increase the size of the output in the desired dimensions.

    • Pooling Layers based on lpnorm have been added – LPPool, GlobalLPPool, and AdaptiveLPPool.

    - + \ No newline at end of file diff --git a/dev/manual/autodiff.html b/dev/manual/autodiff.html index 38d3682eb3..ad0cc3d1f7 100644 --- a/dev/manual/autodiff.html +++ b/dev/manual/autodiff.html @@ -5,12 +5,12 @@ Automatic Differentiation | Lux.jl Docs - - + + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Automatic Differentiation ​

    Lux is not an AD package, but it composes well with most of the AD packages available in the Julia ecosystem. This document lists the current level of support for various AD packages in Lux. Additionally, we provide some convenience functions for working with AD.

    Overview ​

    AD PackageModeCPUGPUNested 2nd Order ADSupport Class
    ChainRules.jl[1]Reverse✔️✔️✔️Tier I
    Enzyme.jlReverse✔️❓[2]❓[2:1]Tier I[3]
    Zygote.jlReverse✔️✔️✔️Tier I
    ForwardDiff.jlForward✔️✔️✔️Tier I
    ReverseDiff.jlReverse✔️❌❌Tier II
    Tracker.jlReverse✔️✔️❌Tier II
    Mooncake.jlReverse❓[2:2]❌❌Tier III
    Diffractor.jlForward❓[2:3]❓[2:4]❓[2:5]Tier III

    Recommendations ​

    • For CPU Usacases:

      1. Use Zygote.jl for the best performance. This is the most reliable and fastest option for CPU for the time-being. (We are working on faster Enzyme support for CPU)

      2. Use Enzyme.jl, if there are mutations in the code and/or Zygote.jl fails.

      3. If Enzyme.jl fails for some reason, (open an issue and) try ReverseDiff.jl (possibly with compiled mode).

    • For GPU Usacases:

      1. Use Zygote.jl for the best performance. This is the most reliable and fastest option for GPU for the time-being. We are working on supporting Enzyme.jl for GPU as well.

    Support Class ​

    1. Tier I: These packages are fully supported and have been tested extensively. Often have special rules to enhance performance. Issues for these backends take the highest priority.

    2. Tier II: These packages are supported and extensively tested but often don't have the best performance. Issues against these backends are less critical, but we fix them when possible. (Some specific edge cases, especially with AMDGPU, are known to fail here)

    3. Tier III: We don't know if these packages currently work with Lux. We'd love to add tests for these backends, but currently these are not our priority.

    Footnotes ​

    1. Note that ChainRules.jl is not really an AD package, but we have first-class support for packages that use rrules. ↩︎

    2. This feature is supported downstream, but we don't extensively test it to ensure that it works with Lux. ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

    3. Currently Enzyme outperforms other AD packages in terms of CPU performance. However, there are some edge cases where it might not work with Lux. We are working on improving the compatibility. Please report any issues you encounter. ↩︎

    - + \ No newline at end of file diff --git a/dev/manual/compiling_lux_models.html b/dev/manual/compiling_lux_models.html index 5abb452992..6412e97df6 100644 --- a/dev/manual/compiling_lux_models.html +++ b/dev/manual/compiling_lux_models.html @@ -5,12 +5,12 @@ Compiling Lux Models using Reactant.jl | Lux.jl Docs - - + + - + - + @@ -102,7 +102,7 @@ Iter: [ 800/1000] Loss: 0.03070261 Iter: [ 900/1000] Loss: 0.02143306 Iter: [1000/1000] Loss: 0.01542492 - + \ No newline at end of file diff --git a/dev/manual/debugging.html b/dev/manual/debugging.html index a4780a34a2..7cf55b645b 100644 --- a/dev/manual/debugging.html +++ b/dev/manual/debugging.html @@ -5,14 +5,14 @@ Debugging Lux Models | Lux.jl Docs - - + + - + - + - + @@ -69,7 +69,7 @@ [ Info: Input Type: Matrix{Float32} | Input Structure: (3, 2). [ Info: Running Layer: Dense(1 => 1) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2)! ┌ Error: Layer Dense(1 => 1) failed!! This layer is present at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2). -└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/src/contrib/debug.jl:102 +└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-11/julialang/lux-dot-jl/src/contrib/debug.jl:102 DimensionMismatch("A has shape (1, 1) but B has shape (3, 2)")

    See now we know that model.layers.layer_2.layers.layer_2 is the problematic layer. Let us fix that layer and see what happens:

    julia
    model = Chain(Dense(1 => 16, relu),
     Chain(
         Dense(16 => 3),  
@@ -147,7 +147,7 @@
 [ Info: Running Layer: BatchNorm(1, affine=true, track_stats=true) at location KeyPath(:model, :layers, :layer_3)!
 [ Info: Output Type: Matrix{Float32} | Output Structure: (1, 2).
 DomainError(Float32[NaN 0.0], "NaNs detected in pullback output (x)  of layer WrappedFunction(offending_layer) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2).")

    And there you go our debug layer prints that the problem is in WrappedFunction(offending_layer) at location model.layers.layer_2.layers.layer_2! Once we fix the pullback of the layer, we will fix the NaNs.

    Conclusion ​

    In this manual section, we have discussed tracking down errors in Lux models. We have covered tracking incorrect model specifications and NaNs in forward and backward passes. However, remember that this is an Experimental feature, and there might be edge cases that don't work correctly. If you find any such cases, please open an issue on GitHub!

    - + \ No newline at end of file diff --git a/dev/manual/dispatch_custom_input.html b/dev/manual/dispatch_custom_input.html index c4dac5f095..91020961ad 100644 --- a/dev/manual/dispatch_custom_input.html +++ b/dev/manual/dispatch_custom_input.html @@ -5,12 +5,12 @@ Dispatching on Custom Input Types | Lux.jl Docs - - + + - + - + @@ -88,7 +88,7 @@ ps, st = Lux.setup(rng, model) model(xt, ps, st)[1]
    Main.ArrayAndTime{Matrix{Float32}, Float32}(Float32[0.33758628 -0.079208925], 10.0f0)
    - + \ No newline at end of file diff --git a/dev/manual/distributed_utils.html b/dev/manual/distributed_utils.html index 3863f9603f..7ee02a629d 100644 --- a/dev/manual/distributed_utils.html +++ b/dev/manual/distributed_utils.html @@ -5,12 +5,12 @@ Distributed Data Parallel Training | Lux.jl Docs - - + + - + - + @@ -31,7 +31,7 @@ st = DistributedUtils.synchronize!!(backend, st)
    julia
    data = DistributedUtils.DistributedDataContainer(backend, data)
    • Wrap the optimizer in DistributedUtils.DistributedOptimizer to ensure that the optimizer is correctly synchronized across all processes before parameter updates. After initializing the state of the optimizer, synchronize the state across all processes.
    julia
    opt = DistributedUtils.DistributedOptimizer(backend, opt)
 opt_state = Optimisers.setup(opt, ps)
 opt_state = DistributedUtils.synchronize!!(backend, opt_state)
    • Finally change all logging and serialization code to trigger on local_rank(backend) == 0. This ensures that only the master process logs and serializes the model.

    Migration Guide from FluxMPI.jl ​

    Let's compare the changes we need to make wrt the FluxMPI.jl integration guide.

    1. FluxMPI.Init is now DistributedUtils.initialize.

    2. FluxMPI.synchronize!(x) needs to be changed to x_new = DistributedUtils.synchronize!!(backend, x).

    3. DistributedUtils.DistributedDataContainer, DistributedUtils.local_rank, and DistributedUtils.DistributedOptimizer need backend as the first input.

    And that's pretty much it!

    Removed Functionality ​

    1. FluxMPI.allreduce_gradients no longer exists. Previously this was needed when CUDA communication was flaky, with NCCL.jl this is no longer the case.

    2. FluxMPIFluxModel has been removed. DistributedUtils no longer works with Flux.

    Key Differences ​

    1. FluxMPI.synchronize! is now DistributedUtils.synchronize!! to highlight the fact that some of the inputs are not updated in-place.

    2. All of the functions now require a communication backend as input.

    3. We don't automatically determine if the MPI Implementation is CUDA or ROCM aware. See GPU-aware MPI for more information.

    4. Older (now non-existent) Lux.gpu implementations used to "just work" with FluxMPI.jl. We expect gpu_device to continue working as expected, however, we recommend using gpu_device after calling DistributedUtils.initialize to avoid any mismatch between the device set via DistributedUtils and the device stores in CUDADevice or AMDGPUDevice.

    Known Shortcomings ​

    1. Currently we don't run tests with CUDA or ROCM aware MPI, use those features at your own risk. We are working on adding tests for these features.

    2. AMDGPU support is mostly experimental and causes deadlocks in certain situations, this is being investigated. If you have a minimal reproducer for this, please open an issue.

    - + \ No newline at end of file diff --git a/dev/manual/freezing_model_parameters.html b/dev/manual/freezing_model_parameters.html index def82d6bdc..a4d3954b40 100644 --- a/dev/manual/freezing_model_parameters.html +++ b/dev/manual/freezing_model_parameters.html @@ -5,14 +5,14 @@ Freezing Model Parameters | Lux.jl Docs - - + + - + - + - + @@ -46,7 +46,7 @@ model_frozen, ps_frozen, st_frozen = Lux.Experimental.layer_map(freeze_dense, model, ps, st) -model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
+model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
@@ -55,7 +55,7 @@
 function freeze_dense(d::Dense, ps, st, _)
     return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
 end
-freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
+freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight,))
@@ -78,7 +78,7 @@
 x = randn(rng, Float32, 3, 2)
 
 model_frozen(x, ps, st)
    (Float32[0.7429947 -1.2904677], (layer_1 = (layer_1 = NamedTuple(), layer_2 = NamedTuple()), layer_2 = (frozen_params = (layer_3 = NamedTuple(), layer_4 = (scale = Float32[1.0, 1.0, 1.0, 1.0], bias = Float32[0.0, 0.0, 0.0, 0.0])), states = (layer_3 = (rng = Random.TaskLocalRNG(), training = Val{true}()), layer_4 = (running_mean = Float32[0.0, 0.048522998, 0.0, 0.015937408], running_var = Float32[0.9, 0.9470896, 0.9, 0.90508], training = Val{true}()))), layer_3 = NamedTuple()))
    - + \ No newline at end of file diff --git a/dev/manual/gpu_management.html b/dev/manual/gpu_management.html index 28d7e43c56..09e6b0e08f 100644 --- a/dev/manual/gpu_management.html +++ b/dev/manual/gpu_management.html @@ -5,12 +5,12 @@ GPU Management | Lux.jl Docs - - + + - + - + @@ -54,7 +54,7 @@ end (x_gpu |> cdev) ≈ x_cpu
    true
    - + \ No newline at end of file diff --git a/dev/manual/interface.html b/dev/manual/interface.html index 01dbc7557f..9993b800c8 100644 --- a/dev/manual/interface.html +++ b/dev/manual/interface.html @@ -5,12 +5,12 @@ Lux Interface | Lux.jl Docs - - + + - + - + @@ -118,7 +118,7 @@ ps = DenseLayerParameters(ps_default.weight, ps_default.bias) println("Result with `DenseLayerParameters` parameters: ", first(d(x, ps, st)))
    Result with `DenseLayerParameters` parameters: Float32[0.23710957; 0.1003911; -0.57671577;;]

    The takeaway from this shouldn't be – lets define weird parameter types. Simply because you can do weird things like this doesn't mean you should, since it only leads to bugs.

    Instead this shows the flexibility you have for how your parameters can be structured.

    State Interface ​

    States are always type constrained to be NamedTuple. The structure of the input state must match that of the output state, i.e. keys(st_in) == keys(st_out). This doesn't imply that types of the input and output state match. To generate efficient code, we often do dispatch on the state, for example, Dropout, BatchNorm, etc.

    - + \ No newline at end of file diff --git a/dev/manual/migrate_from_flux.html b/dev/manual/migrate_from_flux.html index 3e391f0417..5ea09484dd 100644 --- a/dev/manual/migrate_from_flux.html +++ b/dev/manual/migrate_from_flux.html @@ -5,14 +5,14 @@ Migrating from Flux to Lux | Lux.jl Docs - - + + - + - + - + @@ -27,7 +27,7 @@ -
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Migrating from Flux to Lux ​

    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.

    julia
    using Lux, Random, NNlib, Zygote
+    
    Lux.jl Docs
    Appearance

     Trusted by 
    Scientific Computing
    SciML.ai
    Machine Learning
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
 
 model = Chain(Dense(2 => 4), BatchNorm(4, relu), Dense(4 => 2))
 rng = Random.default_rng()
@@ -47,7 +47,7 @@
 
 model(x)
 
-gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes A×B×x where A is not trainable and B is trainable.
    julia
    using Lux, Random, NNlib, Zygote
+gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes A×B×x where A is not trainable and B is trainable.
    julia
    using Lux, Random, NNlib, Zygote
 
 struct LuxLinear <: Lux.AbstractLuxLayer
     init_A
@@ -85,7 +85,7 @@
 # Needed so that both `A` and `B` can be transferred between devices
 Flux.@functor FluxLinear
 
-(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
+(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
 model = LuxLinear(randn(rng, 2, 4), randn(rng, 4, 2))
 x = randn(rng, 2, 1)
 
@@ -102,7 +102,7 @@
 model(x)
 
 gradient(model -> sum(model(x)), model)
    To reiterate some important points:
    • Don't store mutables like Arrays inside a Lux Layer.
    • Parameters and States should be constructured inside the respective initial* functions.
    Certain Important Implementation Details ​
    Training/Inference Mode ​
    Flux supports a mode called :auto which automatically decides if the user is training the model or running inference. This is the default mode for Flux.BatchNorm, Flux.GroupNorm, Flux.Dropout, etc. Lux doesn't support this mode (specifically to keep code simple and do exactly what the user wants), hence our default mode is training. This can be changed using Lux.testmode.
    Can we still use Flux Layers? ​
    If you have Flux loaded in your code, you can use the function FromFluxAdaptor to automatically convert your model to Lux. Note that in case a native Lux counterpart isn't available, we fallback to using Optimisers.destructure.
    
-    
+    
     
   
 
\ No newline at end of file
diff --git a/dev/manual/nested_autodiff.html b/dev/manual/nested_autodiff.html
index 151aeb6d20..160e9c36d2 100644
--- a/dev/manual/nested_autodiff.html
+++ b/dev/manual/nested_autodiff.html
@@ -5,12 +5,12 @@
     
     Nested Automatic Differentiation | Lux.jl Docs
     
-    
-    
+    
+    
     
-    
+    
     
-    
+    
     
     
     
@@ -146,7 +146,7 @@
 ∞-norm(∂ps using vjp): 0.0
 ∞-norm(∂x using full jacobian): 9.536743e-7
 ∞-norm(∂ps using full jacobian): 1.4305115e-6
    - + \ No newline at end of file diff --git a/dev/manual/nn_inside_gpu_kernels.html b/dev/manual/nn_inside_gpu_kernels.html index 4fd24ac7d6..c179c19414 100644 --- a/dev/manual/nn_inside_gpu_kernels.html +++ b/dev/manual/nn_inside_gpu_kernels.html @@ -5,12 +5,12 @@ Neural Networks Inside GPU Kernels | Lux.jl Docs - - + + - + - + @@ -130,7 +130,7 @@ [2.390200114697191; 0.5267549745189349; -1.657670184695808; -1.7089496198123055;;] [2.1846486482317626; -0.031414255389526885; -1.3279041356366077; -1.6909446526419574;;] [1.3650193059617517; 0.5210742834996898; -0.7689272356710357; -0.6642563709240284;;] - + \ No newline at end of file diff --git a/dev/manual/performance_pitfalls.html b/dev/manual/performance_pitfalls.html index 0c64952033..2d6b8303ec 100644 --- a/dev/manual/performance_pitfalls.html +++ b/dev/manual/performance_pitfalls.html @@ -5,12 +5,12 @@ Performance Pitfalls & How to Catch Them | Lux.jl Docs - - + + - + - + @@ -55,7 +55,7 @@ # do some computation # ... end

    Here, X and y are on the gpu device gdev and the data transfer happens in the worker processes. Additionally, it behaves similar to CuIterator from CUDA.jl and eagerly frees the data after every iteration (this is device agnostic and works on all supported GPU backends).

    - + \ No newline at end of file diff --git a/dev/manual/preferences.html b/dev/manual/preferences.html index c5bc23c3ea..46a3162905 100644 --- a/dev/manual/preferences.html +++ b/dev/manual/preferences.html @@ -5,12 +5,12 @@ Preferences for Lux.jl | Lux.jl Docs - - + + - + - + @@ -30,7 +30,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Preferences for Lux.jl ​

    How to set Preferences

    PreferenceTools.jl provides an interactive way to set preferences. First run the following command:

    julia
    using PreferenceTools

    Then in the pkg mode (press ] in the REPL), run the following command:

    julia
    pkg> preference add Lux <preference-name>=<value>
 pkg> preference add LuxLib <preference-name>=<value>
 pkg> preference add LuxCore <preference-name>=<value>

    Lux.jl relies on several preferences to make decision on how to run your code. Here is an exhaustive list of preferences that Lux.jl uses.

    Nested Automatic Differentiation ​

    1. automatic_nested_ad_switching - Set this to false to disable automatic switching of backends for nested automatic differentiation. See the manual section on nested automatic differentiation for more details.

    GPU-Aware MPI Support ​

    If you are using a custom MPI build that supports CUDA or ROCM, you can use the following preferences with Preferences.jl:

    1. cuda_aware_mpi - Set this to true if your MPI build is CUDA aware.

    2. rocm_aware_mpi - Set this to true if your MPI build is ROCM aware.

    By default, both of these preferences are set to false.

    GPU Backend Selection ​

    1. gpu_backend - Set this to bypass the automatic backend selection and use a specific gpu backend. Valid options are "cuda", "rocm", "metal", and "oneapi". This preference needs to be set for MLDataDevices package. It is recommended to use MLDataDevices.gpu_backend! to set this preference.

    Automatic Eltype Conversion ​

    1. eltype_mismatch_handling - Preference controlling what happens when layers get different eltypes as input. See the documentation on match_eltype for more details.

    Dispatch Doctor ​

    1. instability_check - Preference controlling the dispatch doctor. See the documentation on Lux.set_dispatch_doctor_preferences! for more details. The preferences need to be set for LuxCore and LuxLib packages. Both of them default to disable.

    • Setting the LuxCore preference sets the check at the level of LuxCore.apply. This essentially activates the dispatch doctor for all Lux layers.

    • Setting the LuxLib preference sets the check at the level of functional layer of Lux, for example, fused_dense_bias_activation. These functions are supposed to be type stable for common input types and can be used to guarantee type stability.

    Disabling Loop Vectorization / Octavian ​

    LoopVectorization.jl and Octavian.jl are optional dependencies that are used to accelerate certain CPU operations. However, these packages are tightly coupled with julia and might not work with all julia versions and systems. If these packages are loaded in any form LuxLib will use the optimized versions of the functions. But it might be desirable to disable these packages and use the default implementations instead. This can be done by setting the disable_loop_vectorization preference to true for LuxLib.

    - + \ No newline at end of file diff --git a/dev/manual/weight_initializers.html b/dev/manual/weight_initializers.html index 3477e8bcd2..2323453765 100644 --- a/dev/manual/weight_initializers.html +++ b/dev/manual/weight_initializers.html @@ -5,14 +5,14 @@ Initializing Weights | Lux.jl Docs - - + + - + - + - + @@ -47,17 +47,17 @@ 0.486214 0.321506 -0.306641 0.145296 0.206476

    To generate weights directly on GPU, pass in a CUDA.RNG. For a complete list of supported RNG types, see Supported RNG Types.

    julia
    using LuxCUDA
 
 weights = kaiming_normal(CUDA.default_rng(), 2, 5)
    2×5 CuArray{Float32, 2, CUDA.DeviceMemory}:
- -0.298409  -0.0152843   0.0946174   0.33094   0.573845
-  1.2014    -0.637114   -0.302317   -0.791806  0.184668

    You can also generate Complex Numbers:

    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
- -0.0197287-0.5158im    -0.97285-0.00571707im  …  -0.279012-0.493142im
-  -0.957288+0.343211im   0.51363-0.130897im        0.438734+0.601441im

    Quick examples ​

    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.

    julia
    weights = init(rng, dims...)

    The rng is optional, if not specified a default one will be used.

    julia
    weights = init(dims...)

    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.

    julia
    weights_init = init(rng; kwargs...)
+ -0.262623    0.690434  -0.74887    1.15873   -0.149648
+ -0.0928834  -0.241977   0.162357  -0.505488   0.222534

    You can also generate Complex Numbers:

    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
+ -0.139314+1.11577im  -0.107044+0.256751im  …   0.201611+0.76422im
+  0.900935-1.04194im   0.726115-0.348918im     -0.639297+0.205454im

    Quick examples ​

    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.

    julia
    weights = init(rng, dims...)

    The rng is optional, if not specified a default one will be used.

    julia
    weights = init(dims...)

    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.

    julia
    weights_init = init(rng; kwargs...)
 weights = weights_init(rng, dims...)
 
 # Or
 
 weights_init = init(; kwargs...)
 weights = weights_init(dims...)
    - + \ No newline at end of file diff --git a/dev/tutorials/advanced/1_GravitationalWaveForm.html b/dev/tutorials/advanced/1_GravitationalWaveForm.html index 5cd939f961..7fabf26ff7 100644 --- a/dev/tutorials/advanced/1_GravitationalWaveForm.html +++ b/dev/tutorials/advanced/1_GravitationalWaveForm.html @@ -5,14 +5,14 @@ Training a Neural ODE to Model Gravitational Waveforms | Lux.jl Docs - - + + - + - + - + @@ -184,7 +184,7 @@ Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32), Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32), Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32)) -ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[3.189577f-5; -6.391393f-5; -3.3405635f-5; 4.4343187f-6; 0.000115499526; -0.0001018731; 0.00023788413; 0.00013548152; 6.836329f-5; -7.0168346f-5; -1.1637023f-5; 0.000107990956; 0.00010711853; -8.205676f-5; -3.5767007f-5; 0.0001381862; -4.684011f-5; 0.000164203; -3.8421153f-5; 5.4418593f-5; 9.1661295f-6; 2.394569f-5; -2.2449845f-5; 4.2435263f-6; 0.00014469436; -7.2968054f-5; 8.1639504f-5; -0.00010950737; 0.000111710426; -4.2097818f-5; -0.00023246874; -6.5241285f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-4.072868f-5 -5.0945444f-5 -0.0001259869 -2.931564f-5 0.000166747 7.638867f-5 -5.6004457f-5 -0.00014100932 -2.1586391f-5 -1.2109375f-5 -9.704629f-5 0.00017422912 0.00027027391 8.124638f-5 2.312072f-5 -1.3258273f-5 -6.2884916f-5 -5.023768f-5 -6.0434613f-5 -5.904619f-5 5.3575146f-5 8.3898245f-5 6.905054f-6 -9.50151f-7 -3.3204353f-5 -0.000321201 0.0001560693 6.604745f-5 -4.809239f-5 0.00012601794 -2.430164f-5 7.2888855f-5; 5.081071f-5 0.00017587443 -0.0001901438 -0.00016309925 -1.4872606f-5 0.00010889661 -9.051517f-5 -9.4335606f-5 0.00014208801 -4.8255f-5 -0.00015105841 3.318696f-5 -0.00010077599 0.00012599488 1.5010563f-5 8.972074f-5 -1.3046683f-5 -0.00015159532 0.00011351562 0.0001516642 0.00014350485 -0.00010061943 -0.00016779696 0.00016463436 6.636019f-5 0.00010201224 0.00011553895 -0.00013985121 9.045295f-5 3.6739464f-5 4.8656864f-5 0.000121753; 0.00022025035 -5.518999f-6 -9.600911f-5 0.00013558794 -5.8217807f-5 -7.169174f-5 8.9917674f-5 -0.00010023812 0.00017261699 -3.40887f-5 -0.00019824217 0.0001871318 -8.1081045f-5 3.311501f-5 -0.00010184848 -5.5567038f-5 -6.2686864f-5 -0.00011883248 0.0001333885 -2.6586393f-5 4.5853876f-6 -8.5089574f-5 -2.3479493f-5 9.173001f-6 7.11846f-5 0.000114722425 -1.7309303f-5 -8.865217f-5 5.1577266f-5 -9.788526f-5 5.233557f-5 -7.615518f-5; 0.00020963635 -1.8638202f-5 0.00011834405 -6.729295f-5 -0.00015167797 -7.0826565f-5 3.4370183f-5 -0.00016984323 9.748238f-5 6.600134f-5 -6.2498555f-5 -0.0001225902 -0.000192871 -0.00012371842 -0.00024446193 -3.0057638f-5 0.00016294271 9.313347f-5 -0.00013989632 -1.6167292f-5 -9.897745f-6 -3.6781636f-5 3.8724313f-5 0.000102805214 -6.287945f-5 3.452113f-5 3.6691803f-5 1.649534f-5 -1.0298279f-5 -3.4320224f-6 6.5838605f-5 -4.1367566f-6; 3.1436957f-5 0.00010077432 -0.0001992841 -8.427237f-6 2.0229614f-5 -1.8158896f-7 -4.3141594f-5 -6.0499427f-5 0.0001370642 -8.7448054f-5 -8.16965f-5 1.0385415f-5 -1.1151988f-5 2.0163849f-5 -8.430852f-5 0.00014713164 -1.2928104f-5 4.095135f-5 -0.00015691279 -0.00013806451 5.4008604f-5 -8.881955f-5 7.982801f-5 -6.0442053f-5 0.00010303316 1.1154679f-5 -6.417079f-5 -7.0635576f-7 1.0161936f-5 -3.7932576f-5 -9.273091f-5 6.587933f-5; 0.00015628956 -0.000100306606 0.0001417675 -1.8419723f-5 -0.00010336675 -3.745216f-6 1.3287905f-5 0.000110201225 -0.00012162571 0.00010194623 -0.00021031174 -0.00013408852 -0.00016633335 -0.00011588641 -0.00014500602 0.00023872862 -0.00019082129 0.00020116469 4.6417983f-5 4.0384627f-8 -4.1172363f-5 5.739644f-5 0.0002493159 1.0108878f-6 -6.098719f-6 1.7906605f-5 0.000208925 0.00012901663 5.7656558f-5 5.2600393f-5 -1.7499613f-5 -0.00023694857; 7.6452096f-5 -0.00013919755 5.0454488f-5 -0.00010899347 8.444795f-5 9.3733055f-5 4.9273887f-5 9.575293f-5 -0.00017926394 5.2648138f-5 -0.00015499145 6.0000508f-5 -0.00012347617 -1.0212578f-5 -3.7480448f-5 -2.8889195f-5 -2.5839225f-5 -0.00014959453 -4.838798f-5 -0.00010264639 -2.2373182f-5 0.000187367 -1.4591332f-5 7.587859f-5 6.715484f-5 -8.455441f-6 0.00010300705 9.579943f-5 6.2988795f-5 0.00013727028 5.8433718f-5 -7.581316f-5; 9.867598f-6 0.00013001259 -0.00015904017 0.00010726528 -7.944118f-5 -0.00010838457 -3.0477504f-5 -5.3470885f-5 -0.00017066523 5.8966172f-5 -4.431984f-5 0.00011518578 9.390664f-5 -3.4953282f-5 -2.5257921f-5 3.6932663f-5 5.5787146f-5 8.749019f-5 7.14501f-6 -4.0959952f-5 -6.223889f-5 -8.734941f-5 0.00012043471 -0.00015020274 5.2771924f-5 0.0002360519 -6.228447f-5 6.4803426f-5 0.00010642961 -5.2180483f-5 3.147583f-5 0.00012207228; 0.0001617969 -3.8236438f-5 7.3801225f-6 -6.36821f-5 5.370217f-5 -5.6450346f-5 2.389117f-5 1.3119574f-5 3.8583716f-5 3.039198f-5 0.00014524818 -9.020171f-5 -8.770791f-5 -7.468266f-5 0.00012475614 -1.5612637f-5 2.97265f-5 -5.011583f-5 -2.4989173f-5 6.159547f-5 3.4962607f-6 1.9588915f-6 -9.422292f-5 -9.7407676f-5 -9.5720214f-5 -1.895673f-7 -6.3542095f-5 -8.419821f-5 6.416182f-5 -0.00019523122 6.9585396f-5 -1.4270458f-5; -2.4227842f-5 8.622542f-5 0.00013249219 -0.00016822782 -9.379966f-5 -5.4564578f-5 -7.1605165f-5 -9.777959f-5 -1.5141534f-5 -2.2505346f-5 1.7262204f-5 0.00020515101 -0.00015698072 0.00014193162 -8.208326f-5 0.0001984161 2.1720332f-5 -7.3809155f-5 8.412848f-5 7.0492674f-6 0.00014390076 1.1922104f-5 -0.00014427994 -4.40556f-6 -9.2878225f-5 1.861306f-5 8.723424f-5 0.00021786611 3.6548522f-6 -4.011293f-5 2.7519702f-5 -1.2173145f-5; 2.1144622f-5 0.00014967582 -0.00022895339 0.00024803935 6.106838f-5 7.726797f-5 -9.1495414f-5 -0.00013571442 -1.2743402f-5 3.7127043f-5 5.741774f-6 5.7482393f-6 1.4281251f-5 1.6610835f-5 3.506546f-6 0.00017465782 -3.9547882f-5 -7.454391f-5 -7.768746f-5 -4.087552f-5 -9.2923765f-6 -7.646117f-5 -5.933055f-5 5.02955f-5 3.072864f-5 -5.4951495f-5 -9.214813f-5 4.7591155f-5 -1.3358998f-6 0.00012437423 -4.2005086f-5 -6.518599f-5; 5.2166448f-5 2.3049663f-6 -2.9499244f-5 6.810257f-5 4.4267512f-5 0.00014037451 -9.013224f-6 3.2751344f-5 -6.298081f-6 2.6862037f-5 -0.00011300167 -0.00017712849 0.00022498281 9.065984f-5 -1.6968424f-5 9.56821f-6 9.605599f-6 1.79725f-5 3.0613257f-5 -4.2208558f-5 4.7292133f-6 -4.070843f-6 -0.00016515724 4.4079825f-6 1.49199095f-5 -0.00012453321 1.52478315f-5 -4.9945927f-5 -0.00010899424 5.9470847f-5 4.765956f-5 -0.00019273692; -0.00014856191 4.997596f-5 0.00014711094 7.5645374f-5 -5.5391956f-5 -0.000105570536 8.8837296f-5 7.1964474f-5 -4.419326f-5 -7.311366f-5 4.8815167f-5 -0.0001266497 7.6390796f-5 -0.000102230515 0.00017258363 -2.6130465f-5 -4.5941284f-5 0.00018645074 0.00014535875 -1.6320302f-5 -0.00025578658 -2.491066f-5 -1.5766662f-5 4.3251843f-5 0.00015311797 -0.00011235082 -1.9891374f-6 -0.00019587998 -0.00021126727 8.496006f-5 7.781786f-5 6.961107f-5; 9.386831f-5 0.00015165155 5.2710024f-5 -9.78637f-5 6.203806f-5 0.00011791612 -8.990891f-5 -3.199821f-6 8.466118f-6 -7.408915f-5 -1.6125816f-5 -0.00015418157 0.00010376295 1.5213385f-5 0.000108165565 -5.207923f-5 1.2379622f-5 0.00018003177 -2.9956736f-5 -5.492753f-5 -8.3133375f-5 8.4283674f-5 -5.026297f-5 2.7448626f-5 6.964033f-5 -1.6386199f-5 -9.4f-6 -0.00011314431 0.00013795459 5.831961f-5 -2.2579597f-5 3.1594696f-5; 4.5288183f-5 0.000151896 0.00012069575 -0.0001195369 -7.6831355f-5 -2.757172f-5 -1.6826662f-5 -0.00013169323 6.2199084f-5 -7.3993283f-6 7.438026f-5 -0.00017041463 -7.112954f-5 -2.846346f-5 6.357693f-5 -6.5696972f-6 -4.935456f-5 -9.412396f-6 -5.3623813f-7 -1.5585098f-5 -2.8759412f-5 2.6243073f-5 0.00010650766 -8.971806f-6 -8.316659f-5 -8.4486215f-5 -0.0001286409 1.9484713f-5 -6.270333f-5 6.095372f-5 -0.00020690112 -2.4946847f-5; -9.633735f-5 2.9961959f-5 -8.814054f-5 0.00029132818 -8.873218f-5 -8.915458f-5 2.4145293f-5 -0.00016413987 -0.00013065261 -0.00016233047 -5.391122f-5 -1.16300425f-5 7.1261215f-6 -4.5942532f-5 0.00019776101 -9.504735f-5 -0.00012708372 -9.55583f-5 -4.6812784f-5 -0.00014748046 -4.5817545f-5 -0.00012706364 -3.134673f-5 -5.041207f-5 -9.92932f-5 0.00024251099 7.7417964f-5 -9.606522f-5 -3.9172697f-5 -7.114121f-5 0.00012291323 2.004699f-5; 2.6071615f-5 0.00023533189 4.6388315f-5 -4.4312666f-5 4.156144f-5 1.0031484f-5 0.00021525362 -5.070823f-6 8.614582f-5 -8.926076f-5 -7.542077f-5 7.453099f-6 -3.379711f-5 -0.0001935182 6.443749f-6 -6.77989f-5 0.00024368793 -0.00022429347 -8.256922f-5 0.00023213631 -6.02441f-5 3.6378533f-5 -5.4333955f-6 0.00017070145 -0.00035845037 0.000118600394 -0.00014000213 -2.6946222f-5 4.9569262f-5 7.846937f-5 -0.00016146098 0.00016950133; -7.044077f-5 7.863811f-5 -3.5487312f-5 -3.3633118f-5 0.00018373223 -0.00015216439 -3.533486f-5 3.2348104f-5 -7.659576f-5 0.00013982822 -6.3890446f-5 5.1099894f-5 -3.6681195f-5 9.415396f-5 0.00014207355 3.2957967f-5 -1.5705988f-5 -7.967105f-5 -2.180348f-5 -0.00013320909 -0.0001059336 8.3487f-5 2.282055f-5 5.110875f-5 -3.27335f-5 4.104298f-5 8.464971f-5 -1.7010881f-5 -1.4406375f-5 2.8203609f-5 5.213847f-5 0.0002157011; 0.00019730491 -0.00011324993 6.4773994f-5 0.000120778604 -5.4715092f-5 0.000103474944 -0.00018960478 4.083243f-5 -1.7024828f-5 -7.3320516f-5 -1.5217254f-5 -6.2785366f-5 6.3320535f-5 -4.9529168f-5 -1.9787989f-5 1.2121187f-5 7.928497f-5 -7.016558f-5 -0.00024311207 9.609716f-5 -2.9102543f-5 -3.4667796f-6 -2.930603f-5 5.678993f-5 8.58904f-5 -3.9016104f-5 0.00010542277 -4.1077605f-5 -9.56267f-5 -0.0001131797 -5.798505f-7 -6.970267f-5; -0.00013635393 5.3281066f-5 1.2604041f-5 7.2100185f-5 8.5174026f-5 -4.909403f-5 -2.3897335f-5 -1.1963698f-5 -7.487817f-5 0.00014533561 -5.2880594f-5 0.00010019395 -2.3352486f-5 0.00010543097 4.388293f-5 -5.3105534f-5 3.143798f-5 -6.4436004f-5 1.1926574f-5 -9.614401f-5 -0.00024020273 5.829732f-7 -0.0001988018 -3.7190214f-6 0.00012052626 2.0961197f-6 -6.0311537f-5 -6.3592706f-5 -0.00012088332 -5.7647132f-5 -1.2393728f-5 8.2915765f-5; 6.1345386f-6 -7.6362536f-5 5.3765398f-5 2.2559276f-5 -0.00020616876 -0.00017645433 0.000102468526 -7.835819f-5 -2.275776f-5 1.1462957f-5 -5.1330848f-5 -2.8339622f-5 3.2293367f-5 -5.2020674f-5 1.1928305f-5 -0.00014077095 -5.1057017f-5 -5.2451684f-5 4.570732f-5 -6.1287014f-6 -2.429942f-5 6.568433f-5 9.884206f-5 -0.00016328818 1.984889f-5 -7.699378f-5 2.0554815f-5 -0.00013284909 0.00014174149 4.0874344f-5 8.8765184f-5 0.00012290153; -7.284601f-5 0.00015113266 -8.370722f-5 -0.00010180767 -3.4999375f-5 -0.00018356094 0.00010763813 3.3040036f-5 6.449167f-5 -0.000103754785 0.00010147674 9.896282f-5 -8.177997f-5 1.5517104f-6 0.00017423871 -6.703495f-5 9.930151f-5 -0.00017044914 -4.4303477f-5 4.5368917f-5 -0.000130783 2.013616f-5 6.454598f-5 -0.00020466343 4.2728916f-5 -0.00015525491 9.641593f-6 -8.8821635f-6 0.00013840995 -7.628754f-5 5.2094652f-5 7.693837f-5; -1.936521f-6 -0.00012887 9.0301095f-5 0.00021892734 -5.0441475f-5 3.4005883f-5 0.0001176917 8.137417f-5 -6.481463f-5 3.448867f-5 8.3453546f-7 -0.00012861127 3.8067217f-5 0.00012599569 -0.000100411744 -8.965601f-5 4.6392575f-5 0.00017151685 2.4286144f-5 -4.66038f-5 5.022698f-6 0.00015643479 -4.898879f-5 -3.461263f-5 0.00012742943 6.658684f-5 -6.0342363f-6 -3.1366388f-5 -1.57381f-5 -7.21164f-5 0.00014184127 8.698141f-5; 0.00010477184 -0.00021518409 -0.00020540101 -0.00017861997 -8.4404535f-5 -3.70074f-5 3.9613344f-5 1.9685473f-5 -5.9209102f-5 -0.00015158781 6.52442f-5 6.160428f-5 -9.181076f-5 0.0001256964 0.00015453056 4.0800082f-6 2.2091914f-5 -3.218061f-5 -0.00012707147 -0.000215632 -1.9369683f-5 2.834695f-5 4.242915f-5 4.7958943f-6 -3.186571f-5 6.009155f-5 0.00021139008 4.160474f-5 -2.7288033f-6 4.625632f-5 -4.3435106f-5 7.146487f-5; 4.374089f-5 7.488287f-5 -9.8469856f-5 -0.00012004942 0.0001945894 2.170725f-5 1.7937835f-5 2.7179507f-5 -2.1116573f-5 -5.5853932f-5 3.3326544f-6 -9.3491435f-6 0.00010537235 6.931845f-5 -3.983922f-5 -4.7289675f-5 1.5653004f-5 -0.00022400402 5.7491754f-5 1.715078f-5 2.8108143f-6 5.3561467f-5 -5.169309f-5 -2.5652089f-5 -8.073286f-5 -2.4996185f-5 4.34875f-5 -0.00016427264 1.611109f-5 6.0825427f-5 3.671255f-5 -3.9133418f-5; 0.0002289462 9.411186f-5 -2.7011092f-5 -7.508443f-5 3.0149811f-5 -8.611271f-5 -1.721116f-5 0.00014790527 1.8963052f-5 0.0003108386 -0.00011547375 1.3582856f-5 -4.9441587f-5 -8.9482746f-5 8.650211f-5 -0.00014843841 3.330396f-5 7.7672965f-5 -2.6569027f-5 -5.260992f-5 -5.758783f-5 0.00012981516 8.676786f-5 -0.00012019002 0.00013454087 0.00012962654 -0.000145846 -2.2925251f-5 0.00014683808 0.000111876994 -6.814566f-5 -8.991099f-5; 0.00015247412 -5.2220308f-5 3.303562f-5 -4.0516537f-5 5.0913433f-5 4.4803448f-5 -9.616838f-5 1.784521f-5 7.0508744f-5 0.00018072716 -8.2390274f-5 -0.00016511144 6.84019f-5 -6.5895496f-5 0.00013130245 -0.00010432372 1.1481878f-5 -1.4197938f-5 -2.5276172f-6 1.1729372f-5 3.9101567f-5 5.2090098f-5 5.913563f-5 -2.8327304f-5 -9.641792f-5 0.00017448186 -5.872737f-5 1.536804f-5 7.327565f-5 0.00012235604 7.6064957f-6 1.5156055f-5; -0.000154704 1.4018213f-5 -4.0804447f-5 0.0001416553 4.0145318f-5 3.6607282f-6 8.820955f-5 3.786613f-5 -0.00020226216 4.6105897f-6 5.550528f-6 -0.00010234916 6.716201f-5 -6.043639f-5 -2.6334108f-5 -7.417747f-5 -7.517043f-5 1.1571982f-5 -5.7862057f-5 -3.0656676f-5 -3.4289274f-6 9.702859f-5 0.00014102783 -3.5850444f-5 1.851716f-5 8.838451f-5 -0.00016915894 -5.8839592f-5 0.00017786834 4.4703054f-5 -8.068646f-5 -0.00016312697; 4.442865f-5 -0.00012006202 -2.6558766f-5 5.672547f-5 5.6829463f-6 -6.2722575f-5 -5.5414985f-5 -6.616521f-5 -0.0002716475 -3.1431762f-5 2.5350297f-5 -1.0963497f-5 5.8565315f-6 1.1280099f-5 3.505499f-5 -2.5387439f-5 -4.471935f-5 1.5624903f-5 4.9636583f-5 0.00013870077 -0.00010204634 -1.8144423f-5 6.716244f-5 7.007615f-5 8.5767846f-5 2.7059134f-5 -0.0001835215 -5.475444f-5 -2.0878162f-5 -0.00017755636 -1.3773209f-5 0.00010980106; 5.2602372f-5 -6.791942f-6 -6.0015762f-5 -8.829484f-5 -7.536287f-5 -3.9357703f-5 -0.0002533797 -0.00019349939 -2.6312156f-5 -0.00014407777 5.654903f-6 -8.596633f-5 0.00010207755 -0.00014197214 -0.00021095501 -8.949662f-5 -8.78758f-5 8.230468f-5 5.0060156f-5 1.3623026f-5 8.192343f-5 9.648081f-5 2.901343f-5 -2.8943532f-5 0.00014445429 -0.00014949995 2.8125742f-5 9.118809f-6 3.9546387f-5 0.00011677827 0.00013162084 -9.251267f-5; 0.000111023226 8.540807f-5 -9.632401f-5 -2.0421841f-5 -0.00012313662 -5.2018618f-6 -5.1790317f-5 -0.00016129599 3.4691588f-5 0.00012465993 -5.4743723f-5 -7.375791f-5 -4.2607066f-5 5.131311f-5 0.000105108054 9.257429f-6 7.963778f-5 -0.00022747654 -6.015767f-5 6.143162f-5 0.0001239677 -7.84972f-5 -9.273924f-5 -9.1272304f-5 -5.0631555f-5 1.960298f-5 6.176991f-5 1.42621575f-5 -0.00014048498 -9.409552f-5 -6.69471f-5 -0.00012498979; -9.983786f-5 -9.002071f-5 6.0303675f-5 -4.6285055f-5 -4.9538403f-6 -2.9290824f-5 -5.955555f-5 -0.00017678842 5.0389506f-5 9.9634635f-6 -0.00013858659 7.5097923f-6 -7.146621f-5 -8.4323365f-5 -0.00018398082 4.5715612f-7 0.00019981673 7.600076f-5 -0.000113493494 6.6738976f-5 4.4671888f-5 5.1766674f-6 4.957016f-5 -5.165908f-5 -5.7047462f-5 -0.0001406546 2.3864255f-5 -3.3130655f-5 -3.6680107f-5 1.23412165f-5 0.000109933746 4.6290555f-5], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-9.012873f-5 7.417051f-5 0.00025161545 4.9274546f-5 0.00010122614 -5.0163922f-5 -0.00012842058 -0.0001206786 -2.989958f-5 -8.010971f-5 1.5434154f-5 -0.00016001904 -1.2780457f-5 -2.7766466f-5 7.45467f-5 2.1860842f-5 -0.00012804814 -7.033757f-5 -0.00016812379 -7.759783f-5 3.6243237f-6 9.2822374f-5 -2.3109782f-5 0.00027652478 -4.066539f-5 0.0001177831 2.4505709f-5 -0.00012254887 7.928667f-5 0.00022943318 1.2903607f-5 1.035586f-6; 5.3552827f-5 0.00012438484 0.00010002195 -4.2746266f-5 -8.1811064f-5 -6.5830077f-6 -9.607851f-5 -1.5115101f-5 1.7638582f-5 -0.00012033503 2.8257547f-5 -5.0352584f-5 -1.6083617f-5 0.00010446825 -0.0002548531 4.9123146f-6 -0.00016278515 1.3834254f-5 -0.00015796158 0.00020403395 -3.1378113f-5 -7.720094f-6 -7.166389f-5 -1.7671651f-5 -5.650439f-5 9.3580245f-5 -7.139312f-5 7.729014f-5 -0.00015755462 -2.9734461f-5 -6.472176f-5 -4.8089343f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))

    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64

    julia
    const params = ComponentArray(ps |> f64)
+ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[0.000101433325; -0.00014970472; -3.5847308f-5; -3.0095046f-6; -5.837021f-5; -5.530635f-5; -0.00014913769; -8.2657076f-5; 1.8703817f-6; 2.8848355f-5; 1.4417893f-5; 4.5086595f-5; 4.1238396f-5; 5.863067f-5; 0.000118168115; -2.7185877f-5; -2.2295013f-5; 0.00021102077; -2.7826984f-5; -7.046538f-5; 3.6819965f-5; -0.000122086; 4.216995f-5; 3.7783706f-5; -0.00029299365; 0.00012795332; 2.003964f-5; -7.480524f-5; -1.2783274f-5; 8.586675f-5; -0.00016691082; -0.00014107962;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-5.25561f-5 5.0319617f-5 0.000121188066 -0.0001252668 -0.00012370043 1.3147543f-5 6.7192734f-5 0.00028501067 -4.615136f-5 5.010297f-5 0.0001109413 0.00012366676 0.00012497939 -5.900952f-6 1.5949308f-5 -0.00010800126 -0.00019976492 8.1667444f-5 -0.00020031813 9.338467f-5 -6.742277f-5 4.060099f-5 -8.8431494f-5 -9.7043405f-5 4.6010473f-6 -0.0001405853 5.4320717f-5 8.4588464f-5 -7.09648f-5 -8.0628546f-5 6.95526f-5 -9.106946f-5; 5.6481567f-5 -3.276788f-5 -0.0001785962 -9.946028f-5 -0.00015278214 -6.958734f-5 -8.3435276f-5 6.671795f-5 0.00021108559 3.278153f-5 0.000119060285 5.4152362f-5 6.264283f-5 7.926161f-5 -0.00019341416 -6.553496f-6 7.6160606f-5 -9.66632f-5 -9.5061914f-5 -4.2958072f-5 0.00013052954 -0.00019412015 8.9826804f-5 7.1718045f-5 0.00019083949 6.192604f-6 -3.821931f-6 4.3647802f-7 -0.00014268934 5.3210428f-5 8.25371f-5 -1.2173807f-6; 4.5715973f-5 -5.495184f-5 -0.00011186839 -5.385712f-5 2.0592182f-5 -0.00011375335 -0.00012681524 -5.3366508f-5 8.4260675f-5 -0.000104376966 -9.465132f-5 9.176428f-5 -0.00019846226 4.180067f-5 2.441713f-5 -2.2916829f-5 -3.6965794f-5 8.515369f-5 -4.7670128f-5 -2.4597295f-5 -0.000115552284 5.7741276f-5 -2.149032f-5 0.0001659628 -3.8053244f-5 9.7449265f-5 1.5938956f-5 0.0001071606 0.00015616491 -0.0002005834 -2.687522f-5 1.2993707f-5; 5.1912615f-5 -9.1409915f-5 4.7006324f-5 0.0001274501 0.00010122003 3.093884f-5 4.6041696f-5 -2.819437f-5 7.968095f-6 5.1355753f-5 -9.012398f-5 -0.00016678765 4.6333556f-5 -1.850147f-5 -0.00011283306 2.4248398f-5 -0.0001531043 -0.00020931265 0.00019833563 5.7953574f-5 -8.870693f-5 -9.257044f-5 2.1976448f-5 -3.2718603f-5 -5.585929f-5 7.810437f-5 0.00022372045 3.585174f-5 -2.6321854f-5 9.45166f-5 0.00018568635 -0.000118498785; -8.421658f-5 0.00017808175 0.00015194518 -0.000113057125 -0.00012066419 -0.00018935993 -1.0009757f-5 -6.276838f-5 -2.1114492f-5 -1.8010991f-5 -0.00012293975 0.00010749552 4.4360055f-5 1.2918086f-5 -0.00012972708 1.2805929f-5 0.00014863386 -0.00019670448 -0.00028618306 -8.474048f-5 4.8318067f-5 -0.00013781365 -2.8484217f-5 1.2321886f-5 3.064775f-5 -6.4827924f-5 3.7555023f-6 1.5734184f-5 2.1832733f-5 0.0001064967 -3.4004093f-5 0.00016090772; -4.9504306f-5 1.5704822f-5 9.3366885f-5 6.956209f-5 -5.8176258f-5 -3.641004f-5 1.0974424f-5 -3.3011274f-5 6.968066f-5 -0.00012221691 -2.241614f-5 -0.00021536481 9.290237f-5 -4.6644625f-5 -1.2897161f-5 0.00013195348 -0.00010431149 -0.00011168598 9.9419194f-5 -0.00010479866 -2.6149884f-5 4.048816f-5 -5.4505414f-5 9.3042545f-5 -3.5675401f-7 3.167108f-5 0.00010144765 -2.0471824f-5 7.499174f-5 0.00012745548 -1.9343881f-5 -4.0860472f-5; 3.480437f-5 7.530175f-5 -0.00013266527 0.00024938013 0.00016262576 -8.980506f-5 7.534006f-6 7.878023f-5 4.098693f-5 7.753387f-7 6.413197f-5 -4.2840365f-5 -0.00011310589 4.09235f-5 -4.548707f-5 -4.3824835f-5 -3.7166952f-5 -0.0001460987 -8.8912086f-5 -4.724518f-5 -6.30762f-5 0.0001248795 -2.1304417f-5 0.000109929126 9.500971f-5 -6.1527586f-5 3.875689f-5 -1.0796941f-5 -8.256911f-5 3.3877268f-6 2.0451178f-5 -9.746762f-5; -8.7214146f-5 0.00010691131 -7.3489726f-5 -0.00017095437 1.7527753f-5 8.161695f-5 -8.896779f-5 1.6436641f-5 0.00014055088 3.1515363f-5 -8.762248f-5 -3.935946f-5 -0.00022448925 0.00015336409 1.1825823f-5 0.00012907668 -0.00011167856 0.00010605673 -1.8883887f-6 6.4180625f-5 -0.00012380801 -2.5067176f-5 4.1593237f-5 5.507989f-5 0.00016835274 0.00010800524 -5.2248088f-5 -2.9934452f-5 5.9419897f-5 7.210349f-5 -9.05596f-5 -4.981542f-5; -8.3190884f-5 -0.00013716334 9.333262f-5 3.0901905f-5 0.00018078604 3.6113823f-5 -2.474314f-5 -1.1500239f-5 -2.5257072f-5 -7.240517f-5 -7.065769f-5 -0.0002254977 1.833079f-5 -7.38984f-5 8.5560125f-5 0.0001946904 8.675337f-5 6.741229f-5 -6.8029505f-5 -5.6107692f-5 -0.00012546312 -9.770373f-6 -3.158323f-5 -2.5108455f-5 0.00012979835 5.185795f-5 -3.5403886f-5 0.00020563042 -0.00017649106 9.987908f-5 8.9251225f-5 -6.09851f-5; 0.00015459415 -1.9436993f-5 3.1855652f-5 -2.8804572f-5 0.00010122694 4.723484f-5 1.5064943f-6 5.0922765f-5 1.0278612f-6 -4.2696785f-5 -5.4365966f-5 -0.00020424122 5.340165f-6 -5.1149924f-5 5.5771772f-5 -0.00011463864 -1.7955612f-5 -3.0032486f-5 -0.00012965355 1.8046007f-5 -0.00013679774 2.9808103f-5 4.8935217f-5 -3.587022f-5 -5.076042f-5 -1.201444f-5 -4.7412987f-6 0.00022637536 6.3304215f-5 -1.0715298f-5 -1.32741f-5 -3.9808005f-5; -0.00017152919 8.569154f-5 -0.00015427994 0.0001518926 -7.394991f-5 6.4593505f-6 0.00010508767 0.0001652748 -0.00011657923 8.97161f-6 3.1926393f-5 -0.00014740185 3.5467187f-5 0.00016369366 0.00010648671 0.000103018654 0.00016443468 1.6788868f-5 6.344208f-5 2.0847823f-5 9.915987f-6 0.00013888412 -0.00020764141 0.00014734587 3.127726f-5 0.0001764748 6.793151f-5 -2.079785f-5 -7.642907f-5 0.00010638243 4.9633407f-5 -0.00011511093; 4.7663376f-5 5.5237564f-5 0.00010843301 -5.452355f-5 3.5808926f-6 -3.5294324f-5 2.4117173f-6 0.00012329305 -0.0001269051 -7.7190236f-7 -0.000119915334 -6.727885f-5 0.0002076928 -6.7405424f-5 -4.3333755f-5 5.2977874f-5 -0.00011457315 0.00019208078 -5.5476045f-5 -5.8278398f-5 9.9641446f-5 7.6247874f-5 -3.2882388f-5 2.0014795f-5 5.4410673f-5 2.1790565f-5 6.1696584f-5 8.6672895f-5 0.00012334628 1.748379f-5 0.0001684268 -0.00013439129; 6.154388f-6 -0.000103090264 1.8575416f-5 4.248712f-5 -2.0510362f-5 2.7966196f-5 -4.1805106f-6 0.000100216464 -3.9296287f-5 0.00015753518 -0.00010629292 0.00015053408 5.062221f-6 0.00017994971 2.6507929f-5 6.469896f-5 3.0912197f-5 -3.787202f-5 -9.180992f-5 9.380597f-6 1.7937424f-5 0.0001484272 0.00014581943 -0.00015441813 2.5858413f-5 0.00018289514 5.500523f-5 -1.8279512f-5 0.00018384538 -2.7853945f-5 1.1434443f-5 9.653837f-5; 1.1493156f-5 0.00012901722 -0.00022245743 0.00010968725 -5.9707843f-5 -5.9917904f-5 -3.61098f-5 8.152264f-5 -7.820346f-5 -3.3505003f-5 -3.7833874f-5 0.00011963542 -0.000101400816 -2.5519417f-5 -3.9913408f-5 -1.4419071f-5 1.0952946f-5 3.0841573f-5 -3.0217423f-5 -8.11942f-5 0.00012752488 0.00019579084 -7.06113f-6 0.00011654196 0.00023443296 0.000111647285 0.00014851333 4.3228825f-5 0.00012218115 0.00017196768 -2.9187097f-5 0.00010714686; 2.3869788f-5 -2.8650662f-5 0.00021611001 -1.7511142f-5 2.731428f-5 7.015406f-5 0.00011499756 3.0493577f-5 -1.3791175f-5 -9.098295f-5 -8.075956f-5 6.571922f-5 -4.6558005f-5 3.8361726f-5 -6.263231f-5 9.8436394f-5 -5.444673f-5 -0.0001525929 -2.8079692f-5 0.00020451599 -0.00011710578 8.0096115f-6 1.7739687f-5 -0.00017146657 -1.1216441f-5 5.582888f-5 2.102432f-5 -3.2097083f-5 -5.654088f-5 -0.0002029514 0.00012005623 -4.6535497f-5; 4.0600895f-5 -0.00016371673 -0.00017366491 2.4281542f-5 3.0944266f-5 9.9873614f-5 -2.5847257f-5 0.00020838533 -8.538478f-5 -4.1817697f-5 0.00021643919 1.0913612f-5 -0.00013417867 -0.00011062858 -5.303284f-5 -1.5524723f-6 -2.2601067f-5 3.064764f-5 -0.00012356581 9.590809f-5 0.00014010115 5.2191863f-6 -0.0001268768 -5.8050922f-5 -8.722839f-5 -7.693819f-5 -9.255185f-5 0.00025757347 3.3973716f-5 -0.00011420436 -3.4002467f-5 0.00013258631; 3.1190964f-5 0.00014198378 -0.000109832414 3.0770072f-5 -9.37026f-5 -6.4940985f-5 -6.932041f-6 0.00013815345 0.00014333393 -6.7661975f-5 -4.511986f-5 -2.4029983f-5 -5.3136577f-5 0.00026887178 -0.00015120419 6.6838904f-5 0.00013478157 0.00014922114 2.1898917f-5 -9.330819f-5 2.4437604f-5 -3.936817f-5 -0.00010707723 -1.4017039f-5 6.4641405f-5 4.3191016f-5 7.486379f-5 -9.5469855f-5 -1.4938067f-5 0.00012933458 9.605733f-5 -0.00014525898; 0.00016662048 9.602477f-6 -5.0595074f-5 -0.00010192549 -0.00010132676 -7.178766f-5 -4.139689f-5 6.0371243f-5 -1.2646691f-5 -0.00015618675 -4.537753f-6 0.00019719187 1.20949235f-5 -0.00019831482 -0.00012459549 -3.3654065f-5 1.3041183f-6 8.898287f-5 -4.7517875f-5 -9.4529514f-7 -2.2393571f-5 -7.230482f-5 -5.2415697f-5 0.00015502863 -1.2729995f-5 -3.9127535f-5 -6.104951f-5 -2.6234753f-5 -0.00013506036 -5.3432243f-5 -0.000111437635 8.188779f-5; 9.936778f-5 9.609397f-5 1.8602841f-5 8.174792f-5 3.0406422f-5 5.6035467f-5 2.6022355f-5 8.32676f-5 -0.00011590395 0.0001480582 3.0665713f-5 -8.2545f-5 -3.1364634f-5 3.6824866f-5 6.449255f-5 5.598043f-5 0.00011874976 0.00021320413 -9.1623195f-5 6.0359876f-6 -1.1393765f-5 -0.00022938936 -9.551152f-5 6.278853f-5 0.00015201893 4.6772777f-5 3.4416792f-6 -5.5460823f-5 -3.434725f-5 -0.0001919227 -0.00016547991 -8.913108f-6; -1.0548262f-5 -0.00022290568 6.221811f-5 -3.8161495f-5 -3.203011f-5 5.9161786f-5 2.3230212f-5 3.1728236f-5 8.554471f-5 1.3226349f-5 0.00014533396 7.015475f-5 0.00011469073 8.4968866f-5 0.000119700926 3.688433f-5 0.000105032515 -0.0001410187 4.3452623f-5 -0.00012144251 4.901844f-6 4.2582407f-5 0.0001310712 2.195234f-5 8.650805f-5 4.625665f-5 -7.943035f-5 -7.426819f-5 -9.102839f-5 -4.7004334f-5 6.47525f-5 6.314009f-5; -4.3685573f-5 -7.0061484f-5 -8.564832f-5 -2.5506959f-5 -3.3139113f-5 -0.00012296431 5.0306004f-5 -2.598046f-5 3.1052587f-5 -2.203405f-5 -2.4192013f-5 -1.2408228f-7 0.00018310527 8.597894f-5 -0.00016243967 -8.256807f-5 4.5036827f-6 -4.9110735f-5 -9.529562f-5 0.00011435464 1.8392873f-5 1.6941232f-7 -5.0154013f-5 7.089302f-5 2.9975381f-5 -5.9508693f-5 -2.4033567f-5 0.00012779109 -0.00018340634 -6.473091f-5 -0.00015620314 -8.254381f-5; -2.5668716f-5 4.8003552f-5 -3.2946562f-5 4.2170883f-5 8.5557236f-5 0.0001284341 -5.9376423f-5 -2.011942f-5 -3.5316505f-5 4.182381f-5 -2.0610467f-5 -8.933989f-6 2.7255755f-5 -0.00017342126 2.8045699f-5 2.8669716f-5 -7.505687f-5 -0.00023754647 -0.0002875935 -0.00011733108 -7.4658514f-5 8.5557935f-5 5.5539273f-5 0.00014043871 7.694104f-5 0.0001427624 -7.485815f-5 3.3148644f-5 -9.5938376f-5 -7.424506f-5 8.785261f-5 -4.980164f-5; -9.200019f-5 -0.00024000471 4.1095227f-5 -4.7544443f-5 -8.818522f-6 0.00020778744 7.85882f-6 6.168399f-5 0.0001310395 -2.998988f-5 -2.7495777f-5 0.00013481667 9.5430165f-5 -0.000120056735 -5.0351315f-5 1.976981f-5 -3.72669f-5 4.0843f-6 3.3779917f-5 1.3540122f-5 0.00011324852 3.395571f-5 3.1503678f-5 -0.00010043496 7.797303f-5 0.00017457195 -1.5625114f-5 5.3623786f-5 -0.00012425627 -1.9527579f-5 4.4190932f-5 6.129173f-5; -0.00019794701 0.0002352912 -4.594587f-5 3.4971752f-5 -6.8977897f-6 4.942128f-5 5.3490952f-5 5.9408554f-5 -0.00011715945 2.3265313f-6 0.00010682193 -8.7325025f-5 0.00018497492 -7.528666f-5 6.642244f-5 -8.35431f-6 5.2583666f-5 -1.5171533f-5 -0.00012801825 6.694276f-5 -0.00013729051 -2.2952234f-5 -2.2788714f-5 -6.407778f-5 -0.00012958882 -1.7841387f-5 5.423201f-5 3.9521266f-5 -8.18932f-5 -7.0958435f-5 -1.9372379f-5 5.161007f-5; -7.9233185f-5 -7.5692906f-6 9.44239f-5 -5.664948f-5 -6.476559f-5 -6.323061f-5 -0.00011693359 7.1488954f-5 -4.89746f-5 0.00010430032 3.8002876f-5 -0.00014357506 -3.477876f-5 -1.2791332f-5 3.5682064f-5 0.00010522572 0.00013691728 4.3447893f-5 -0.0001987566 -7.380969f-5 1.4467216f-5 -1.857528f-5 5.0088503f-5 -2.678466f-5 -3.583864f-5 0.000121977566 -0.00011532569 8.403785f-5 -0.00012407296 0.000100155325 -0.00011512785 0.00010114831; 0.00012335247 2.6316393f-6 7.941742f-5 0.00013124838 -1.791447f-5 0.000108683664 1.9190291f-5 3.1741965f-5 4.0499726f-5 2.5934998f-5 0.00022457454 2.7964175f-5 -7.64013f-5 -6.803024f-6 6.500241f-5 1.1759023f-5 -4.710968f-5 -4.3526998f-5 2.3740988f-5 -4.677318f-5 5.2313593f-5 4.8671518f-5 -1.3229032f-5 -5.2332278f-5 -3.818454f-5 -2.7911294f-5 3.0976364f-5 0.000104312705 -1.6617805f-5 1.4596436f-5 -0.00011833158 3.6496947f-6; -2.8291538f-6 1.3292323f-5 0.00013545212 5.787567f-5 -0.00013302402 1.1834459f-5 -0.00031529382 0.00012097406 -1.0305068f-5 -9.146762f-5 -3.9248484f-5 7.469065f-5 6.0470862f-5 -0.0001779905 1.3491451f-5 -7.1607115f-5 -6.8425274f-5 0.00010432296 -4.0927673f-5 9.4371106f-5 -1.8413417f-5 9.450085f-5 6.1815495f-5 0.00010917549 0.00013303825 -0.0001648625 0.00019893328 9.623225f-5 0.000281235 4.8627415f-5 0.00021276702 1.8762299f-5; 0.00020731823 5.1270174f-5 4.631842f-5 -5.7199144f-5 -1.864361f-5 -9.590181f-5 -3.1887441f-6 -6.806665f-5 -5.867507f-5 5.4535365f-5 0.00013211838 2.669618f-5 0.00025670705 0.0001305444 -0.0001418291 4.950635f-5 -6.779437f-5 0.000101174875 1.6520089f-5 -3.5010715f-5 8.299325f-5 -7.4818556f-5 -3.9792998f-5 8.352244f-5 0.00018409271 -5.325691f-5 -2.133156f-5 -0.00020594428 -0.0001694123 -8.223999f-5 -8.586627f-5 7.546048f-5; 3.154289f-5 3.5365934f-5 2.9560859f-5 5.9459628f-5 -0.00012007089 -2.0401096f-5 3.01665f-6 0.00011454203 -2.1529537f-5 9.529565f-5 5.5518474f-5 -6.0801143f-5 0.00011015713 -3.3875676f-5 1.8247714f-5 6.0837523f-5 2.8100952f-5 -6.3356914f-5 -1.7917748f-5 -5.492708f-6 -0.00024452427 0.00015829744 -0.00016907782 -0.000117363714 4.680081f-5 8.013908f-5 -9.65113f-5 -0.00019136128 -0.00011032749 3.5442154f-5 -9.167989f-5 -7.021357f-5; -0.00011151572 -0.00011673683 -7.134072f-5 0.000118420925 -9.90588f-5 -3.7308397f-5 -0.00015447076 1.6448188f-7 -5.2847383f-5 -8.245325f-5 2.159159f-5 5.021692f-5 2.6484282f-5 0.0001992939 -0.000114767594 -1.20055f-5 3.6866324f-5 -2.5806152f-5 8.276754f-6 1.4364273f-5 0.00019493535 3.649333f-5 0.00010192474 9.985735f-5 4.2894528f-5 -0.00013973896 -3.5243203f-5 3.8965212f-5 -5.715798f-5 -4.9403927f-5 0.00021121927 3.575224f-5; 0.00014715974 -2.641252f-5 0.00017523114 -4.8744627f-5 0.00014663808 6.443972f-5 5.1060815f-5 -0.00017406323 7.325399f-5 -3.0334577f-5 -3.290726f-5 -2.8855047f-5 -0.00010346927 0.000109804154 6.330436f-5 0.000105802435 0.00013098773 -9.474001f-6 -8.608316f-6 -7.789192f-5 -3.9351817f-5 -3.7995953f-5 -2.9535186f-5 -4.6726905f-6 -2.3550838f-5 -3.8522256f-5 -0.00021247026 4.007786f-6 0.00018809095 -9.240431f-5 0.00011848084 -8.009428f-5; -0.00012021001 0.00016060712 -0.00013262503 8.1844984f-5 0.000104574036 0.00010485861 -0.00019663687 3.4952027f-5 -0.00017723769 -2.6319058f-5 3.1220454f-5 -9.912223f-5 -0.00012967248 9.97342f-5 -4.808475f-6 -0.0001229508 2.4496549f-5 3.486451f-5 -0.00023266925 8.925509f-5 7.762511f-5 5.531554f-6 3.353919f-5 5.509532f-5 -5.4878186f-5 0.00011217324 -1.2349672f-6 0.00019266526 4.035966f-5 7.287375f-5 9.475787f-5 -6.576237f-5], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-9.368759f-5 1.6582828f-5 -3.5511866f-6 3.0926352f-5 7.026318f-5 -0.00013142922 3.7274265f-5 -6.480119f-5 9.593723f-5 -2.6824922f-5 -6.485373f-5 2.9562175f-6 -9.454155f-5 -6.707946f-5 2.5691726f-5 -0.00011177807 3.0246403f-5 7.450661f-5 -0.00012384634 6.464391f-5 7.969684f-5 -4.669152f-5 7.165037f-6 0.00011992512 4.711118f-5 7.2177645f-5 4.6727633f-5 -4.2947895f-6 -4.7917452f-5 -8.243532f-5 -0.00013366244 -5.2003056f-6; 8.115342f-5 7.3465126f-5 1.5182278f-5 -4.776809f-6 0.00017834424 3.881097f-5 0.00015579326 -4.561653f-5 2.4572842f-5 4.7196598f-5 -2.3822335f-5 -8.144347f-5 -3.6691436f-5 -0.00021805007 -7.189355f-5 9.418967f-5 -4.648693f-5 3.2287953f-5 -3.575809f-5 0.00014702333 -0.00015603313 -5.8873688f-5 -8.486313f-5 7.7675686f-5 2.935245f-5 0.000111844696 -0.00010060676 -1.0563847f-6 -8.139529f-5 -1.7000513f-5 -0.00029249687 -8.839286f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))

    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64

    julia
    const params = ComponentArray(ps |> f64)
 
 const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
     Chain(
@@ -229,13 +229,13 @@
         ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
 
     fig
-end

    Setting Up for Training the Neural Network ​

    Next, we define the objective (loss) function to be minimized when training the neural differential equations.

    julia
    const mseloss = MSELoss()
+end

    Setting Up for Training the Neural Network ​

    Next, we define the objective (loss) function to be minimized when training the neural differential equations.

    julia
    const mseloss = MSELoss()
 
 function loss(θ)
     pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
     pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
     return mseloss(pred_waveform, waveform)
-end
    loss (generic function with 1 method)

    Warmup the loss function

    julia
    loss(params)
    0.0007080899236081684

    Now let us define a callback function to store the loss over time

    julia
    const losses = Float64[]
+end
    loss (generic function with 1 method)

    Warmup the loss function

    julia
    loss(params)
    0.0006994921614618403

    Now let us define a callback function to store the loss over time

    julia
    const losses = Float64[]
 
 function callback(θ, l)
     push!(losses, l)
@@ -247,7 +247,7 @@
 res = Optimization.solve(
     optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
     callback, maxiters=1000)
    retcode: Success
-u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [3.189576818839681e-5; -6.391393253576819e-5; -3.340563489471322e-5; 4.434318725546035e-6; 0.00011549952614571395; -0.00010187309817400318; 0.0002378841309108547; 0.00013548151764520408; 6.836328975620413e-5; -7.016834570084262e-5; -1.1637022907966216e-5; 0.00010799095616670282; 0.00010711853246899926; -8.205675840138618e-5; -3.5767006920644584e-5; 0.00013818619481746985; -4.6840108552708576e-5; 0.0001642029965294685; -3.842115256698744e-5; 5.441859320852504e-5; 9.166129530056265e-6; 2.3945689463266466e-5; -2.244984534627751e-5; 4.243526291244579e-6; 0.00014469436428041194; -7.296805415528246e-5; 8.163950406014773e-5; -0.00010950736759688676; 0.00011171042569901284; -4.209781764071136e-5; -0.0002324687375221788; -6.524128548331127e-5;;], bias = [9.824179453448208e-18, -3.283579809077081e-17, -2.4121052656024447e-17, -2.820816984579966e-18, 1.7800006399111622e-16, -1.2252003989273822e-16, 1.3596170846996523e-16, 3.7697396587690263e-16, 3.469541374929302e-17, -1.2239306697871398e-16, 4.732456804452487e-18, 6.710255596708555e-17, 3.620084831552526e-17, -7.37712055415936e-17, -6.498955501547213e-18, 7.565232283015596e-17, -7.142283934572076e-18, 1.969496943298209e-16, -4.3998375610902485e-17, 1.4517763361935025e-17, 2.2528160954460315e-18, 2.6719434458280483e-17, 5.412172157702465e-18, 3.58740882046328e-18, 3.1671129438060196e-17, -8.253106300035819e-17, 2.757169253238071e-18, -8.791378034823515e-17, 1.3521537011914334e-16, -4.6599032319848886e-17, 4.036589722165776e-17, -4.872073037560773e-17]), layer_3 = (weight = [-4.072823511703495e-5 -5.09450001936559e-5 -0.0001259864585244711 -2.9315197107257145e-5 0.00016674744975718865 7.63891147562606e-5 -5.600401354057221e-5 -0.00014100888082791929 -2.158594733371077e-5 -1.2108931596208436e-5 -9.704584962489625e-5 0.0001742295643544126 0.0002702743585450876 8.124682054587932e-5 2.3121163312703365e-5 -1.3257828950352319e-5 -6.28844719267135e-5 -5.0237235054718365e-5 -6.0434169508857686e-5 -5.904574581106778e-5 5.3575589763591236e-5 8.389868881752165e-5 6.905497602952288e-6 -9.497072391919446e-7 -3.320390905171548e-5 -0.0003212005468321654 0.0001560697491277318 6.604789078667043e-5 -4.8091948056004725e-5 0.00012601838526281895 -2.4301196472136584e-5 7.288929910830549e-5; 5.081267089944231e-5 0.00017587639239744914 -0.0001901418340891797 -0.00016309728881215715 -1.4870646289379445e-5 0.00010889856866223033 -9.05132087680134e-5 -9.43336467641714e-5 0.000142089973898955 -4.825303904700446e-5 -0.0001510564520755974 3.31889191259365e-5 -0.00010077403340593758 0.00012599683622492008 1.5012522901353627e-5 8.972270233451054e-5 -1.3044723807714801e-5 -0.00015159335953988042 0.00011351757938636337 0.0001516661527532875 0.0001435068065732527 -0.00010061746934998852 -0.00016779499629383977 0.00016463631845207847 6.636214902053133e-5 0.00010201420308684551 0.0001155409067835426 -0.0001398492509067325 9.045491105306053e-5 3.674142362096196e-5 4.865882373656356e-5 0.00012175496021181036; 0.00022024982202853847 -5.519529771869915e-6 -9.600964373387555e-5 0.00013558740496225476 -5.821833807569876e-5 -7.169227288980036e-5 8.991714306465219e-5 -0.00010023864857796439 0.00017261645583491674 -3.408923263489361e-5 -0.00019824269808307467 0.00018713126367945243 -8.108157604876883e-5 3.3114480480411414e-5 -0.00010184901444884173 -5.5567568473656066e-5 -6.268739495137848e-5 -0.00011883301309719392 0.00013338797028697872 -2.658692361731209e-5 4.584856775156783e-6 -8.509010482523759e-5 -2.3480024232437663e-5 9.172469923189455e-6 7.118406968945824e-5 0.000114721893929503 -1.7309833914647513e-5 -8.865269743548675e-5 5.1576735437242696e-5 -9.788579306043833e-5 5.233503937777888e-5 -7.615570908822224e-5; 0.00020963441425483624 -1.864013388039402e-5 0.00011834211550667836 -6.72948844519828e-5 -0.00015167990563421714 -7.08284969230501e-5 3.436825141329051e-5 -0.0001698451632771909 9.74804461795482e-5 6.599940757515103e-5 -6.250048676956555e-5 -0.00012259212593998138 -0.0001928729288418158 -0.00012372035047955503 -0.0002444638630047816 -3.005956927589448e-5 0.0001629407816428321 9.313154082994632e-5 -0.0001398982530597972 -1.6169224131586253e-5 -9.899676974598491e-6 -3.678356728850944e-5 3.872238090043803e-5 0.00010280328211156535 -6.28813831506549e-5 3.451919842992198e-5 3.668987124475024e-5 1.649340816148732e-5 -1.0300211165293852e-5 -3.4339540841721874e-6 6.583667350718321e-5 -4.138688332017845e-6; 3.143527677819532e-5 0.00010077263899800934 -0.0001992857793629967 -8.428917371946206e-6 2.0227933728156163e-5 -1.8326939597203966e-7 -4.314327421523914e-5 -6.050110792785467e-5 0.00013706251806068834 -8.744973425565604e-5 -8.169817707313807e-5 1.0383734970663711e-5 -1.1153668001507944e-5 2.0162168166267354e-5 -8.43102022008836e-5 0.000147129955021222 -1.2929784734312002e-5 4.0949668200481244e-5 -0.00015691447302744298 -0.00013806619213939137 5.400692347391857e-5 -8.882123043805916e-5 7.982632845594918e-5 -6.044373336409806e-5 0.00010303148186721462 1.115299832234561e-5 -6.417246882848665e-5 -7.080362002041241e-7 1.0160255751533918e-5 -3.7934256088207734e-5 -9.273258880055931e-5 6.58776483214611e-5; 0.000156290270588567 -0.0001003058989041292 0.00014176820133833353 -1.841901592307562e-5 -0.00010336604298537744 -3.74450853961715e-6 1.328861237231886e-5 0.00011020193216643964 -0.00012162500041050782 0.00010194693624761005 -0.00021031103658934895 -0.00013408781358486153 -0.00016633264583980773 -0.00011588570321656757 -0.00014500531522536515 0.0002387293296309346 -0.0001908205796822708 0.00020116539401022254 4.6418690381312165e-5 4.109205071819074e-8 -4.1171655702431014e-5 5.739714635673128e-5 0.0002493166046591676 1.0115952094448078e-6 -6.098011617376218e-6 1.7907312049270067e-5 0.00020892571212278687 0.00012901733300203338 5.765726547941438e-5 5.2601100669914204e-5 -1.7498905599156158e-5 -0.00023694786630550335; 7.645258701715071e-5 -0.00013919706143595875 5.0454979281233715e-5 -0.00010899297522072281 8.444844242959467e-5 9.373354659957113e-5 4.927437876076855e-5 9.575341791688568e-5 -0.00017926344626274568 5.26486295702061e-5 -0.00015499095352523564 6.000099929125259e-5 -0.0001234756815402654 -1.0212086273065653e-5 -3.7479956155117537e-5 -2.8888703674137614e-5 -2.5838733200622807e-5 -0.00014959404106874985 -4.838748973369222e-5 -0.00010264589545939696 -2.2372690728922117e-5 0.00018736748861457612 -1.4590840329298094e-5 7.58790814860857e-5 6.715533199795584e-5 -8.454949785197743e-6 0.00010300754038227083 9.579991856199895e-5 6.29892860151009e-5 0.00013727077421109564 5.843420913052272e-5 -7.581266581936409e-5; 9.868717612555944e-6 0.00013001370713595114 -0.00015903904680613504 0.0001072664006335285 -7.944006323101522e-5 -0.00010838345154639748 -3.0476384594093793e-5 -5.3469765802004945e-5 -0.0001706641062207913 5.896729159416418e-5 -4.431872115106359e-5 0.00011518689895296008 9.390775700579875e-5 -3.4952162635599276e-5 -2.5256801776556165e-5 3.693378199953781e-5 5.578826564146093e-5 8.749130826457307e-5 7.1461293066412356e-6 -4.0958833081965815e-5 -6.223777130231299e-5 -8.734828700995126e-5 0.000120435832983497 -0.00015020162503220176 5.2773043322517866e-5 0.00023605301863994222 -6.228334790081066e-5 6.480454500546205e-5 0.00010643072779770449 -5.217936380518698e-5 3.147694839410321e-5 0.0001220733963839318; 0.00016179558928054307 -3.8237744104818825e-5 7.378816223378737e-6 -6.36834049720832e-5 5.370086340629576e-5 -5.645165243838028e-5 2.38898644391641e-5 1.3118267576678817e-5 3.85824093762756e-5 3.0390674270919503e-5 0.00014524687479645457 -9.020301901106715e-5 -8.770921364263631e-5 -7.468396894470992e-5 0.0001247548386680384 -1.5613943344910157e-5 2.9725193359652275e-5 -5.011713461786774e-5 -2.4990479017262914e-5 6.159416419616535e-5 3.494954386793892e-6 1.957585242805049e-6 -9.42242297816603e-5 -9.740898191300713e-5 -9.572151999524389e-5 -1.9087357604430672e-7 -6.354340099566823e-5 -8.419951724424006e-5 6.416051086201375e-5 -0.00019523252218566324 6.95840897031777e-5 -1.4271764724227435e-5; -2.422680986988012e-5 8.622644858782885e-5 0.00013249322259448345 -0.0001682267913164207 -9.379862709088791e-5 -5.4563545887372177e-5 -7.160413329608878e-5 -9.77785613539327e-5 -1.5140502142324223e-5 -2.2504314216214893e-5 1.726323611742261e-5 0.00020515204127527292 -0.000156979688795996 0.00014193265710141196 -8.208222530434039e-5 0.00019841713842119285 2.172136450476319e-5 -7.380812272082148e-5 8.412951032746447e-5 7.050299558748259e-6 0.00014390179316628636 1.1923136156973306e-5 -0.00014427890815191073 -4.404527913585472e-6 -9.287719254267265e-5 1.8614092227059548e-5 8.723527466294975e-5 0.00021786714100393905 3.655884342282818e-6 -4.0111896992904666e-5 2.7520733823738473e-5 -1.2172112434066336e-5; 2.1144479453016605e-5 0.00014967567678637593 -0.00022895353047546222 0.00024803920225336914 6.106823550168255e-5 7.726782771156918e-5 -9.149555715169725e-5 -0.00013571456381954395 -1.2743545065074921e-5 3.7126900699024794e-5 5.7416314319765904e-6 5.748096575064684e-6 1.4281108653324745e-5 1.661069202042451e-5 3.50640313796474e-6 0.00017465767725467546 -3.95480246680904e-5 -7.454405293004695e-5 -7.768760131342562e-5 -4.0875661377992795e-5 -9.29251926039576e-6 -7.646131141713119e-5 -5.933069130891232e-5 5.0295358070672964e-5 3.072849633268878e-5 -5.4951638028921765e-5 -9.214827603331037e-5 4.75910119547398e-5 -1.3360425811123943e-6 0.0001243740871558316 -4.200522833174072e-5 -6.518613097626927e-5; 5.216586965387029e-5 2.3043880969255088e-6 -2.9499822396721737e-5 6.810199216258969e-5 4.4266933634826405e-5 0.00014037393362512332 -9.01380257574673e-6 3.275076620182626e-5 -6.2986593934707995e-6 2.6861458770761945e-5 -0.000113002246273502 -0.00017712906605313057 0.00022498223211867276 9.065926520404789e-5 -1.6969002665539284e-5 9.567631657302037e-6 9.605020984488702e-6 1.7971921131432287e-5 3.0612678763170755e-5 -4.2209135920094944e-5 4.7286350831903895e-6 -4.0714210104147525e-6 -0.00016515781657519486 4.407404283008284e-6 1.4919331313177951e-5 -0.0001245337878049409 1.5247253265878265e-5 -4.9946505662337705e-5 -0.00010899481616663255 5.947026898636771e-5 4.765898143658767e-5 -0.00019273749398282397; -0.00014856220460460773 4.997566847050341e-5 0.0001471106521323205 7.564508322429582e-5 -5.5392246768680794e-5 -0.00010557082660925735 8.883700555925537e-5 7.19641835432398e-5 -4.4193551194260185e-5 -7.311395112156149e-5 4.881487674465144e-5 -0.0001266499888613928 7.639050508187e-5 -0.00010223080554242442 0.00017258334318211984 -2.6130755405394208e-5 -4.5941574545216334e-5 0.00018645044527984222 0.00014535845601967287 -1.632059266812866e-5 -0.0002557868714673563 -2.4910950206325143e-5 -1.5766952320339846e-5 4.325155222358853e-5 0.00015311768273859223 -0.00011235110961097501 -1.9894279474126975e-6 -0.000195880274450976 -0.00021126755692494242 8.495977143793123e-5 7.781756775257433e-5 6.961078232888846e-5; 9.387014516933704e-5 0.000151653384923252 5.2711861021368445e-5 -9.786186627506336e-5 6.203989697923951e-5 0.00011791795633571468 -8.99070690914685e-5 -3.1979838107002867e-6 8.467955117070908e-6 -7.408731097284129e-5 -1.6123978535044658e-5 -0.0001541797354414483 0.00010376478541247726 1.5215222551617784e-5 0.0001081674018812073 -5.207739367079272e-5 1.2381458977603318e-5 0.00018003360684332612 -2.995489911453944e-5 -5.492569097013449e-5 -8.313153729003915e-5 8.428551165225825e-5 -5.026113275134092e-5 2.745046378068568e-5 6.964216674049922e-5 -1.6384361411197174e-5 -9.398162442536506e-6 -0.00011314247593445556 0.00013795642381837713 5.8321446080941926e-5 -2.257776021481562e-5 3.1596533621909674e-5; 4.528550362491412e-5 0.00015189332571507132 0.00012069307359096032 -0.00011953957636680179 -7.683403436631394e-5 -2.7574398877464584e-5 -1.6829341070353303e-5 -0.00013169591165451328 6.219640436510303e-5 -7.402007845222216e-6 7.437757977212133e-5 -0.00017041730661249363 -7.11322229060031e-5 -2.846614024266207e-5 6.357425245849798e-5 -6.572376782694359e-6 -4.935723899935839e-5 -9.415075687402303e-6 -5.389176814917797e-7 -1.558777712008384e-5 -2.876209163761416e-5 2.6240393531484048e-5 0.00010650497807117498 -8.974485531067046e-6 -8.316926886863775e-5 -8.448889474922055e-5 -0.0001286435746757973 1.948203299006832e-5 -6.270600915259254e-5 6.095104054592046e-5 -0.00020690380387774357 -2.4949526227566494e-5; -9.634147099579145e-5 2.9957840891336297e-5 -8.814466090999923e-5 0.00029132406089327705 -8.873629812742245e-5 -8.915869657076024e-5 2.4141174802176025e-5 -0.00016414398814676247 -0.00013065672792155194 -0.0001623345884553702 -5.3915336681637446e-5 -1.1634160361519357e-5 7.1220037199041655e-6 -4.594664969491395e-5 0.00019775689034130065 -9.505146553864519e-5 -0.0001270878416079679 -9.55624196620906e-5 -4.681690151930187e-5 -0.00014748457724355295 -4.5821663295026476e-5 -0.00012706775996495492 -3.135084952820556e-5 -5.041618678380332e-5 -9.929731422461133e-5 0.00024250686729200373 7.741384652316938e-5 -9.606933562909097e-5 -3.917681496832297e-5 -7.1145324542991e-5 0.00012290911436426698 2.004287162261593e-5; 2.6072455727442996e-5 0.00023533272563963474 4.638915582927304e-5 -4.43118258435799e-5 4.156228191004157e-5 1.0032324314120804e-5 0.00021525445866880696 -5.069982546962557e-6 8.614665806544275e-5 -8.925991620617663e-5 -7.541993160248347e-5 7.45393957640346e-6 -3.3796269929557296e-5 -0.0001935173529664127 6.444589632336342e-6 -6.779806041141176e-5 0.00024368877005781378 -0.00022429263095819746 -8.25683780594787e-5 0.00023213715415931616 -6.024325899761714e-5 3.637937355688216e-5 -5.432554883556168e-6 0.0001707022859018242 -0.00035844953141623366 0.00011860123435053287 -0.00014000128773023168 -2.6945381035152222e-5 4.957010263478429e-5 7.847020991630263e-5 -0.00016146013855451884 0.00016950217490102439; -7.043909705477776e-5 7.863978838779784e-5 -3.54856367828419e-5 -3.363144265837922e-5 0.00018373390934195612 -0.00015216271118495206 -3.53331836429825e-5 3.234977861049552e-5 -7.659408425913405e-5 0.00013982989343114906 -6.388877069345636e-5 5.1101569386649514e-5 -3.6679519563928514e-5 9.415563703280549e-5 0.0001420752248470576 3.295964210175835e-5 -1.5704312556200466e-5 -7.966937688414516e-5 -2.1801805143487045e-5 -0.0001332074155123078 -0.00010593192406465257 8.348867571678961e-5 2.2822224980909573e-5 5.1110424227072386e-5 -3.2731825069079087e-5 4.104465346083729e-5 8.465138829544799e-5 -1.7009206441595775e-5 -1.4404700106549434e-5 2.8205284014101566e-5 5.2140146852432286e-5 0.0002157027790765166; 0.00019730367283343157 -0.00011325116567307211 6.477275972726577e-5 0.0001207773698145222 -5.471632677356395e-5 0.00010347370924434819 -0.0001896060107507815 4.083119573195054e-5 -1.7026062338780957e-5 -7.332175074837196e-5 -1.5218488912733589e-5 -6.278660068048847e-5 6.331929988234545e-5 -4.953040253597263e-5 -1.9789223658090315e-5 1.2119952301216232e-5 7.928373314213804e-5 -7.016681549131931e-5 -0.00024311330118210217 9.609592460335126e-5 -2.910377726651017e-5 -3.468014271430115e-6 -2.930726397310587e-5 5.678869500413314e-5 8.588916760643641e-5 -3.9017338637430295e-5 0.00010542153402936693 -4.107883934623298e-5 -9.562793776851796e-5 -0.00011318093813018094 -5.810851614175347e-7 -6.970390451599896e-5; -0.00013635585733932896 5.3279143085132605e-5 1.2602118058605339e-5 7.209826193506313e-5 8.517210275390521e-5 -4.90959525086064e-5 -2.389925853996703e-5 -1.1965621215445414e-5 -7.48800909240845e-5 0.00014533368570486814 -5.28825172841227e-5 0.00010019202940388812 -2.3354409367994097e-5 0.00010542904541584576 4.388100691361851e-5 -5.31074571517319e-5 3.143605621520986e-5 -6.44379275944988e-5 1.1924650916664847e-5 -9.614593242072826e-5 -0.0002402046544920797 5.810499283526083e-7 -0.0001988037280716143 -3.7209446508843805e-6 0.00012052433643480654 2.094196443108954e-6 -6.0313460071441674e-5 -6.359462955510926e-5 -0.00012088524332279989 -5.764905532483414e-5 -1.2395651228799042e-5 8.291384196131248e-5; 6.13265146020744e-6 -7.63644228849465e-5 5.3763510631684424e-5 2.2557388628770116e-5 -0.00020617064265970482 -0.00017645621383999068 0.00010246663904548159 -7.836007981531355e-5 -2.27596473545436e-5 1.1461070325456786e-5 -5.13327349021029e-5 -2.834150919661371e-5 3.229147941019859e-5 -5.202256116754194e-5 1.192641783709016e-5 -0.00014077283677615214 -5.1058904237291846e-5 -5.2453570706699625e-5 4.5705431229722175e-5 -6.1305885632492694e-6 -2.430130635767008e-5 6.568244597437593e-5 9.884017080330087e-5 -0.00016329006294691238 1.9847003468029552e-5 -7.699567041563707e-5 2.0552927789685223e-5 -0.00013285097785264679 0.0001417396025222873 4.0872456420034774e-5 8.876329692578255e-5 0.000122899645329742; -7.284701142105474e-5 0.00015113165126769704 -8.370822625345749e-5 -0.00010180867694196524 -3.5000379381177776e-5 -0.00018356194787536639 0.0001076371262514288 3.3039032035799645e-5 6.449066849547358e-5 -0.00010375578868313764 0.00010147573620435553 9.896181281312519e-5 -8.178097060060917e-5 1.5507062187873954e-6 0.00017423770613624525 -6.703595145715941e-5 9.930050864006232e-5 -0.0001704501483856543 -4.4304481076488265e-5 4.536791302036357e-5 -0.00013078399943531718 2.013515576783818e-5 6.454497624310447e-5 -0.00020466443684761645 4.2727911835596774e-5 -0.00015525591161911432 9.640589172541938e-6 -8.883167635009892e-6 0.00013840894088736912 -7.628854664852085e-5 5.2093648167863977e-5 7.693736542371006e-5; -1.9334187523694303e-6 -0.0001288668983303012 9.030419719275371e-5 0.0002189304385414387 -5.043837259072836e-5 3.400898535837695e-5 0.00011769480156576498 8.137727386419124e-5 -6.481152845444233e-5 3.449177061193522e-5 8.376376062284453e-7 -0.00012860816527754854 3.807031939722761e-5 0.00012599879378322225 -0.0001004086417647983 -8.965290648994648e-5 4.6395677464177725e-5 0.0001715199494814266 2.428924640335857e-5 -4.660069785094722e-5 5.025800283338583e-6 0.0001564378934294019 -4.8985687635276715e-5 -3.4609526801058776e-5 0.00012743253578299793 6.658994502723074e-5 -6.0311341709135985e-6 -3.1363285551926876e-5 -1.5734997895927483e-5 -7.211330116623792e-5 0.00014184437427534283 8.698451242744888e-5; 0.00010477013827213785 -0.00021518579044466895 -0.00020540271245962234 -0.00017862167326355638 -8.44062367160201e-5 -3.700910267623787e-5 3.961164227252867e-5 1.968377116456888e-5 -5.9210803307071616e-5 -0.00015158951166639118 6.524250172689278e-5 6.160258023912445e-5 -9.181246307532888e-5 0.00012569470097689044 0.00015452885534060902 4.078306823363043e-6 2.209021229092797e-5 -3.2182312430513194e-5 -0.00012707317248540205 -0.00021563369839539874 -1.9371384702104172e-5 2.834524936944457e-5 4.2427448783123144e-5 4.7941928404855354e-6 -3.1867412622953456e-5 6.008984680549249e-5 0.00021138838317157782 4.1603039129696805e-5 -2.730504753415649e-6 4.625461704009955e-5 -4.343680732386287e-5 7.146316900063252e-5; 4.374030813774188e-5 7.488228711731884e-5 -9.847043711492695e-5 -0.0001200500033519782 0.00019458882288741277 2.170666939487488e-5 1.7937254051470556e-5 2.7178926211450773e-5 -2.111715421055745e-5 -5.5854513084383856e-5 3.332073583762591e-6 -9.349724314275018e-6 0.0001053717727699719 6.931786756463552e-5 -3.983979931949539e-5 -4.729025622513638e-5 1.5652423280655655e-5 -0.00022400460020439108 5.749117317459649e-5 1.7150199164418895e-5 2.8102334908473548e-6 5.35608864083336e-5 -5.1693669411222534e-5 -2.5652669683084067e-5 -8.073344233711706e-5 -2.4996765751026667e-5 4.348691927585022e-5 -0.00016427322279927948 1.6110508477099578e-5 6.082484596841146e-5 3.671196955107326e-5 -3.913399868910976e-5; 0.0002289486765587484 9.411434079108195e-5 -2.7008614742707937e-5 -7.508194899831204e-5 3.0152288709595648e-5 -8.611023436946012e-5 -1.720868200189139e-5 0.00014790775097958878 1.8965529343647317e-5 0.00031084107835364813 -0.00011547126992784686 1.3585333399770132e-5 -4.943910967971433e-5 -8.948026876144895e-5 8.650458406943489e-5 -0.0001484359344974525 3.3306438163226055e-5 7.767544233524609e-5 -2.6566549385943286e-5 -5.2607443612697635e-5 -5.758535186007862e-5 0.00012981763942686558 8.677033842129356e-5 -0.00012018754565336047 0.00013454334479343072 0.00012962901750167164 -0.0001458435253514241 -2.2922773459631004e-5 0.00014684055717249084 0.00011187947169698394 -6.814318108788655e-5 -8.99085097987449e-5; 0.0001524763183963476 -5.221811348996438e-5 3.3037814352846633e-5 -4.0514342765836275e-5 5.091562801430886e-5 4.480564260780046e-5 -9.616618896214645e-5 1.7847404739992356e-5 7.051093874657169e-5 0.00018072935657733683 -8.238807910196516e-5 -0.00016510924884892 6.840409790788863e-5 -6.589330129167569e-5 0.00013130464961910515 -0.00010432152608951083 1.148407230835695e-5 -1.4195743426863102e-5 -2.5254225536087655e-6 1.1731566735086957e-5 3.910376202589805e-5 5.209229224170496e-5 5.913782391622606e-5 -2.8325109235756322e-5 -9.641572520445694e-5 0.00017448405291111741 -5.872517700731323e-5 1.5370234029661434e-5 7.327784718404539e-5 0.0001223582340001114 7.608690372961008e-6 1.515824991407245e-5; -0.00015470543068577935 1.4016783171330845e-5 -4.0805876705333804e-5 0.00014165386899630094 4.014388840086942e-5 3.659298573344575e-6 8.820811861453015e-5 3.786470195418644e-5 -0.0002022635925814998 4.609160060584738e-6 5.549098278507849e-6 -0.00010235058645694412 6.716057767416446e-5 -6.0437818246196376e-5 -2.6335537694747692e-5 -7.417890075803277e-5 -7.5171857969606e-5 1.1570552561265787e-5 -5.7863486216872975e-5 -3.065810589678892e-5 -3.4303570092641735e-6 9.702716131950309e-5 0.00014102640496356923 -3.585187361669241e-5 1.851572968089565e-5 8.838308356725419e-5 -0.0001691603719510617 -5.884102203626996e-5 0.0001778669066706221 4.470162464794341e-5 -8.068789240413099e-5 -0.0001631283993620916; 4.4426369299592315e-5 -0.00012006429676623617 -2.6561045501900908e-5 5.672318877693364e-5 5.680666731479961e-6 -6.272485404699174e-5 -5.5417264207046406e-5 -6.616748794405194e-5 -0.0002716497787107297 -3.143404171644247e-5 2.5348017174812176e-5 -1.0965776141546363e-5 5.8542519807642565e-6 1.127781945322305e-5 3.505270944949432e-5 -2.538971818941064e-5 -4.4721628342109925e-5 1.5622622981749016e-5 4.963430330896957e-5 0.0001386984855569372 -0.00010204861825989456 -1.8146702473339157e-5 6.716015707276895e-5 7.007387250702925e-5 8.576556649567345e-5 2.7056854037051348e-5 -0.0001835237729894497 -5.475672003305463e-5 -2.0880441639035963e-5 -0.00017755863734712595 -1.3775488128143772e-5 0.00010979878209107037; 5.259896126273798e-5 -6.795352858422536e-6 -6.001917289057025e-5 -8.829824808588582e-5 -7.536628116455649e-5 -3.93611139613029e-5 -0.0002533831052302163 -0.00019350280352469694 -2.6315567522647815e-5 -0.00014408118157250598 5.651491862698763e-6 -8.596973792252105e-5 0.00010207414153895346 -0.00014197554854278972 -0.0002109584224696663 -8.950003255691461e-5 -8.787921113496737e-5 8.230126833709747e-5 5.005674531032161e-5 1.3619614507954043e-5 8.192001543404211e-5 9.647740218253537e-5 2.9010019277228988e-5 -2.894694305967406e-5 0.0001444508757388977 -0.00014950335615383206 2.812233062043259e-5 9.115398386105128e-6 3.9542975643902326e-5 0.00011677485737647723 0.00013161742528371577 -9.25160843350352e-5; 0.0001110199960913336 8.540484179145182e-5 -9.63272390271709e-5 -2.0425071290386567e-5 -0.00012313984860331249 -5.205091731042327e-6 -5.17935471924539e-5 -0.00016129921949133171 3.4688357680417196e-5 0.00012465669771524689 -5.474695297555959e-5 -7.376114024911682e-5 -4.2610295515187747e-5 5.130987903437382e-5 0.0001051048244499431 9.254199192663128e-6 7.963454901807268e-5 -0.00022747977289500564 -6.0160898448823084e-5 6.142839321734721e-5 0.00012396446310782835 -7.85004298285759e-5 -9.274246929811238e-5 -9.12755343796869e-5 -5.063478545881434e-5 1.959974962853483e-5 6.1766681590665e-5 1.4258927529299515e-5 -0.00014048820538111552 -9.40987514531405e-5 -6.695032732920575e-5 -0.00012499302045573937; -9.984058251908748e-5 -9.002343122452919e-5 6.030095264244136e-5 -4.628777733493452e-5 -4.956562773853718e-6 -2.9293546871173684e-5 -5.955827273611378e-5 -0.00017679114662646396 5.0386783729061495e-5 9.960741041448202e-6 -0.00013858931324981928 7.507069855223133e-6 -7.146893167705971e-5 -8.432608720218951e-5 -0.00018398354714342134 4.5443363442410884e-7 0.00019981400564241558 7.59980381156852e-5 -0.00011349621618470833 6.673625338984129e-5 4.466916525082814e-5 5.173944915744557e-6 4.9567438142124586e-5 -5.1661803266749204e-5 -5.705018466291802e-5 -0.0001406573295786636 2.3861532641912734e-5 -3.3133377113386027e-5 -3.668282934418554e-5 1.2338493976435376e-5 0.00010993102350438319 4.628783298934402e-5], bias = [4.437515830585584e-10, 1.9594993252322064e-9, -5.308393406822397e-10, -1.9317165401249315e-9, -1.6804369970380884e-9, 7.074235209701761e-10, 4.914899489757273e-10, 1.119375377567678e-9, -1.3062737465727374e-9, 1.0321347406851244e-9, -1.4277078442324252e-10, -5.782044560431486e-10, -2.9050243347228516e-10, 1.8372836237685824e-9, -2.679548484324747e-9, -4.117825409116929e-9, 8.405886536388658e-10, 1.6750511624909356e-9, -1.2346542742130683e-9, -1.923272028122853e-9, -1.887120020134511e-9, -1.0041787219358996e-9, 3.102150377145509e-9, -1.7014154634250893e-9, -5.808033600365818e-10, 2.4775001941826805e-9, 2.1946446036671373e-9, -1.4296521011660889e-9, -2.2795349908595105e-9, -3.4110441278516157e-9, -3.22996305276747e-9, -2.722484775317188e-9]), layer_4 = (weight = [-0.0007867009066537449 -0.0006224015980308057 -0.000444956728781826 -0.0006472975633747372 -0.0005953459865030687 -0.0007467360952999541 -0.0008249927572664053 -0.0008172507531907208 -0.0007264717282625864 -0.0007766818734608853 -0.0006811380285880877 -0.0008565912204912905 -0.0007093526384052144 -0.0007243385789572986 -0.0006220253485736737 -0.000674710999483002 -0.0008246203059352311 -0.0007669096947228696 -0.0008646959428964223 -0.0007741699374504568 -0.0006929477876594768 -0.000603749788953262 -0.0007196817701465205 -0.0004200473523943576 -0.0007372375652553526 -0.0005787889621067207 -0.0006720663781192687 -0.0008191210063249355 -0.0006172854121024927 -0.0004671387862449773 -0.0006836683670708925 -0.000695536447742295; 0.0003091439472017924 0.0003799759317775172 0.00035561306879658736 0.000212844829094219 0.00017378003779430537 0.00024900811024501776 0.00015951260887843164 0.00024047601132138021 0.0002732296908907871 0.00013525608072101174 0.00028384866901007055 0.00020523853473720257 0.00023950750399452675 0.0003600593464662572 7.379666400123996e-7 0.0002605033113944275 9.280597112235319e-5 0.000269425354240891 9.762953174736238e-5 0.00045962503873739547 0.00022421298252437347 0.0002478710205721238 0.00018392716073959746 0.00023791945092090395 0.00019908673084726726 0.00034917132288773224 0.0001841979643931747 0.0003328812460768363 9.803646552512888e-5 0.00022585658063969594 0.00019086928500806669 0.00020750172390734595], bias = [-0.0006965721831938202, 0.00025559112170524283]))

    Visualizing the Results ​

    Let us now plot the loss over time

    julia
    begin
+u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [0.00010143332474399266; -0.00014970471966083986; -3.584730802686131e-5; -3.0095045531156137e-6; -5.837020944455422e-5; -5.530635098688162e-5; -0.00014913768973205503; -8.265707583622401e-5; 1.870381652225233e-6; 2.8848355213978774e-5; 1.4417892998606726e-5; 4.508659549173389e-5; 4.123839607928261e-5; 5.863067053716329e-5; 0.00011816811456800735; -2.7185877115678053e-5; -2.229501296823788e-5; 0.0002110207715302368; -2.7826983568923675e-5; -7.046537939462137e-5; 3.681996531663573e-5; -0.00012208599946444672; 4.2169951484519065e-5; 3.778370592044982e-5; -0.00029299364541612773; 0.00012795331713263893; 2.003964073079977e-5; -7.480524072890363e-5; -1.278327363249883e-5; 8.586674812241179e-5; -0.00016691081691508447; -0.0001410796248819983;;], bias = [7.6713176236549e-19, 3.696847498061422e-17, 2.4055689200670786e-18, -4.74888813316211e-18, 8.733758721019089e-18, -6.464424972716262e-17, 4.685057209353382e-17, -1.141839913481628e-16, -5.842869798050569e-19, 1.8510923759576366e-17, 1.3679823189400309e-18, 6.669020875018921e-18, -4.063228054300107e-18, 8.877328713490479e-17, 1.1229820843349717e-16, -2.356901393761471e-17, -6.315824425403808e-18, 2.5453541817767217e-16, -3.7526064434901786e-17, 3.5326060372288575e-17, 5.257347495428192e-17, -1.9470226826847666e-16, 2.2141221075579763e-17, -3.109513202642436e-18, -5.151280825108199e-16, 1.6121381886129855e-16, 3.3708327975928686e-17, 1.2404157686012208e-17, -2.664045442200898e-17, 1.0050086139984667e-16, -2.8553122257375194e-16, 3.2154322768126957e-17]), layer_3 = (weight = [-5.25564992604587e-5 5.03192185969298e-5 0.00012118766785344962 -0.000125267199157021 -0.00012370083100183914 1.3147144225444096e-5 6.719233528922508e-5 0.00028501027004306206 -4.615175753504133e-5 5.010257332098586e-5 0.00011094090036560689 0.00012366636105791602 0.0001249789874672603 -5.901350552518186e-6 1.5948909956864962e-5 -0.00010800165732878346 -0.00019976532171501866 8.166704533135075e-5 -0.0002003185273243402 9.338426930109939e-5 -6.742316877723749e-5 4.060059104442697e-5 -8.84318924837101e-5 -9.704380322734901e-5 4.6006488882410714e-6 -0.00014058569472770083 5.4320318620669685e-5 8.458806582705095e-5 -7.096519953127111e-5 -8.062894453548825e-5 6.955220482989485e-5 -9.106985640034847e-5; 5.648153107789875e-5 -3.276791524081877e-5 -0.00017859623023947177 -9.946031771074462e-5 -0.0001527821803622454 -6.958737487844602e-5 -8.343531210380995e-5 6.671791213624706e-5 0.000211085549540022 3.278149249557925e-5 0.0001190602489939384 5.4152326060594614e-5 6.264279379569514e-5 7.926157575160787e-5 -0.00019341419813395872 -6.553532171218581e-6 7.616056989240049e-5 -9.666323587703881e-5 -9.506195040126744e-5 -4.2958108520232484e-5 0.0001305295083282009 -0.0001941201843012629 8.982676824847756e-5 7.171800841637475e-5 0.000190839455330881 6.192567684357778e-6 -3.821967152597545e-6 4.3644179731121464e-7 -0.0001426893794134766 5.32103915017989e-5 8.253706635036125e-5 -1.2174169344313458e-6; 4.57142954495314e-5 -5.4953518517909694e-5 -0.00011187006439073063 -5.385879612518597e-5 2.0590504757222437e-5 -0.0001137550249019633 -0.00012681691859838386 -5.336818557921545e-5 8.42589974148187e-5 -0.00010437864384371862 -9.465299499211943e-5 9.176260519083579e-5 -0.0001984639426133598 4.179899351797133e-5 2.4415452028142198e-5 -2.291850669067434e-5 -3.696747134896089e-5 8.515201207261988e-5 -4.76718056212653e-5 -2.4598972775180253e-5 -0.0001155539617663254 5.773959801658752e-5 -2.1491996992752195e-5 0.00016596112564215394 -3.80549214220148e-5 9.74475873611945e-5 1.5937278842412807e-5 0.00010715892440416103 0.00015616323380740252 -0.00020058507343278912 -2.6876896843207e-5 1.2992029407862942e-5; 5.19131751478338e-5 -9.140935461817854e-5 4.700688415124854e-5 0.00012745065733708302 0.00010122059003134992 3.093940150376217e-5 4.6042255880650304e-5 -2.8193810330943946e-5 7.968654890480487e-6 5.1356313369812806e-5 -9.012341642135895e-5 -0.00016678708952002404 4.633411636843179e-5 -1.8500910564565225e-5 -0.00011283250231843832 2.4248958388686553e-5 -0.00015310373445589243 -0.0002093120931842273 0.00019833619418006238 5.7954133544459536e-5 -8.870637274429363e-5 -9.256987713359704e-5 2.1977007881848314e-5 -3.271804261219849e-5 -5.58587310390065e-5 7.810493089874772e-5 0.00022372100621379767 3.5852299069862294e-5 -2.6321293526218726e-5 9.451716296991064e-5 0.0001856869127639691 -0.0001184982250289855; -8.421906372894266e-5 0.0001780792737459982 0.00015194269521963324 -0.0001130596063834579 -0.00012066667100078717 -0.00018936240653637186 -1.0012237771997909e-5 -6.27708576306609e-5 -2.1116973448797237e-5 -1.8013471936127532e-5 -0.00012294222674462564 0.00010749304069541923 4.4357573778699496e-5 1.291560524770234e-5 -0.00012972956014925983 1.2803448180066917e-5 0.00014863137780558722 -0.00019670696187890814 -0.0002861855390168684 -8.474296178099866e-5 4.831558558145079e-5 -0.00013781613220123864 -2.84866981477426e-5 1.2319405095342203e-5 3.064526767928963e-5 -6.483040474483915e-5 3.7530213827364957e-6 1.5731702796634066e-5 2.1830252216760097e-5 0.00010649421906202128 -3.400657428789124e-5 0.0001609052343607146; -4.950440444129138e-5 1.5704723128396553e-5 9.336678675997424e-5 6.956199257790459e-5 -5.817635638716272e-5 -3.641014000943504e-5 1.0974325678648426e-5 -3.3011372860824714e-5 6.968056158318532e-5 -0.0001222170070007688 -2.2416237979383453e-5 -0.00021536490777118708 9.290227507396954e-5 -4.6644723906252673e-5 -1.289725906583554e-5 0.00013195338005123078 -0.00010431158835610957 -0.00011168607698830286 9.941909565047711e-5 -0.0001047987573741247 -2.6149982940593018e-5 4.0488062668986844e-5 -5.450551207952779e-5 9.304244639740679e-5 -3.5685252152966485e-7 3.167098083456447e-5 0.00010144755259777768 -2.0471922024922518e-5 7.499163956770091e-5 0.0001274553830541428 -1.934397942870884e-5 -4.0860570433619975e-5; 3.480445915249443e-5 7.530184260451466e-5 -0.00013266518041143833 0.00024938021991749594 0.00016262584862181272 -8.980496610399982e-5 7.534096345463743e-6 7.878031786878868e-5 4.098702223620674e-5 7.754292474808159e-7 6.413206168565263e-5 -4.28402749670361e-5 -0.00011310579835788056 4.0923590437726185e-5 -4.548698094675063e-5 -4.382474477404435e-5 -3.7166861569291764e-5 -0.00014609860633152372 -8.891199510196862e-5 -4.724508877495954e-5 -6.307611007594211e-5 0.00012487959205520594 -2.1304326276892678e-5 0.00010992921628405631 9.500979971255219e-5 -6.152749525733976e-5 3.875698124522638e-5 -1.0796850313797592e-5 -8.256901898073669e-5 3.3878173204562125e-6 2.0451268516464432e-5 -9.746753172601335e-5; -8.72137428008044e-5 0.00010691171672865347 -7.348932223189744e-5 -0.00017095396861927688 1.7528156321214866e-5 8.161735212402272e-5 -8.896738682901885e-5 1.6437044622444454e-5 0.0001405512845746231 3.151576646692004e-5 -8.762207681806867e-5 -3.9359056528472726e-5 -0.00022448884948184845 0.00015336449331575854 1.1826226626589792e-5 0.00012907708486514048 -0.00011167815347230175 0.00010605713367899186 -1.8879852053473542e-6 6.418102901866489e-5 -0.00012380761137260248 -2.5066772933633293e-5 4.159364078864327e-5 5.508029397665399e-5 0.00016835314062394827 0.00010800564240403152 -5.224768420825822e-5 -2.9934048426502315e-5 5.942030083010798e-5 7.210389202432504e-5 -9.055919900797779e-5 -4.981501507368307e-5; -8.31906623362331e-5 -0.00013716311928972227 9.333284418874019e-5 3.090212652653063e-5 0.0001807862603881634 3.611404412270109e-5 -2.474291846836073e-5 -1.1500018045876406e-5 -2.525685027746876e-5 -7.240495011599202e-5 -7.06574688238783e-5 -0.00022549747933323905 1.8331010723091334e-5 -7.389818034523583e-5 8.556034612313352e-5 0.0001946906153888681 8.675358861994533e-5 6.741251439864033e-5 -6.802928379427108e-5 -5.610747081034315e-5 -0.00012546289923291126 -9.770151847147264e-6 -3.1583008022675615e-5 -2.5108233386230962e-5 0.00012979857587098995 5.185817229987542e-5 -3.540366429758654e-5 0.0002056306433941474 -0.00017649083434760035 9.987929974060993e-5 8.925144669676538e-5 -6.098487989318162e-5; 0.00015459353811902082 -1.9437603677126076e-5 3.18550416894993e-5 -2.880518264565269e-5 0.00010122633169966698 4.723422966062034e-5 1.5058837933777822e-6 5.092215427875536e-5 1.027250724194729e-6 -4.269739589208838e-5 -5.436657671371589e-5 -0.0002042418339070592 5.339554482719717e-6 -5.115053432664885e-5 5.577116166104724e-5 -0.00011463925262351623 -1.7956222344861515e-5 -3.003309689947565e-5 -0.0001296541588627464 1.804539601006992e-5 -0.00013679834888453683 2.98074926294218e-5 4.893460640313953e-5 -3.587083052398726e-5 -5.076103048767231e-5 -1.2015050068933292e-5 -4.7419091598444446e-6 0.00022637474534430537 6.330360405708549e-5 -1.0715908737533902e-5 -1.3274710235328225e-5 -3.9808615164567725e-5; -0.0001715258825111625 8.569484790716378e-5 -0.00015427663882797108 0.00015189590495942777 -7.394660341156399e-5 6.462655359833105e-6 0.00010509097202377471 0.00016527810813790064 -0.00011657592248007039 8.974915285260097e-6 3.19296975380428e-5 -0.00014739854512804722 3.547049137928025e-5 0.00016369696978876612 0.00010649001498171168 0.00010302195888875587 0.0001644379824160282 1.6792173223071718e-5 6.344538425259823e-5 2.0851128023826387e-5 9.919291837329557e-6 0.00013888742406782632 -0.00020763810451676645 0.0001473491735982382 3.128056570344334e-5 0.00017647810610872474 6.793481744757494e-5 -2.07945452398759e-5 -7.642576227731405e-5 0.00010638573595719233 4.96367117324698e-5 -0.00011510762695759204; 4.766572836640417e-5 5.523991656924728e-5 0.00010843535945721786 -5.452119641151299e-5 3.58324503875292e-6 -3.529197184863753e-5 2.414069818099648e-6 0.0001232954042145835 -0.0001269027465375235 -7.69549878640041e-7 -0.00011991298144013656 -6.727649944397793e-5 0.00020769515921099894 -6.740307200263535e-5 -4.3331402971243175e-5 5.2980226051005725e-5 -0.00011457079873682055 0.0001920831369582128 -5.547369202278632e-5 -5.8276045287786293e-5 9.964379804412713e-5 7.625022643448559e-5 -3.288003555528041e-5 2.0017147640174854e-5 5.441302581421965e-5 2.179291785548102e-5 6.169893693848465e-5 8.667524760792516e-5 0.00012334863512049402 1.748614211944747e-5 0.00016842915882558278 -0.0001343889356710264; 6.158470438758418e-6 -0.0001030861814281599 1.8579498698038935e-5 4.249120398230365e-5 -2.0506279855235563e-5 2.7970278876400933e-5 -4.17642799519663e-6 0.00010022054708018434 -3.929220396753885e-5 0.0001575392605961893 -0.00010628883969116538 0.0001505381595568376 5.066303726145864e-6 0.0001799537939011677 2.6512011442468293e-5 6.470304002208984e-5 3.091627955358855e-5 -3.786793890260071e-5 -9.180583505736319e-5 9.384679157608353e-6 1.794150635463917e-5 0.0001484312896143118 0.0001458235136458298 -0.00015441404606759293 2.5862495801101184e-5 0.00018289921795869367 5.500931079654415e-5 -1.8275429413973564e-5 0.00018384945802312768 -2.7849862038479912e-5 1.1438525558973062e-5 9.654245591874819e-5; 1.1497044230215683e-5 0.00012902111012968843 -0.0002224535393759746 0.00010969113901973412 -5.970395539753857e-5 -5.991401593183841e-5 -3.610591118907857e-5 8.152652555161868e-5 -7.81995742580044e-5 -3.350111472519508e-5 -3.7829985725519594e-5 0.00011963930848304569 -0.00010139692784269844 -2.5515529326924823e-5 -3.990951989528911e-5 -1.4415182871570897e-5 1.095683348120538e-5 3.0845460802201694e-5 -3.021353512286594e-5 -8.119031297957508e-5 0.0001275287675673003 0.000195794726838513 -7.057242040728919e-6 0.00011654584779188501 0.00023443684296223387 0.0001116511728616103 0.00014851721317990345 4.32327131567796e-5 0.00012218504236097142 0.00017197156813943473 -2.918320859302846e-5 0.0001071507456946745; 2.38693978535884e-5 -2.865105262750728e-5 0.00021610962220441316 -1.7511532416364836e-5 2.7313888938834663e-5 7.015367235729255e-5 0.00011499716719120773 3.049318592096048e-5 -1.3791565390392404e-5 -9.098333749713619e-5 -8.075994766110806e-5 6.571883067229534e-5 -4.655839576295809e-5 3.8361335544351264e-5 -6.263269962070584e-5 9.843600354363898e-5 -5.444711997549296e-5 -0.0001525932871226935 -2.8080082767673397e-5 0.00020451560202494075 -0.00011710617173955082 8.009220880951566e-6 1.7739296243262974e-5 -0.00017146696110282325 -1.1216831364397432e-5 5.582849052010059e-5 2.1023929082758117e-5 -3.209747366051371e-5 -5.654127154409205e-5 -0.00020295179048113686 0.00012005584220608799 -4.65358875880763e-5; 4.060015640986507e-5 -0.00016371746818351842 -0.0001736656521987529 2.4280803756635753e-5 3.0943527292192e-5 9.98728756368737e-5 -2.5847995800928283e-5 0.0002083845939179537 -8.538551986733346e-5 -4.181843536374034e-5 0.00021643845326449829 1.0912873278510902e-5 -0.0001341794085763753 -0.00011062931656654735 -5.303357824034544e-5 -1.55321077984955e-6 -2.2601805483492784e-5 3.0646901052186956e-5 -0.0001235665512811753 9.590735504586477e-5 0.00014010041564885663 5.218447824674669e-6 -0.00012687753400117012 -5.805166068769852e-5 -8.722913111142489e-5 -7.693892615664247e-5 -9.255258549986294e-5 0.0002575727303903201 3.397297774624188e-5 -0.00011420510048795209 -3.400320559905991e-5 0.0001325855737942889; 3.119263521317027e-5 0.00014198544987980186 -0.00010983074287845628 3.0771742893068196e-5 -9.37009271020907e-5 -6.493931403578232e-5 -6.93036998395775e-6 0.00013815512385723008 0.00014333560567853233 -6.766030390475728e-5 -4.5118190320989156e-5 -2.4028312402278282e-5 -5.313490590391579e-5 0.00026887345040255226 -0.00015120252181272583 6.684057472544877e-5 0.00013478324137200767 0.0001492228158147876 2.1900587960646143e-5 -9.330651926769951e-5 2.443927507361983e-5 -3.9366498533260655e-5 -0.00010707555600869373 -1.4015367816808866e-5 6.46430754548516e-5 4.319268656932053e-5 7.486546120877551e-5 -9.54681844470009e-5 -1.4936395820894294e-5 0.00012933625577521304 9.60590015145911e-5 -0.00014525730774234935; 0.00016661754547165592 9.5995452626037e-6 -5.059800547535577e-5 -0.00010192842397594061 -0.00010132969269982383 -7.179059263073716e-5 -4.1399819895018836e-5 6.03683116765555e-5 -1.264962280622851e-5 -0.0001561896855150079 -4.540684460542714e-6 0.00019718893657916247 1.2091991845930406e-5 -0.0001983177565875168 -0.00012459842342870078 -3.365699682256304e-5 1.301186682732619e-6 8.897994179061968e-5 -4.75208065494625e-5 -9.482267727648984e-7 -2.2396502902144953e-5 -7.230775314602677e-5 -5.241862833138554e-5 0.00015502569352754357 -1.2732927063860792e-5 -3.9130466905054665e-5 -6.105244334542321e-5 -2.6237684653481535e-5 -0.00013506328774604167 -5.343517419753061e-5 -0.00011144056712728296 8.188485720674376e-5; 9.93689685051731e-5 9.609515865262418e-5 1.8604027274294636e-5 8.174910936900174e-5 3.0407607765980987e-5 5.603665368794104e-5 2.602354135118717e-5 8.32687877041703e-5 -0.00011590276670945117 0.00014805938582098773 3.066689924848949e-5 -8.254381072079152e-5 -3.136344793206388e-5 3.682605193857661e-5 6.449373932950347e-5 5.598161470656944e-5 0.00011875094816018567 0.00021320531260383206 -9.162200848409247e-5 6.037173817773901e-6 -1.1392578643109326e-5 -0.00022938817682048194 -9.551033116481576e-5 6.278971733222418e-5 0.00015202011155589635 4.6773963105390396e-5 3.442865419446369e-6 -5.545963694062543e-5 -3.434606304695145e-5 -0.0001919215207530704 -0.00016547872627458 -8.91192183958915e-6; -1.0545969271800749e-5 -0.00022290339151415838 6.222040352394661e-5 -3.815920248644475e-5 -3.202781848859344e-5 5.916407932044661e-5 2.3232504790464034e-5 3.173052860755846e-5 8.55469999437402e-5 1.3228642419102902e-5 0.00014533625759167532 7.015703991737016e-5 0.00011469302367867727 8.497115883027829e-5 0.00011970321898796433 3.6886621418938875e-5 0.00010503480840224687 -0.00014101640303173694 4.3454915757587986e-5 -0.00012144021351606074 4.904137003444741e-6 4.258470031298464e-5 0.00013107348891889498 2.1954633811668815e-5 8.651034400775136e-5 4.625894249282434e-5 -7.94280557242975e-5 -7.426589476427531e-5 -9.102609950833082e-5 -4.70020412108269e-5 6.475479418392951e-5 6.314238376897153e-5; -4.368849663732603e-5 -7.006440687544458e-5 -8.565123976295764e-5 -2.550988215516858e-5 -3.3142036093340545e-5 -0.00012296723257604767 5.030308052229863e-5 -2.598338328679915e-5 3.104966340033266e-5 -2.2036972998469065e-5 -2.4194936534334495e-5 -1.2700544089169337e-7 0.00018310234185919255 8.597602022491649e-5 -0.0001624425913658241 -8.257099221082406e-5 4.500759539220936e-6 -4.911365799922083e-5 -9.529854110296813e-5 0.00011435171649936983 1.8389950080016263e-5 1.664891530819169e-7 -5.015693573368616e-5 7.089009336341235e-5 2.9972457875633954e-5 -5.951161624833396e-5 -2.4036489824359646e-5 0.0001277881692178221 -0.00018340926731325743 -6.473383660049485e-5 -0.00015620606250447454 -8.254673416812025e-5; -2.56702805278856e-5 4.80019880990992e-5 -3.29481262530745e-5 4.216931854108423e-5 8.555567189552139e-5 0.00012843253359398103 -5.9377987716903e-5 -2.0120984053104614e-5 -3.5318069357061005e-5 4.1822244448951675e-5 -2.0612031156531617e-5 -8.935553458910985e-6 2.725419114659701e-5 -0.0001734228207430405 2.8044134588817553e-5 2.8668151636623035e-5 -7.505843671305004e-5 -0.0002375480325360175 -0.0002875950521490153 -0.00011733264539424001 -7.466007803367306e-5 8.555637038745822e-5 5.5537708314034874e-5 0.00014043715061353668 7.69394756128898e-5 0.00014276082954104035 -7.485971575869152e-5 3.314708016779633e-5 -9.59399403006389e-5 -7.424662174224403e-5 8.785104734791074e-5 -4.980320584643474e-5; -9.199854510901844e-5 -0.00024000306265029188 4.109687316864863e-5 -4.7542796970296486e-5 -8.816875457193028e-6 0.0002077890839374628 7.860466134072651e-6 6.168563587758528e-5 0.00013104114797366984 -2.9988234096740654e-5 -2.749413041404722e-5 0.0001348183158499198 9.543181105451772e-5 -0.00012005508877867815 -5.034966860497914e-5 1.977145613106131e-5 -3.726525218174617e-5 4.085946251633898e-6 3.378156353770916e-5 1.3541768488869374e-5 0.00011325016559036582 3.395735431163909e-5 3.150532385385703e-5 -0.00010043331538857763 7.797467871033851e-5 0.00017457359528103169 -1.5623467412146784e-5 5.362543189600014e-5 -0.00012425462599443082 -1.9525932735486006e-5 4.419257849271984e-5 6.129337445068244e-5; -0.00019794775219478724 0.00023529045644837644 -4.594661298169121e-5 3.497101058174528e-5 -6.898531164818111e-6 4.9420538534566074e-5 5.349021082084834e-5 5.9407812632078533e-5 -0.00011716019001168527 2.3257898826220205e-6 0.00010682118660925566 -8.732576609899568e-5 0.00018497418274792557 -7.52874031880243e-5 6.642170116693722e-5 -8.35505145941473e-6 5.258292437220608e-5 -1.5172274497928558e-5 -0.00012801898922615992 6.694201944784054e-5 -0.0001372912513309467 -2.2952975296256756e-5 -2.278945542483876e-5 -6.40785212729435e-5 -0.0001295895628848218 -1.7842128027304344e-5 5.4231267103847656e-5 3.952052499672205e-5 -8.189394359787317e-5 -7.09591761099469e-5 -1.9373120476562817e-5 5.160932848385687e-5; -7.923409045649304e-5 -7.570196224301594e-6 9.442299435870841e-5 -5.665038386102682e-5 -6.4766496300919e-5 -6.323151662688528e-5 -0.00011693449393821139 7.148804818537372e-5 -4.897550731873056e-5 0.00010429941560398894 3.80019703032551e-5 -0.00014357596571532865 -3.4779666542064923e-5 -1.2792237406003304e-5 3.5681158103063064e-5 0.00010522481554910645 0.0001369163767111125 4.344698775352445e-5 -0.00019875751220131186 -7.381059892683991e-5 1.4466310155355671e-5 -1.8576185332829483e-5 5.008759783488835e-5 -2.6785566454393467e-5 -3.5839544373761774e-5 0.00012197666021787196 -0.00011532659461697469 8.403694321425882e-5 -0.00012407386727606787 0.0001001544189665124 -0.00011512875405347932 0.0001011474052819118; 0.0001233550239041024 2.634195995720673e-6 7.941997603407367e-5 0.00013125093676276848 -1.7911913230679333e-5 0.0001086862204166697 1.9192847618920505e-5 3.1744522063627435e-5 4.050228301562713e-5 2.5937554825901e-5 0.00022457709849473415 2.7966732093004386e-5 -7.639874536184391e-5 -6.800467301988405e-6 6.500496522416472e-5 1.1761580142006772e-5 -4.7107122439087137e-5 -4.3524440909899965e-5 2.3743544377026143e-5 -4.6770623951350115e-5 5.231614989239022e-5 4.867407472115042e-5 -1.3226475742017294e-5 -5.2329721141036216e-5 -3.8181981720228886e-5 -2.790873702257031e-5 3.097892033761677e-5 0.00010431526163952301 -1.661524845334837e-5 1.459899260876196e-5 -0.00011832902680259316 3.652251395704443e-6; -2.8260223285790273e-6 1.3295454745602193e-5 0.0001354552542483205 5.787880135302134e-5 -0.00013302088783404502 1.1837590217856658e-5 -0.0003152906837482448 0.00012097718998973196 -1.030193644566294e-5 -9.146448875127744e-5 -3.924535204588754e-5 7.469378314397726e-5 6.0473993552825355e-5 -0.00017798736817951032 1.3494582244101045e-5 -7.160398391589894e-5 -6.842214216759775e-5 0.00010432608823005876 -4.092454119983505e-5 9.437423713223206e-5 -1.8410285438783166e-5 9.450398200839809e-5 6.181862689971448e-5 0.00010917862081385706 0.00013304138255015307 -0.0001648593724081675 0.0001989364128252271 9.623538343419386e-5 0.00028123813089167386 4.863054627037491e-5 0.00021277014738130048 1.8765430549467068e-5; 0.00020731906530684744 5.127101300288545e-5 4.631925818515209e-5 -5.719830463519086e-5 -1.8642770588000753e-5 -9.590096764028852e-5 -3.187904699619065e-6 -6.806580953295295e-5 -5.867422899924916e-5 5.453620448364157e-5 0.0001321192202927102 2.669701913376518e-5 0.000256707887242131 0.00013054524148586346 -0.0001418282672925401 4.950718990997625e-5 -6.779352864711325e-5 0.00010117571488492138 1.6520928479197705e-5 -3.500987535104989e-5 8.299408633732433e-5 -7.481771629997701e-5 -3.979215856403858e-5 8.352328128651912e-5 0.00018409355376616799 -5.325607074653012e-5 -2.1330720333450102e-5 -0.00020594343826402825 -0.0001694114662636514 -8.223914941069638e-5 -8.586542847795008e-5 7.546131652318562e-5; 3.154109318293463e-5 3.536413597195021e-5 2.9559060483186072e-5 5.945782932703545e-5 -0.00012007268496072491 -2.040289409033149e-5 3.0148516611103858e-6 0.00011454023361457142 -2.1531335089642805e-5 9.529384870565706e-5 5.551667591807606e-5 -6.080294160037954e-5 0.00011015533410263506 -3.3877474645670614e-5 1.8245915914490362e-5 6.0835724714158276e-5 2.809915413981493e-5 -6.335871266180674e-5 -1.7919546091982902e-5 -5.4945065063297755e-6 -0.0002445260700302267 0.00015829563809111817 -0.00016907961370871138 -0.00011736551213525923 4.6799012768544025e-5 8.013727957168783e-5 -9.651309782621265e-5 -0.00019136307791322918 -0.00011032929004850627 3.544035526592878e-5 -9.168168734739605e-5 -7.021537139061442e-5; -0.00011151542757454656 -0.0001167365383431639 -7.134042725249435e-5 0.00011842121849019783 -9.905850792586131e-5 -3.730810321451422e-5 -0.00015447046943757196 1.6477537405902708e-7 -5.284708940473613e-5 -8.245295366089095e-5 2.1591883255837038e-5 5.02172139980526e-5 2.6484575487642894e-5 0.00019929420041569243 -0.00011476730041488437 -1.2005206640083375e-5 3.686661757944851e-5 -2.5805858326411852e-5 8.277047259411739e-6 1.4364566327798674e-5 0.00019493563987032264 3.649362288831361e-5 0.00010192503642008395 9.98576458235899e-5 4.289482122236741e-5 -0.0001397386634331515 -3.5242909957219376e-5 3.896550579645388e-5 -5.7157686837513434e-5 -4.940363332446033e-5 0.00021121956770491288 3.5752533863499285e-5; 0.00014716079534976827 -2.641145907561628e-5 0.0001752321965248829 -4.8743566694758174e-5 0.00014663913829266416 6.44407824800849e-5 5.106187531039724e-5 -0.00017406216862597297 7.325505215689046e-5 -3.0333516733809932e-5 -3.290620075873599e-5 -2.885398713275422e-5 -0.00010346820850842252 0.00010980521405130015 6.33054202260911e-5 0.00010580349557116111 0.00013098878629520027 -9.472940619324102e-6 -8.607256277320514e-6 -7.78908600843019e-5 -3.935075652421991e-5 -3.799489273675791e-5 -2.9534125460633956e-5 -4.671630322411254e-6 -2.3549777652255285e-5 -3.852119549280157e-5 -0.00021246919855926692 4.008846317298144e-6 0.00018809201505049565 -9.240325231090934e-5 0.00011848190264556968 -8.009321969454423e-5; -0.00012020967654630444 0.00016060745007611773 -0.00013262470123104556 8.184531804194882e-5 0.00010457436916671864 0.00010485894642092461 -0.00019663653905250167 3.495236057110264e-5 -0.0001772373517577329 -2.6318724061504816e-5 3.12207874832416e-5 -9.912189956035279e-5 -0.0001296721467350404 9.973453312900088e-5 -4.8081411785952726e-6 -0.0001229504715719054 2.4496882239097885e-5 3.486484535291516e-5 -0.0002326689146992544 8.925542248666441e-5 7.762544452466263e-5 5.531887714956414e-6 3.3539522397538664e-5 5.509565277511446e-5 -5.4877852684387734e-5 0.00011217357574982597 -1.2346334950529898e-6 0.00019266559581060106 4.035999415113797e-5 7.287408413153271e-5 9.475820012643837e-5 -6.576203407699251e-5], bias = [-3.984060427337718e-10, -3.6220861999069013e-11, -1.677651797705e-9, 5.59984816741856e-10, -2.480965476587697e-9, -9.850675824529835e-11, 9.052803104856694e-11, 4.035324779008351e-10, 2.2140641303782147e-10, -6.105066009507857e-10, 3.3048440391400475e-9, 2.352485609995227e-9, 4.08259142636652e-9, 3.8879226771262375e-9, -3.9063886060447805e-10, -7.384531561969631e-10, 1.6708489619959403e-9, -2.9316310508200342e-9, 1.1862644435950507e-9, 2.292981203171935e-9, -2.923163995862929e-9, -1.5642660290027388e-9, 1.646107344126006e-9, -7.414476326908213e-10, -9.056504248706273e-10, 2.556705254773653e-9, 3.1314715527015514e-9, 8.394313533953629e-10, -1.798337146897226e-9, 2.934988285790656e-10, 1.060143257221312e-9, 3.336628174683956e-10]), layer_4 = (weight = [-0.0007788534505924469 -0.0006685830389952254 -0.000688716987191034 -0.0006542395072238465 -0.0006149025421776231 -0.00081659508330541 -0.0006478916014476879 -0.000749967054858272 -0.0005892286392886316 -0.0007119907802648645 -0.0007500193367682273 -0.0006822095209065945 -0.0007797070160926858 -0.0007522449748653104 -0.0006594741368423112 -0.0007969439236190279 -0.0006549193992018953 -0.0006106590559730796 -0.0008090121770929926 -0.000620521836429977 -0.0006054688376952504 -0.0007318573278050192 -0.0006780007670647525 -0.000565240734261215 -0.000638054666204854 -0.0006129880701127028 -0.0006384380101355744 -0.0006894606397778792 -0.0007330832427160187 -0.0007676011839588111 -0.0008188282810970212 -0.0006903661698708907; 0.0003245993159796126 0.00031691102393929387 0.00025862815255747214 0.0002386690866586987 0.00042179008933688464 0.00028225686738572416 0.00039923915367428507 0.00019782936787099414 0.00026801874025249095 0.00029064249278658256 0.00021962347088651664 0.00016200238480021362 0.0002067543200442797 2.5395698882187308e-5 0.00017155234894268115 0.0003376355658037225 0.00019695894545278714 0.00027573378158736456 0.00020768779579119 0.0003904691802563027 8.741270119787773e-5 0.00018457219015582057 0.00015858274434813938 0.0003211215798211186 0.00027279834140029273 0.00035529054072876593 0.00014283905766451167 0.00024238950764526578 0.00016205058459968245 0.00022644538420962983 -4.9050983374899115e-5 0.0001550530385393652], bias = [-0.000685165866837486, 0.00024344589819855688]))

    Visualizing the Results ​

    Let us now plot the loss over time

    julia
    begin
     fig = Figure()
     ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
 
@@ -255,7 +255,7 @@
     scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
 
     fig
-end

    Finally let us visualize the results

    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
+end

    Finally let us visualize the results

    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
 soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
 waveform_nn_trained = first(compute_waveform(
     dt_data, soln_nn, mass_ratio, ode_model_params))
@@ -281,7 +281,7 @@
         position=:lb)
 
     fig
-end

    Appendix ​

    julia
    using InteractiveUtils
+end

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -313,7 +313,7 @@
   JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
   JULIA_PKG_PRECOMPILE_AUTO = 0
   JULIA_DEBUG = Literate

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/dev/tutorials/beginner/1_Basics.html b/dev/tutorials/beginner/1_Basics.html index 62a51eb029..8b355dd3b5 100644 --- a/dev/tutorials/beginner/1_Basics.html +++ b/dev/tutorials/beginner/1_Basics.html @@ -5,12 +5,12 @@ Julia & Lux for the Uninitiated | Lux.jl Docs - - + + - + - + @@ -244,7 +244,7 @@ 1 device: 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.627 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/dev/tutorials/beginner/2_PolynomialFitting.html b/dev/tutorials/beginner/2_PolynomialFitting.html index a381321110..11ddda362a 100644 --- a/dev/tutorials/beginner/2_PolynomialFitting.html +++ b/dev/tutorials/beginner/2_PolynomialFitting.html @@ -5,12 +5,12 @@ Fitting a Polynomial using MLP | Lux.jl Docs - - + + - + - + @@ -151,7 +151,7 @@ 1 device: 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.609 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/dev/tutorials/beginner/3_SimpleRNN.html b/dev/tutorials/beginner/3_SimpleRNN.html index 87d311efac..a13707ace9 100644 --- a/dev/tutorials/beginner/3_SimpleRNN.html +++ b/dev/tutorials/beginner/3_SimpleRNN.html @@ -5,14 +5,14 @@ Training a Simple LSTM | Lux.jl Docs - - + + - + - + - + @@ -126,455 +126,455 @@ return (train_state.parameters, train_state.states) |> cpu_device() end -ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.60876
-Epoch [  1]: Loss 0.58947
-Epoch [  1]: Loss 0.56685
-Epoch [  1]: Loss 0.55803
-Epoch [  1]: Loss 0.52538
-Epoch [  1]: Loss 0.49019
-Epoch [  1]: Loss 0.48089
-Validation: Loss 0.47082 Accuracy 1.00000
-Validation: Loss 0.48051 Accuracy 1.00000
-Epoch [  2]: Loss 0.46293
-Epoch [  2]: Loss 0.45149
-Epoch [  2]: Loss 0.44166
-Epoch [  2]: Loss 0.43669
-Epoch [  2]: Loss 0.41220
-Epoch [  2]: Loss 0.39264
-Epoch [  2]: Loss 0.40298
-Validation: Loss 0.37454 Accuracy 1.00000
-Validation: Loss 0.38588 Accuracy 1.00000
-Epoch [  3]: Loss 0.36447
-Epoch [  3]: Loss 0.36221
-Epoch [  3]: Loss 0.33435
-Epoch [  3]: Loss 0.32705
-Epoch [  3]: Loss 0.33662
-Epoch [  3]: Loss 0.32659
-Epoch [  3]: Loss 0.28894
-Validation: Loss 0.29056 Accuracy 1.00000
-Validation: Loss 0.30277 Accuracy 1.00000
-Epoch [  4]: Loss 0.28560
-Epoch [  4]: Loss 0.27737
-Epoch [  4]: Loss 0.27287
-Epoch [  4]: Loss 0.24844
-Epoch [  4]: Loss 0.25384
-Epoch [  4]: Loss 0.23238
-Epoch [  4]: Loss 0.24520
-Validation: Loss 0.22104 Accuracy 1.00000
-Validation: Loss 0.23303 Accuracy 1.00000
-Epoch [  5]: Loss 0.20778
-Epoch [  5]: Loss 0.21737
-Epoch [  5]: Loss 0.17930
-Epoch [  5]: Loss 0.19193
-Epoch [  5]: Loss 0.18534
-Epoch [  5]: Loss 0.20299
-Epoch [  5]: Loss 0.17725
-Validation: Loss 0.16526 Accuracy 1.00000
-Validation: Loss 0.17602 Accuracy 1.00000
-Epoch [  6]: Loss 0.15675
-Epoch [  6]: Loss 0.16486
-Epoch [  6]: Loss 0.15707
-Epoch [  6]: Loss 0.13994
-Epoch [  6]: Loss 0.13805
-Epoch [  6]: Loss 0.12533
-Epoch [  6]: Loss 0.12193
-Validation: Loss 0.12156 Accuracy 1.00000
-Validation: Loss 0.13025 Accuracy 1.00000
-Epoch [  7]: Loss 0.12611
-Epoch [  7]: Loss 0.11352
-Epoch [  7]: Loss 0.09855
-Epoch [  7]: Loss 0.10333
-Epoch [  7]: Loss 0.10313
-Epoch [  7]: Loss 0.09685
-Epoch [  7]: Loss 0.09368
-Validation: Loss 0.08737 Accuracy 1.00000
-Validation: Loss 0.09384 Accuracy 1.00000
-Epoch [  8]: Loss 0.08845
-Epoch [  8]: Loss 0.07892
-Epoch [  8]: Loss 0.07708
-Epoch [  8]: Loss 0.07038
-Epoch [  8]: Loss 0.07520
-Epoch [  8]: Loss 0.06707
-Epoch [  8]: Loss 0.05914
-Validation: Loss 0.06083 Accuracy 1.00000
-Validation: Loss 0.06521 Accuracy 1.00000
-Epoch [  9]: Loss 0.05626
-Epoch [  9]: Loss 0.05812
-Epoch [  9]: Loss 0.05727
-Epoch [  9]: Loss 0.04751
-Epoch [  9]: Loss 0.04994
-Epoch [  9]: Loss 0.04992
-Epoch [  9]: Loss 0.05267
-Validation: Loss 0.04513 Accuracy 1.00000
-Validation: Loss 0.04821 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04409
-Epoch [ 10]: Loss 0.04117
-Epoch [ 10]: Loss 0.04124
-Epoch [ 10]: Loss 0.03994
-Epoch [ 10]: Loss 0.04008
-Epoch [ 10]: Loss 0.03930
-Epoch [ 10]: Loss 0.03853
-Validation: Loss 0.03653 Accuracy 1.00000
-Validation: Loss 0.03901 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03718
-Epoch [ 11]: Loss 0.03205
-Epoch [ 11]: Loss 0.03301
-Epoch [ 11]: Loss 0.03540
-Epoch [ 11]: Loss 0.03260
-Epoch [ 11]: Loss 0.03353
-Epoch [ 11]: Loss 0.02814
-Validation: Loss 0.03100 Accuracy 1.00000
-Validation: Loss 0.03314 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03047
-Epoch [ 12]: Loss 0.03012
-Epoch [ 12]: Loss 0.03002
-Epoch [ 12]: Loss 0.03018
-Epoch [ 12]: Loss 0.02835
-Epoch [ 12]: Loss 0.02538
-Epoch [ 12]: Loss 0.02536
-Validation: Loss 0.02699 Accuracy 1.00000
-Validation: Loss 0.02888 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02532
-Epoch [ 13]: Loss 0.02466
-Epoch [ 13]: Loss 0.02676
-Epoch [ 13]: Loss 0.02630
-Epoch [ 13]: Loss 0.02537
-Epoch [ 13]: Loss 0.02363
-Epoch [ 13]: Loss 0.02499
-Validation: Loss 0.02390 Accuracy 1.00000
-Validation: Loss 0.02561 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02328
-Epoch [ 14]: Loss 0.02500
-Epoch [ 14]: Loss 0.02232
-Epoch [ 14]: Loss 0.02259
-Epoch [ 14]: Loss 0.02060
-Epoch [ 14]: Loss 0.02168
-Epoch [ 14]: Loss 0.02158
-Validation: Loss 0.02141 Accuracy 1.00000
-Validation: Loss 0.02297 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02163
-Epoch [ 15]: Loss 0.02014
-Epoch [ 15]: Loss 0.02060
-Epoch [ 15]: Loss 0.02013
-Epoch [ 15]: Loss 0.01927
-Epoch [ 15]: Loss 0.01935
-Epoch [ 15]: Loss 0.02165
-Validation: Loss 0.01935 Accuracy 1.00000
-Validation: Loss 0.02078 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01828
-Epoch [ 16]: Loss 0.01926
-Epoch [ 16]: Loss 0.01919
-Epoch [ 16]: Loss 0.01910
-Epoch [ 16]: Loss 0.01779
-Epoch [ 16]: Loss 0.01724
-Epoch [ 16]: Loss 0.01498
-Validation: Loss 0.01759 Accuracy 1.00000
-Validation: Loss 0.01892 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01661
-Epoch [ 17]: Loss 0.01720
-Epoch [ 17]: Loss 0.01668
-Epoch [ 17]: Loss 0.01709
-Epoch [ 17]: Loss 0.01608
-Epoch [ 17]: Loss 0.01699
-Epoch [ 17]: Loss 0.01488
-Validation: Loss 0.01609 Accuracy 1.00000
+ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.61512
+Epoch [  1]: Loss 0.60524
+Epoch [  1]: Loss 0.54823
+Epoch [  1]: Loss 0.54691
+Epoch [  1]: Loss 0.52434
+Epoch [  1]: Loss 0.50498
+Epoch [  1]: Loss 0.49311
+Validation: Loss 0.46981 Accuracy 1.00000
+Validation: Loss 0.45988 Accuracy 1.00000
+Epoch [  2]: Loss 0.47294
+Epoch [  2]: Loss 0.45229
+Epoch [  2]: Loss 0.44374
+Epoch [  2]: Loss 0.42407
+Epoch [  2]: Loss 0.41602
+Epoch [  2]: Loss 0.40209
+Epoch [  2]: Loss 0.39644
+Validation: Loss 0.37335 Accuracy 1.00000
+Validation: Loss 0.36179 Accuracy 1.00000
+Epoch [  3]: Loss 0.37796
+Epoch [  3]: Loss 0.36535
+Epoch [  3]: Loss 0.34490
+Epoch [  3]: Loss 0.33645
+Epoch [  3]: Loss 0.32186
+Epoch [  3]: Loss 0.31185
+Epoch [  3]: Loss 0.29527
+Validation: Loss 0.28839 Accuracy 1.00000
+Validation: Loss 0.27590 Accuracy 1.00000
+Epoch [  4]: Loss 0.29135
+Epoch [  4]: Loss 0.28370
+Epoch [  4]: Loss 0.26049
+Epoch [  4]: Loss 0.26873
+Epoch [  4]: Loss 0.24797
+Epoch [  4]: Loss 0.23672
+Epoch [  4]: Loss 0.18915
+Validation: Loss 0.21823 Accuracy 1.00000
+Validation: Loss 0.20570 Accuracy 1.00000
+Epoch [  5]: Loss 0.21983
+Epoch [  5]: Loss 0.22068
+Epoch [  5]: Loss 0.20337
+Epoch [  5]: Loss 0.18072
+Epoch [  5]: Loss 0.18741
+Epoch [  5]: Loss 0.18099
+Epoch [  5]: Loss 0.15911
+Validation: Loss 0.16261 Accuracy 1.00000
+Validation: Loss 0.15102 Accuracy 1.00000
+Epoch [  6]: Loss 0.16786
+Epoch [  6]: Loss 0.14418
+Epoch [  6]: Loss 0.14681
+Epoch [  6]: Loss 0.15733
+Epoch [  6]: Loss 0.13193
+Epoch [  6]: Loss 0.13234
+Epoch [  6]: Loss 0.13811
+Validation: Loss 0.11957 Accuracy 1.00000
+Validation: Loss 0.11001 Accuracy 1.00000
+Epoch [  7]: Loss 0.12181
+Epoch [  7]: Loss 0.10795
+Epoch [  7]: Loss 0.10904
+Epoch [  7]: Loss 0.09474
+Epoch [  7]: Loss 0.10197
+Epoch [  7]: Loss 0.10666
+Epoch [  7]: Loss 0.09703
+Validation: Loss 0.08569 Accuracy 1.00000
+Validation: Loss 0.07868 Accuracy 1.00000
+Epoch [  8]: Loss 0.09038
+Epoch [  8]: Loss 0.08012
+Epoch [  8]: Loss 0.08368
+Epoch [  8]: Loss 0.07743
+Epoch [  8]: Loss 0.06634
+Epoch [  8]: Loss 0.06049
+Epoch [  8]: Loss 0.06252
+Validation: Loss 0.05944 Accuracy 1.00000
+Validation: Loss 0.05485 Accuracy 1.00000
+Epoch [  9]: Loss 0.05594
+Epoch [  9]: Loss 0.05566
+Epoch [  9]: Loss 0.05580
+Epoch [  9]: Loss 0.05207
+Epoch [  9]: Loss 0.05399
+Epoch [  9]: Loss 0.04829
+Epoch [  9]: Loss 0.03966
+Validation: Loss 0.04414 Accuracy 1.00000
+Validation: Loss 0.04086 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04453
+Epoch [ 10]: Loss 0.04691
+Epoch [ 10]: Loss 0.03821
+Epoch [ 10]: Loss 0.03967
+Epoch [ 10]: Loss 0.04043
+Epoch [ 10]: Loss 0.03839
+Epoch [ 10]: Loss 0.03051
+Validation: Loss 0.03575 Accuracy 1.00000
+Validation: Loss 0.03306 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03488
+Epoch [ 11]: Loss 0.03574
+Epoch [ 11]: Loss 0.03342
+Epoch [ 11]: Loss 0.03265
+Epoch [ 11]: Loss 0.03160
+Epoch [ 11]: Loss 0.03473
+Epoch [ 11]: Loss 0.03210
+Validation: Loss 0.03038 Accuracy 1.00000
+Validation: Loss 0.02803 Accuracy 1.00000
+Epoch [ 12]: Loss 0.02933
+Epoch [ 12]: Loss 0.03052
+Epoch [ 12]: Loss 0.02711
+Epoch [ 12]: Loss 0.03082
+Epoch [ 12]: Loss 0.02924
+Epoch [ 12]: Loss 0.02724
+Epoch [ 12]: Loss 0.02788
+Validation: Loss 0.02645 Accuracy 1.00000
+Validation: Loss 0.02436 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02505
+Epoch [ 13]: Loss 0.02758
+Epoch [ 13]: Loss 0.02597
+Epoch [ 13]: Loss 0.02615
+Epoch [ 13]: Loss 0.02482
+Epoch [ 13]: Loss 0.02354
+Epoch [ 13]: Loss 0.02294
+Validation: Loss 0.02339 Accuracy 1.00000
+Validation: Loss 0.02151 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02298
+Epoch [ 14]: Loss 0.02550
+Epoch [ 14]: Loss 0.02284
+Epoch [ 14]: Loss 0.02288
+Epoch [ 14]: Loss 0.02034
+Epoch [ 14]: Loss 0.02120
+Epoch [ 14]: Loss 0.02200
+Validation: Loss 0.02094 Accuracy 1.00000
+Validation: Loss 0.01923 Accuracy 1.00000
+Epoch [ 15]: Loss 0.02073
+Epoch [ 15]: Loss 0.02081
+Epoch [ 15]: Loss 0.01883
+Epoch [ 15]: Loss 0.02020
+Epoch [ 15]: Loss 0.01974
+Epoch [ 15]: Loss 0.02074
+Epoch [ 15]: Loss 0.02274
+Validation: Loss 0.01891 Accuracy 1.00000
 Validation: Loss 0.01733 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01658
-Epoch [ 18]: Loss 0.01422
-Epoch [ 18]: Loss 0.01632
-Epoch [ 18]: Loss 0.01551
-Epoch [ 18]: Loss 0.01564
-Epoch [ 18]: Loss 0.01396
-Epoch [ 18]: Loss 0.01366
-Validation: Loss 0.01479 Accuracy 1.00000
-Validation: Loss 0.01595 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01597
-Epoch [ 19]: Loss 0.01463
-Epoch [ 19]: Loss 0.01458
-Epoch [ 19]: Loss 0.01428
-Epoch [ 19]: Loss 0.01315
-Epoch [ 19]: Loss 0.01243
-Epoch [ 19]: Loss 0.01202
-Validation: Loss 0.01364 Accuracy 1.00000
-Validation: Loss 0.01472 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01785
+Epoch [ 16]: Loss 0.01951
+Epoch [ 16]: Loss 0.01925
+Epoch [ 16]: Loss 0.01797
+Epoch [ 16]: Loss 0.01738
+Epoch [ 16]: Loss 0.01879
+Epoch [ 16]: Loss 0.01609
+Validation: Loss 0.01717 Accuracy 1.00000
+Validation: Loss 0.01572 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01831
+Epoch [ 17]: Loss 0.01727
+Epoch [ 17]: Loss 0.01735
+Epoch [ 17]: Loss 0.01683
+Epoch [ 17]: Loss 0.01505
+Epoch [ 17]: Loss 0.01624
+Epoch [ 17]: Loss 0.01405
+Validation: Loss 0.01568 Accuracy 1.00000
+Validation: Loss 0.01433 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01569
+Epoch [ 18]: Loss 0.01639
+Epoch [ 18]: Loss 0.01485
+Epoch [ 18]: Loss 0.01613
+Epoch [ 18]: Loss 0.01420
+Epoch [ 18]: Loss 0.01461
+Epoch [ 18]: Loss 0.01515
+Validation: Loss 0.01440 Accuracy 1.00000
+Validation: Loss 0.01314 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01552
+Epoch [ 19]: Loss 0.01451
+Epoch [ 19]: Loss 0.01378
+Epoch [ 19]: Loss 0.01444
+Epoch [ 19]: Loss 0.01317
+Epoch [ 19]: Loss 0.01395
+Epoch [ 19]: Loss 0.01025
+Validation: Loss 0.01325 Accuracy 1.00000
+Validation: Loss 0.01208 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01351
 Epoch [ 20]: Loss 0.01270
-Epoch [ 20]: Loss 0.01350
-Epoch [ 20]: Loss 0.01361
-Epoch [ 20]: Loss 0.01174
-Epoch [ 20]: Loss 0.01357
-Epoch [ 20]: Loss 0.01328
-Epoch [ 20]: Loss 0.01111
-Validation: Loss 0.01260 Accuracy 1.00000
-Validation: Loss 0.01361 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01262
-Epoch [ 21]: Loss 0.01259
-Epoch [ 21]: Loss 0.01207
-Epoch [ 21]: Loss 0.01204
-Epoch [ 21]: Loss 0.01107
-Epoch [ 21]: Loss 0.01176
+Epoch [ 20]: Loss 0.01306
+Epoch [ 20]: Loss 0.01216
+Epoch [ 20]: Loss 0.01388
+Epoch [ 20]: Loss 0.01223
+Epoch [ 20]: Loss 0.01327
+Validation: Loss 0.01217 Accuracy 1.00000
+Validation: Loss 0.01110 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01264
+Epoch [ 21]: Loss 0.01238
+Epoch [ 21]: Loss 0.01103
+Epoch [ 21]: Loss 0.01147
+Epoch [ 21]: Loss 0.01289
+Epoch [ 21]: Loss 0.01081
 Epoch [ 21]: Loss 0.01118
-Validation: Loss 0.01157 Accuracy 1.00000
-Validation: Loss 0.01250 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01136
-Epoch [ 22]: Loss 0.01084
-Epoch [ 22]: Loss 0.01072
-Epoch [ 22]: Loss 0.01103
-Epoch [ 22]: Loss 0.01129
-Epoch [ 22]: Loss 0.01068
-Epoch [ 22]: Loss 0.01013
-Validation: Loss 0.01044 Accuracy 1.00000
-Validation: Loss 0.01127 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01126
-Epoch [ 23]: Loss 0.01079
-Epoch [ 23]: Loss 0.00862
-Epoch [ 23]: Loss 0.00926
-Epoch [ 23]: Loss 0.00956
-Epoch [ 23]: Loss 0.00950
-Epoch [ 23]: Loss 0.00933
-Validation: Loss 0.00923 Accuracy 1.00000
-Validation: Loss 0.00994 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00898
-Epoch [ 24]: Loss 0.00918
-Epoch [ 24]: Loss 0.00925
-Epoch [ 24]: Loss 0.00852
-Epoch [ 24]: Loss 0.00775
-Epoch [ 24]: Loss 0.00819
-Epoch [ 24]: Loss 0.00957
-Validation: Loss 0.00824 Accuracy 1.00000
-Validation: Loss 0.00886 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00832
-Epoch [ 25]: Loss 0.00852
-Epoch [ 25]: Loss 0.00766
-Epoch [ 25]: Loss 0.00720
-Epoch [ 25]: Loss 0.00726
-Epoch [ 25]: Loss 0.00801
-Epoch [ 25]: Loss 0.00786
-Validation: Loss 0.00755 Accuracy 1.00000
-Validation: Loss 0.00809 Accuracy 1.00000

    We can also train the compact model with the exact same code!

    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.61512
-Epoch [  1]: Loss 0.60242
-Epoch [  1]: Loss 0.55694
-Epoch [  1]: Loss 0.53467
-Epoch [  1]: Loss 0.51785
-Epoch [  1]: Loss 0.50398
-Epoch [  1]: Loss 0.46705
-Validation: Loss 0.46429 Accuracy 1.00000
-Validation: Loss 0.49806 Accuracy 1.00000
-Epoch [  2]: Loss 0.46969
-Epoch [  2]: Loss 0.44272
-Epoch [  2]: Loss 0.43718
-Epoch [  2]: Loss 0.42512
-Epoch [  2]: Loss 0.42391
-Epoch [  2]: Loss 0.39088
-Epoch [  2]: Loss 0.38009
-Validation: Loss 0.36741 Accuracy 1.00000
-Validation: Loss 0.40898 Accuracy 1.00000
-Epoch [  3]: Loss 0.36482
-Epoch [  3]: Loss 0.36399
-Epoch [  3]: Loss 0.34310
-Epoch [  3]: Loss 0.33063
-Epoch [  3]: Loss 0.31571
-Epoch [  3]: Loss 0.31828
-Epoch [  3]: Loss 0.28813
-Validation: Loss 0.28291 Accuracy 1.00000
-Validation: Loss 0.32788 Accuracy 1.00000
-Epoch [  4]: Loss 0.28119
-Epoch [  4]: Loss 0.26874
-Epoch [  4]: Loss 0.27318
-Epoch [  4]: Loss 0.24913
-Epoch [  4]: Loss 0.24549
-Epoch [  4]: Loss 0.23824
-Epoch [  4]: Loss 0.23989
-Validation: Loss 0.21332 Accuracy 1.00000
-Validation: Loss 0.25775 Accuracy 1.00000
-Epoch [  5]: Loss 0.22898
-Epoch [  5]: Loss 0.21470
-Epoch [  5]: Loss 0.18468
-Epoch [  5]: Loss 0.18042
-Epoch [  5]: Loss 0.17828
-Epoch [  5]: Loss 0.18276
-Epoch [  5]: Loss 0.17560
-Validation: Loss 0.15792 Accuracy 1.00000
-Validation: Loss 0.19791 Accuracy 1.00000
-Epoch [  6]: Loss 0.16034
-Epoch [  6]: Loss 0.16505
-Epoch [  6]: Loss 0.13701
-Epoch [  6]: Loss 0.14388
-Epoch [  6]: Loss 0.13054
-Epoch [  6]: Loss 0.12183
-Epoch [  6]: Loss 0.15076
-Validation: Loss 0.11535 Accuracy 1.00000
-Validation: Loss 0.14781 Accuracy 1.00000
-Epoch [  7]: Loss 0.12367
-Epoch [  7]: Loss 0.11161
-Epoch [  7]: Loss 0.11171
-Epoch [  7]: Loss 0.09255
-Epoch [  7]: Loss 0.10395
-Epoch [  7]: Loss 0.09041
-Epoch [  7]: Loss 0.07027
-Validation: Loss 0.08214 Accuracy 1.00000
-Validation: Loss 0.10572 Accuracy 1.00000
-Epoch [  8]: Loss 0.08794
-Epoch [  8]: Loss 0.07938
-Epoch [  8]: Loss 0.07477
-Epoch [  8]: Loss 0.06629
-Epoch [  8]: Loss 0.07276
-Epoch [  8]: Loss 0.06286
-Epoch [  8]: Loss 0.05999
-Validation: Loss 0.05727 Accuracy 1.00000
-Validation: Loss 0.07302 Accuracy 1.00000
-Epoch [  9]: Loss 0.05903
-Epoch [  9]: Loss 0.05738
-Epoch [  9]: Loss 0.05522
-Epoch [  9]: Loss 0.05230
-Epoch [  9]: Loss 0.04565
-Epoch [  9]: Loss 0.04576
-Epoch [  9]: Loss 0.03811
-Validation: Loss 0.04256 Accuracy 1.00000
-Validation: Loss 0.05375 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04291
-Epoch [ 10]: Loss 0.04001
-Epoch [ 10]: Loss 0.03806
-Epoch [ 10]: Loss 0.04058
-Epoch [ 10]: Loss 0.04058
-Epoch [ 10]: Loss 0.03729
-Epoch [ 10]: Loss 0.03779
-Validation: Loss 0.03443 Accuracy 1.00000
-Validation: Loss 0.04360 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03438
-Epoch [ 11]: Loss 0.03180
-Epoch [ 11]: Loss 0.03152
-Epoch [ 11]: Loss 0.03531
-Epoch [ 11]: Loss 0.03175
-Epoch [ 11]: Loss 0.03104
-Epoch [ 11]: Loss 0.03834
-Validation: Loss 0.02918 Accuracy 1.00000
-Validation: Loss 0.03715 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03147
-Epoch [ 12]: Loss 0.02983
-Epoch [ 12]: Loss 0.02744
-Epoch [ 12]: Loss 0.02762
+Validation: Loss 0.01103 Accuracy 1.00000
+Validation: Loss 0.01008 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01158
+Epoch [ 22]: Loss 0.00968
+Epoch [ 22]: Loss 0.01026
+Epoch [ 22]: Loss 0.01091
+Epoch [ 22]: Loss 0.01081
+Epoch [ 22]: Loss 0.01023
+Epoch [ 22]: Loss 0.01178
+Validation: Loss 0.00977 Accuracy 1.00000
+Validation: Loss 0.00896 Accuracy 1.00000
+Epoch [ 23]: Loss 0.00874
+Epoch [ 23]: Loss 0.00915
+Epoch [ 23]: Loss 0.01013
+Epoch [ 23]: Loss 0.00944
+Epoch [ 23]: Loss 0.00958
+Epoch [ 23]: Loss 0.00952
+Epoch [ 23]: Loss 0.00815
+Validation: Loss 0.00866 Accuracy 1.00000
+Validation: Loss 0.00796 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00897
+Epoch [ 24]: Loss 0.00868
+Epoch [ 24]: Loss 0.00832
+Epoch [ 24]: Loss 0.00880
+Epoch [ 24]: Loss 0.00824
+Epoch [ 24]: Loss 0.00784
+Epoch [ 24]: Loss 0.00653
+Validation: Loss 0.00786 Accuracy 1.00000
+Validation: Loss 0.00724 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00807
+Epoch [ 25]: Loss 0.00791
+Epoch [ 25]: Loss 0.00809
+Epoch [ 25]: Loss 0.00746
+Epoch [ 25]: Loss 0.00784
+Epoch [ 25]: Loss 0.00691
+Epoch [ 25]: Loss 0.00709
+Validation: Loss 0.00729 Accuracy 1.00000
+Validation: Loss 0.00672 Accuracy 1.00000

    We can also train the compact model with the exact same code!

    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.62544
+Epoch [  1]: Loss 0.58391
+Epoch [  1]: Loss 0.57736
+Epoch [  1]: Loss 0.53167
+Epoch [  1]: Loss 0.51671
+Epoch [  1]: Loss 0.49884
+Epoch [  1]: Loss 0.48786
+Validation: Loss 0.47871 Accuracy 1.00000
+Validation: Loss 0.46396 Accuracy 1.00000
+Epoch [  2]: Loss 0.47308
+Epoch [  2]: Loss 0.45026
+Epoch [  2]: Loss 0.43611
+Epoch [  2]: Loss 0.42725
+Epoch [  2]: Loss 0.40983
+Epoch [  2]: Loss 0.39966
+Epoch [  2]: Loss 0.37452
+Validation: Loss 0.38291 Accuracy 1.00000
+Validation: Loss 0.36610 Accuracy 1.00000
+Epoch [  3]: Loss 0.37263
+Epoch [  3]: Loss 0.36606
+Epoch [  3]: Loss 0.34123
+Epoch [  3]: Loss 0.33349
+Epoch [  3]: Loss 0.32480
+Epoch [  3]: Loss 0.29689
+Epoch [  3]: Loss 0.28596
+Validation: Loss 0.29870 Accuracy 1.00000
+Validation: Loss 0.28058 Accuracy 1.00000
+Epoch [  4]: Loss 0.29523
+Epoch [  4]: Loss 0.28169
+Epoch [  4]: Loss 0.25398
+Epoch [  4]: Loss 0.25350
+Epoch [  4]: Loss 0.24640
+Epoch [  4]: Loss 0.23055
+Epoch [  4]: Loss 0.19666
+Validation: Loss 0.22878 Accuracy 1.00000
+Validation: Loss 0.21061 Accuracy 1.00000
+Epoch [  5]: Loss 0.20867
+Epoch [  5]: Loss 0.21100
+Epoch [  5]: Loss 0.19381
+Epoch [  5]: Loss 0.18173
+Epoch [  5]: Loss 0.19679
+Epoch [  5]: Loss 0.17571
+Epoch [  5]: Loss 0.16454
+Validation: Loss 0.17258 Accuracy 1.00000
+Validation: Loss 0.15561 Accuracy 1.00000
+Epoch [  6]: Loss 0.15738
+Epoch [  6]: Loss 0.15615
+Epoch [  6]: Loss 0.15424
+Epoch [  6]: Loss 0.13024
+Epoch [  6]: Loss 0.13682
+Epoch [  6]: Loss 0.13057
+Epoch [  6]: Loss 0.11993
+Validation: Loss 0.12772 Accuracy 1.00000
+Validation: Loss 0.11373 Accuracy 1.00000
+Epoch [  7]: Loss 0.12709
+Epoch [  7]: Loss 0.11010
+Epoch [  7]: Loss 0.10527
+Epoch [  7]: Loss 0.10227
+Epoch [  7]: Loss 0.09745
+Epoch [  7]: Loss 0.08796
+Epoch [  7]: Loss 0.09235
+Validation: Loss 0.09154 Accuracy 1.00000
+Validation: Loss 0.08131 Accuracy 1.00000
+Epoch [  8]: Loss 0.08486
+Epoch [  8]: Loss 0.07846
+Epoch [  8]: Loss 0.08010
+Epoch [  8]: Loss 0.06830
+Epoch [  8]: Loss 0.07391
+Epoch [  8]: Loss 0.05747
+Epoch [  8]: Loss 0.07270
+Validation: Loss 0.06335 Accuracy 1.00000
+Validation: Loss 0.05660 Accuracy 1.00000
+Epoch [  9]: Loss 0.05451
+Epoch [  9]: Loss 0.05514
+Epoch [  9]: Loss 0.05569
+Epoch [  9]: Loss 0.05111
+Epoch [  9]: Loss 0.04776
+Epoch [  9]: Loss 0.04815
+Epoch [  9]: Loss 0.04527
+Validation: Loss 0.04674 Accuracy 1.00000
+Validation: Loss 0.04191 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04214
+Epoch [ 10]: Loss 0.04455
+Epoch [ 10]: Loss 0.04306
+Epoch [ 10]: Loss 0.03731
+Epoch [ 10]: Loss 0.03861
+Epoch [ 10]: Loss 0.03507
+Epoch [ 10]: Loss 0.03270
+Validation: Loss 0.03776 Accuracy 1.00000
+Validation: Loss 0.03378 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03550
+Epoch [ 11]: Loss 0.03377
+Epoch [ 11]: Loss 0.03181
+Epoch [ 11]: Loss 0.03216
+Epoch [ 11]: Loss 0.03193
+Epoch [ 11]: Loss 0.03174
+Epoch [ 11]: Loss 0.03156
+Validation: Loss 0.03209 Accuracy 1.00000
+Validation: Loss 0.02861 Accuracy 1.00000
+Epoch [ 12]: Loss 0.03212
+Epoch [ 12]: Loss 0.02678
+Epoch [ 12]: Loss 0.02960
+Epoch [ 12]: Loss 0.02662
 Epoch [ 12]: Loss 0.02777
-Epoch [ 12]: Loss 0.02654
-Epoch [ 12]: Loss 0.02210
-Validation: Loss 0.02531 Accuracy 1.00000
-Validation: Loss 0.03235 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02393
+Epoch [ 12]: Loss 0.02676
+Epoch [ 12]: Loss 0.02374
+Validation: Loss 0.02795 Accuracy 1.00000
+Validation: Loss 0.02484 Accuracy 1.00000
 Epoch [ 13]: Loss 0.02618
-Epoch [ 13]: Loss 0.02483
-Epoch [ 13]: Loss 0.02462
-Epoch [ 13]: Loss 0.02438
+Epoch [ 13]: Loss 0.02595
+Epoch [ 13]: Loss 0.02439
+Epoch [ 13]: Loss 0.02525
 Epoch [ 13]: Loss 0.02377
-Epoch [ 13]: Loss 0.02443
-Validation: Loss 0.02236 Accuracy 1.00000
-Validation: Loss 0.02872 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02299
-Epoch [ 14]: Loss 0.02280
-Epoch [ 14]: Loss 0.02283
-Epoch [ 14]: Loss 0.02205
-Epoch [ 14]: Loss 0.02132
-Epoch [ 14]: Loss 0.01942
-Epoch [ 14]: Loss 0.02104
-Validation: Loss 0.01997 Accuracy 1.00000
-Validation: Loss 0.02578 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02167
-Epoch [ 15]: Loss 0.01947
-Epoch [ 15]: Loss 0.02176
-Epoch [ 15]: Loss 0.01962
-Epoch [ 15]: Loss 0.01752
+Epoch [ 13]: Loss 0.02294
+Epoch [ 13]: Loss 0.02093
+Validation: Loss 0.02477 Accuracy 1.00000
+Validation: Loss 0.02195 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02317
+Epoch [ 14]: Loss 0.02142
+Epoch [ 14]: Loss 0.02263
+Epoch [ 14]: Loss 0.02210
+Epoch [ 14]: Loss 0.02096
+Epoch [ 14]: Loss 0.02121
+Epoch [ 14]: Loss 0.02011
+Validation: Loss 0.02221 Accuracy 1.00000
+Validation: Loss 0.01962 Accuracy 1.00000
 Epoch [ 15]: Loss 0.01816
-Epoch [ 15]: Loss 0.01755
-Validation: Loss 0.01800 Accuracy 1.00000
-Validation: Loss 0.02335 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01851
-Epoch [ 16]: Loss 0.01911
-Epoch [ 16]: Loss 0.01715
-Epoch [ 16]: Loss 0.01675
-Epoch [ 16]: Loss 0.01767
+Epoch [ 15]: Loss 0.02028
+Epoch [ 15]: Loss 0.01997
+Epoch [ 15]: Loss 0.01986
+Epoch [ 15]: Loss 0.02098
+Epoch [ 15]: Loss 0.01841
+Epoch [ 15]: Loss 0.01955
+Validation: Loss 0.02010 Accuracy 1.00000
+Validation: Loss 0.01770 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01771
+Epoch [ 16]: Loss 0.01766
+Epoch [ 16]: Loss 0.01857
+Epoch [ 16]: Loss 0.01865
+Epoch [ 16]: Loss 0.01644
+Epoch [ 16]: Loss 0.01765
 Epoch [ 16]: Loss 0.01711
-Epoch [ 16]: Loss 0.01785
-Validation: Loss 0.01634 Accuracy 1.00000
-Validation: Loss 0.02131 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01649
-Epoch [ 17]: Loss 0.01484
-Epoch [ 17]: Loss 0.01512
-Epoch [ 17]: Loss 0.01680
-Epoch [ 17]: Loss 0.01660
-Epoch [ 17]: Loss 0.01670
-Epoch [ 17]: Loss 0.01695
-Validation: Loss 0.01494 Accuracy 1.00000
-Validation: Loss 0.01957 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01493
-Epoch [ 18]: Loss 0.01534
-Epoch [ 18]: Loss 0.01605
-Epoch [ 18]: Loss 0.01395
-Epoch [ 18]: Loss 0.01464
-Epoch [ 18]: Loss 0.01375
-Epoch [ 18]: Loss 0.01502
-Validation: Loss 0.01372 Accuracy 1.00000
-Validation: Loss 0.01803 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01358
-Epoch [ 19]: Loss 0.01467
-Epoch [ 19]: Loss 0.01368
-Epoch [ 19]: Loss 0.01324
-Epoch [ 19]: Loss 0.01336
-Epoch [ 19]: Loss 0.01362
-Epoch [ 19]: Loss 0.01173
-Validation: Loss 0.01266 Accuracy 1.00000
-Validation: Loss 0.01667 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01371
-Epoch [ 20]: Loss 0.01225
-Epoch [ 20]: Loss 0.01174
-Epoch [ 20]: Loss 0.01224
-Epoch [ 20]: Loss 0.01298
-Epoch [ 20]: Loss 0.01183
-Epoch [ 20]: Loss 0.01526
-Validation: Loss 0.01170 Accuracy 1.00000
-Validation: Loss 0.01541 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01193
-Epoch [ 21]: Loss 0.01288
-Epoch [ 21]: Loss 0.01119
-Epoch [ 21]: Loss 0.01071
-Epoch [ 21]: Loss 0.01207
-Epoch [ 21]: Loss 0.01124
-Epoch [ 21]: Loss 0.00995
-Validation: Loss 0.01074 Accuracy 1.00000
-Validation: Loss 0.01412 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01029
-Epoch [ 22]: Loss 0.01079
-Epoch [ 22]: Loss 0.01077
-Epoch [ 22]: Loss 0.01157
-Epoch [ 22]: Loss 0.01110
-Epoch [ 22]: Loss 0.00930
-Epoch [ 22]: Loss 0.00945
-Validation: Loss 0.00970 Accuracy 1.00000
-Validation: Loss 0.01269 Accuracy 1.00000
-Epoch [ 23]: Loss 0.00912
-Epoch [ 23]: Loss 0.00974
-Epoch [ 23]: Loss 0.00977
-Epoch [ 23]: Loss 0.00978
-Epoch [ 23]: Loss 0.00924
-Epoch [ 23]: Loss 0.00950
-Epoch [ 23]: Loss 0.00840
-Validation: Loss 0.00862 Accuracy 1.00000
-Validation: Loss 0.01118 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00903
-Epoch [ 24]: Loss 0.00880
-Epoch [ 24]: Loss 0.00833
-Epoch [ 24]: Loss 0.00852
-Epoch [ 24]: Loss 0.00776
-Epoch [ 24]: Loss 0.00803
-Epoch [ 24]: Loss 0.00902
-Validation: Loss 0.00774 Accuracy 1.00000
-Validation: Loss 0.00995 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00774
-Epoch [ 25]: Loss 0.00778
-Epoch [ 25]: Loss 0.00731
-Epoch [ 25]: Loss 0.00812
-Epoch [ 25]: Loss 0.00823
-Epoch [ 25]: Loss 0.00713
-Epoch [ 25]: Loss 0.00560
-Validation: Loss 0.00710 Accuracy 1.00000
-Validation: Loss 0.00909 Accuracy 1.00000

    Saving the Model ​

    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.

    julia
    @save "trained_model.jld2" ps_trained st_trained

    Let's try loading the model

    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
+Validation: Loss 0.01829 Accuracy 1.00000
+Validation: Loss 0.01607 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01641
+Epoch [ 17]: Loss 0.01635
+Epoch [ 17]: Loss 0.01721
+Epoch [ 17]: Loss 0.01516
+Epoch [ 17]: Loss 0.01558
+Epoch [ 17]: Loss 0.01598
+Epoch [ 17]: Loss 0.01744
+Validation: Loss 0.01676 Accuracy 1.00000
+Validation: Loss 0.01468 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01574
+Epoch [ 18]: Loss 0.01612
+Epoch [ 18]: Loss 0.01448
+Epoch [ 18]: Loss 0.01332
+Epoch [ 18]: Loss 0.01484
+Epoch [ 18]: Loss 0.01454
+Epoch [ 18]: Loss 0.01437
+Validation: Loss 0.01542 Accuracy 1.00000
+Validation: Loss 0.01349 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01310
+Epoch [ 19]: Loss 0.01445
+Epoch [ 19]: Loss 0.01387
+Epoch [ 19]: Loss 0.01386
+Epoch [ 19]: Loss 0.01291
+Epoch [ 19]: Loss 0.01382
+Epoch [ 19]: Loss 0.01327
+Validation: Loss 0.01428 Accuracy 1.00000
+Validation: Loss 0.01247 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01262
+Epoch [ 20]: Loss 0.01312
+Epoch [ 20]: Loss 0.01335
+Epoch [ 20]: Loss 0.01283
+Epoch [ 20]: Loss 0.01258
+Epoch [ 20]: Loss 0.01185
+Epoch [ 20]: Loss 0.01087
+Validation: Loss 0.01326 Accuracy 1.00000
+Validation: Loss 0.01157 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01242
+Epoch [ 21]: Loss 0.01076
+Epoch [ 21]: Loss 0.01183
+Epoch [ 21]: Loss 0.01136
+Epoch [ 21]: Loss 0.01204
+Epoch [ 21]: Loss 0.01192
+Epoch [ 21]: Loss 0.01236
+Validation: Loss 0.01233 Accuracy 1.00000
+Validation: Loss 0.01077 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01078
+Epoch [ 22]: Loss 0.01177
+Epoch [ 22]: Loss 0.01122
+Epoch [ 22]: Loss 0.01035
+Epoch [ 22]: Loss 0.01076
+Epoch [ 22]: Loss 0.01075
+Epoch [ 22]: Loss 0.01061
+Validation: Loss 0.01143 Accuracy 1.00000
+Validation: Loss 0.00999 Accuracy 1.00000
+Epoch [ 23]: Loss 0.01001
+Epoch [ 23]: Loss 0.00999
+Epoch [ 23]: Loss 0.00999
+Epoch [ 23]: Loss 0.01055
+Epoch [ 23]: Loss 0.00962
+Epoch [ 23]: Loss 0.01039
+Epoch [ 23]: Loss 0.01015
+Validation: Loss 0.01047 Accuracy 1.00000
+Validation: Loss 0.00918 Accuracy 1.00000
+Epoch [ 24]: Loss 0.01096
+Epoch [ 24]: Loss 0.00877
+Epoch [ 24]: Loss 0.00900
+Epoch [ 24]: Loss 0.00879
+Epoch [ 24]: Loss 0.00932
+Epoch [ 24]: Loss 0.00853
+Epoch [ 24]: Loss 0.00848
+Validation: Loss 0.00936 Accuracy 1.00000
+Validation: Loss 0.00825 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00799
+Epoch [ 25]: Loss 0.00883
+Epoch [ 25]: Loss 0.00804
+Epoch [ 25]: Loss 0.00830
+Epoch [ 25]: Loss 0.00800
+Epoch [ 25]: Loss 0.00805
+Epoch [ 25]: Loss 0.00808
+Validation: Loss 0.00831 Accuracy 1.00000
+Validation: Loss 0.00736 Accuracy 1.00000

    Saving the Model ​

    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.

    julia
    @save "trained_model.jld2" ps_trained st_trained

    Let's try loading the model

    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
  :ps_trained
  :st_trained

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
@@ -636,8 +636,8 @@
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.453 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.422 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    + \ No newline at end of file diff --git a/dev/tutorials/beginner/4_SimpleChains.html b/dev/tutorials/beginner/4_SimpleChains.html index e967d2666d..455fe67999 100644 --- a/dev/tutorials/beginner/4_SimpleChains.html +++ b/dev/tutorials/beginner/4_SimpleChains.html @@ -5,14 +5,14 @@ MNIST Classification with SimpleChains | Lux.jl Docs - - + + - + - + - + @@ -119,25 +119,25 @@ end return tr_acc, te_acc -end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 86.21s 	 Training Accuracy: 22.06% 	 Test Accuracy: 19.00%
-[ 2/10] 	 Time 80.57s 	 Training Accuracy: 47.28% 	 Test Accuracy: 46.50%
-[ 3/10] 	 Time 81.04s 	 Training Accuracy: 60.33% 	 Test Accuracy: 60.00%
-[ 4/10] 	 Time 85.15s 	 Training Accuracy: 69.61% 	 Test Accuracy: 66.50%
-[ 5/10] 	 Time 93.32s 	 Training Accuracy: 75.44% 	 Test Accuracy: 73.50%
-[ 6/10] 	 Time 88.87s 	 Training Accuracy: 79.00% 	 Test Accuracy: 76.50%
-[ 7/10] 	 Time 85.97s 	 Training Accuracy: 81.00% 	 Test Accuracy: 81.50%
-[ 8/10] 	 Time 92.55s 	 Training Accuracy: 83.00% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 109.11s 	 Training Accuracy: 84.89% 	 Test Accuracy: 81.50%
-[10/10] 	 Time 108.74s 	 Training Accuracy: 86.17% 	 Test Accuracy: 85.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.89s 	 Training Accuracy: 30.22% 	 Test Accuracy: 26.00%
-[ 2/10] 	 Time 17.83s 	 Training Accuracy: 46.56% 	 Test Accuracy: 44.50%
-[ 3/10] 	 Time 17.84s 	 Training Accuracy: 62.22% 	 Test Accuracy: 55.00%
-[ 4/10] 	 Time 17.83s 	 Training Accuracy: 66.39% 	 Test Accuracy: 58.50%
-[ 5/10] 	 Time 17.88s 	 Training Accuracy: 73.17% 	 Test Accuracy: 70.00%
-[ 6/10] 	 Time 17.84s 	 Training Accuracy: 78.78% 	 Test Accuracy: 74.50%
-[ 7/10] 	 Time 17.84s 	 Training Accuracy: 82.44% 	 Test Accuracy: 79.00%
-[ 8/10] 	 Time 17.85s 	 Training Accuracy: 84.06% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 17.84s 	 Training Accuracy: 86.50% 	 Test Accuracy: 82.00%
-[10/10] 	 Time 17.84s 	 Training Accuracy: 88.39% 	 Test Accuracy: 85.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
+end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 112.65s 	 Training Accuracy: 21.89% 	 Test Accuracy: 18.50%
+[ 2/10] 	 Time 100.45s 	 Training Accuracy: 46.44% 	 Test Accuracy: 43.00%
+[ 3/10] 	 Time 108.58s 	 Training Accuracy: 59.94% 	 Test Accuracy: 61.50%
+[ 4/10] 	 Time 110.91s 	 Training Accuracy: 67.94% 	 Test Accuracy: 65.50%
+[ 5/10] 	 Time 113.76s 	 Training Accuracy: 73.28% 	 Test Accuracy: 71.50%
+[ 6/10] 	 Time 114.94s 	 Training Accuracy: 77.33% 	 Test Accuracy: 76.00%
+[ 7/10] 	 Time 114.72s 	 Training Accuracy: 80.44% 	 Test Accuracy: 78.50%
+[ 8/10] 	 Time 114.19s 	 Training Accuracy: 83.00% 	 Test Accuracy: 82.50%
+[ 9/10] 	 Time 114.70s 	 Training Accuracy: 84.44% 	 Test Accuracy: 81.00%
+[10/10] 	 Time 113.65s 	 Training Accuracy: 87.06% 	 Test Accuracy: 83.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 19.01s 	 Training Accuracy: 30.39% 	 Test Accuracy: 26.00%
+[ 2/10] 	 Time 17.90s 	 Training Accuracy: 45.67% 	 Test Accuracy: 42.00%
+[ 3/10] 	 Time 17.91s 	 Training Accuracy: 60.89% 	 Test Accuracy: 55.50%
+[ 4/10] 	 Time 17.91s 	 Training Accuracy: 68.28% 	 Test Accuracy: 66.50%
+[ 5/10] 	 Time 17.93s 	 Training Accuracy: 77.39% 	 Test Accuracy: 73.50%
+[ 6/10] 	 Time 17.93s 	 Training Accuracy: 79.17% 	 Test Accuracy: 74.50%
+[ 7/10] 	 Time 17.90s 	 Training Accuracy: 81.28% 	 Test Accuracy: 76.00%
+[ 8/10] 	 Time 17.97s 	 Training Accuracy: 84.33% 	 Test Accuracy: 81.00%
+[ 9/10] 	 Time 17.95s 	 Training Accuracy: 86.22% 	 Test Accuracy: 84.50%
+[10/10] 	 Time 17.92s 	 Training Accuracy: 87.44% 	 Test Accuracy: 83.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -170,7 +170,7 @@
   JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
   JULIA_PKG_PRECOMPILE_AUTO = 0
   JULIA_DEBUG = Literate

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/dev/tutorials/beginner/5_OptimizationIntegration.html b/dev/tutorials/beginner/5_OptimizationIntegration.html index 6121c61e55..590c87ddb8 100644 --- a/dev/tutorials/beginner/5_OptimizationIntegration.html +++ b/dev/tutorials/beginner/5_OptimizationIntegration.html @@ -5,14 +5,14 @@ Training Lux Models using Optimization.jl | Lux.jl Docs - - + + - + - + - + @@ -109,31 +109,31 @@ return StatefulLuxLayer{true}(model, res.u, smodel.st) end -trained_model = train_model(dataloader)
    Iteration:     1, Loss: 2.991784e-01
-Iteration:    26, Loss: 8.993525e-02
-Iteration:    51, Loss: 1.784816e-01
-Iteration:    76, Loss: 2.487762e-01
-Iteration:     1, Loss: 1.481039e-01
-Iteration:     1, Loss: 2.541728e-02
-Iteration:    26, Loss: 3.857330e-02
-Iteration:    51, Loss: 2.923982e-02
-Iteration:    76, Loss: 2.677913e-02
-Iteration:   101, Loss: 2.588445e-02
-Iteration:   126, Loss: 2.508674e-02
-Iteration:   151, Loss: 2.428122e-02
-Iteration:   176, Loss: 2.341577e-02
-Iteration:   201, Loss: 2.247446e-02
-Iteration:   226, Loss: 2.149151e-02
-Iteration:   251, Loss: 2.047292e-02
-Iteration:   276, Loss: 1.935390e-02
-Iteration:   301, Loss: 1.811985e-02
-Iteration:   326, Loss: 1.687357e-02
-Iteration:   351, Loss: 1.563711e-02
-Iteration:   376, Loss: 1.438919e-02
-Iteration:   401, Loss: 1.313944e-02
-Iteration:   426, Loss: 1.188992e-02
-Iteration:   451, Loss: 1.067601e-02
-Iteration:   476, Loss: 1.076645e-02

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
+trained_model = train_model(dataloader)
    Iteration:     1, Loss: 2.966199e-02
+Iteration:    26, Loss: 2.974361e-02
+Iteration:    51, Loss: 1.571838e-01
+Iteration:    76, Loss: 2.171316e-01
+Iteration:     1, Loss: 1.832428e-01
+Iteration:     1, Loss: 1.933464e-02
+Iteration:    26, Loss: 3.901346e-02
+Iteration:    51, Loss: 2.986553e-02
+Iteration:    76, Loss: 2.560794e-02
+Iteration:   101, Loss: 2.308636e-02
+Iteration:   126, Loss: 2.079723e-02
+Iteration:   151, Loss: 1.886188e-02
+Iteration:   176, Loss: 1.737600e-02
+Iteration:   201, Loss: 1.608310e-02
+Iteration:   226, Loss: 1.487726e-02
+Iteration:   251, Loss: 1.359568e-02
+Iteration:   276, Loss: 1.214328e-02
+Iteration:   301, Loss: 1.045338e-02
+Iteration:   326, Loss: 2.002797e-02
+Iteration:   351, Loss: 1.186746e-02
+Iteration:   376, Loss: 9.847032e-03
+Iteration:   401, Loss: 8.622359e-03
+Iteration:   426, Loss: 7.740140e-03
+Iteration:   451, Loss: 6.804725e-03
+Iteration:   476, Loss: 5.482031e-03

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
 prob = ODEProblem(dudt, gdev(u0), (tspan[1], tspan[2]), trained_model.ps)
 sol = solve(prob, Tsit5(); saveat=t)
 pred = convert(AbstractArray, sol) |> cdev
@@ -147,7 +147,7 @@
     lines!(ax, t, pred[2, :]; label=L"\hat{u}_2(t)", color=:red, linewidth=4)
     axislegend(ax; position=:lt)
     fig
-end

    Appendix ​

    julia
    using InteractiveUtils
+end

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -207,8 +207,8 @@
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.359 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    + \ No newline at end of file diff --git a/dev/tutorials/index.html b/dev/tutorials/index.html index 80e27f3ed7..7301b7e2ff 100644 --- a/dev/tutorials/index.html +++ b/dev/tutorials/index.html @@ -5,12 +5,12 @@ Tutorials | Lux.jl Docs - - + + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Tutorials ​

    Beginner Tutorials ​

    Julia & Lux for the Uninitiated

    How to get started with Julia and Lux for those who have never used Julia before.

    Fitting a Polynomial using MLP

    Learn the Basics of Lux by fitting a Multi-Layer Perceptron to a Polynomial.

    Training a Simple LSTM

    Learn how to define custom layers and train an RNN on time-series data.

    Use SimpleChains.jl as a Backend

    Learn how to train small neural networks really fast on CPU.

    Fitting with Optimization.jl

    Learn how to use Optimization.jl with Lux (on GPUs).

    Pre-Built Deep Learning Models

    Use Boltz.jl to load pre-built DL and SciML models.

    Intermediate Tutorials ​

    MNIST Classification using Neural ODE

    Train a Neural Ordinary Differential Equations to classify MNIST Images.

    Bayesian Neural Networks

    Figure out how to use Probabilistic Programming Frameworks like Turing with Lux.

    Training a HyperNetwork

    Train a hypernetwork to work on multiple datasets by predicting NN parameters.

    Training a PINN

    Train a PINN to solve 2D PDEs (using Nested AD).

    Advanced Tutorials ​

    Neural ODE to Model Gravitational Waveforms

    Training a Neural ODE to fit simulated data of gravitational waveforms.

    Optimal Control with Symbolic UDE

    Train a UDE and replace a part of it with Symbolic Regression.

    Larger Models ​

    WARNING

    These models are part of the Lux examples, however, these are larger model that cannot be run on CI and aren't frequently tested. If you find a bug in one of these models, please open an issue or PR to fix it.

    ImageNet Classification

    Train Large Image Classifiers using Lux (on Distributed GPUs).

    Denoising Diffusion Implicit Model (DDIM)

    Train a Diffusion Model to generate images from Gaussian noises.

    ConvMixer on CIFAR-10

    Train ConvMixer on CIFAR-10 to 90% accuracy within 10 minutes.

    Selected 3rd Party Tutorials ​

    WARNING

    These tutorials are developed by the community and may not be up-to-date with the latest version of Lux.jl. Please refer to the official documentation for the most up-to-date information.

    Please open an issue (ideally both at Lux.jl and at the downstream linked package) if any of them are non-functional and we will try to get them updated.

    GPU-Accelerated Physics-Informed Neural Networks

    Use Machine Learning (PINNs) to solve the Heat Equation PDE on a GPU.

    Weather Forecasting with Neural ODEs

    Train a neural ODEs to a multidimensional weather dataset and use it for weather forecasting.

    Controlling Stochastic Differential Equations

    Control the time evolution of a continuously monitored qubit described by an SDE with multiplicative scalar noise.

    Medical Image Segmentation

    Explore various aspects of deep learning for medical imaging and a comprehensive overview of Julia packages.

    Neural PDE closures

    Learn an unknown term in a PDE using convolutional neural networks and Fourier neural operators.

    TIP

    If you found an amazing tutorial showcasing Lux.jl online, or wrote one yourself, please open an issue or PR to add it to the list!

    - + \ No newline at end of file diff --git a/dev/tutorials/intermediate/1_NeuralODE.html b/dev/tutorials/intermediate/1_NeuralODE.html index 448bc9e272..d5c2a049fd 100644 --- a/dev/tutorials/intermediate/1_NeuralODE.html +++ b/dev/tutorials/intermediate/1_NeuralODE.html @@ -5,14 +5,14 @@ MNIST Classification using Neural ODEs | Lux.jl Docs - - + + - + - + - + @@ -136,47 +136,47 @@ end end -train(NeuralODECompact)
    [1/9]	Time 117.1921s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.5168s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
-[3/9]	Time 0.6670s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
-[4/9]	Time 0.4899s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
-[5/9]	Time 0.5107s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
-[6/9]	Time 0.7411s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
-[7/9]	Time 0.4911s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
-[8/9]	Time 0.4896s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
-[9/9]	Time 0.7356s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 35.1691s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4975s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5030s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
-[4/9]	Time 0.4983s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
-[5/9]	Time 0.7347s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
-[6/9]	Time 0.4963s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
-[7/9]	Time 0.5021s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
-[8/9]	Time 0.7346s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
-[9/9]	Time 0.4987s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 41.5691s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4795s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
-[3/9]	Time 0.4873s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
-[4/9]	Time 0.7266s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
-[5/9]	Time 0.4854s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
-[6/9]	Time 0.4804s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
-[7/9]	Time 0.7027s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
-[8/9]	Time 0.4765s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
-[9/9]	Time 0.4773s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 102.8680s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 14.2136s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
-[3/9]	Time 19.1814s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
-[4/9]	Time 17.8224s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
-[5/9]	Time 21.2390s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
-[6/9]	Time 18.3580s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 17.0590s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
-[8/9]	Time 20.5356s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
-[9/9]	Time 26.9693s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 50.5271s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 22.8199s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
-[3/9]	Time 23.3855s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
-[4/9]	Time 30.0392s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
-[5/9]	Time 29.7560s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
-[6/9]	Time 28.8348s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 30.3525s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
-[8/9]	Time 31.6796s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
-[9/9]	Time 25.7284s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
+train(NeuralODECompact)
    [1/9]	Time 119.9114s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4894s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
+[3/9]	Time 0.5984s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
+[4/9]	Time 0.4816s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
+[5/9]	Time 0.4886s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
+[6/9]	Time 0.6129s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
+[7/9]	Time 0.4809s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
+[8/9]	Time 0.6183s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
+[9/9]	Time 0.4846s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 36.1949s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4934s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
+[3/9]	Time 0.6716s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
+[4/9]	Time 0.4907s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
+[5/9]	Time 0.5005s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
+[6/9]	Time 0.5111s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
+[7/9]	Time 0.5156s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
+[8/9]	Time 0.5218s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4958s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 42.7072s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4585s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
+[3/9]	Time 0.5124s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
+[4/9]	Time 0.6816s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
+[5/9]	Time 0.4651s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
+[6/9]	Time 0.4662s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
+[7/9]	Time 0.7050s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
+[8/9]	Time 0.4780s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
+[9/9]	Time 0.4721s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 99.0023s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 16.3657s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
+[3/9]	Time 16.9835s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
+[4/9]	Time 19.8992s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
+[5/9]	Time 19.0555s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
+[6/9]	Time 18.0183s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 19.1584s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
+[8/9]	Time 19.9653s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
+[9/9]	Time 21.4458s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 52.5981s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 28.1671s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
+[3/9]	Time 29.9169s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
+[4/9]	Time 30.6306s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
+[5/9]	Time 27.4726s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
+[6/9]	Time 27.3794s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 27.8209s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
+[8/9]	Time 21.9799s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
+[9/9]	Time 29.7374s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
        Lux.AbstractLuxWrapperLayer{:model}
     model::M
     solver::So
@@ -194,15 +194,15 @@
     dudt(u, p, t) = st_model(u, p)
     prob = ODEProblem{false}(ODEFunction{false}(dudt), x, n.tspan, ps)
     return solve(prob, n.solver; n.kwargs...), st_model.st
-end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 38.1173s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4742s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
-[3/9]	Time 0.5017s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
-[4/9]	Time 0.7221s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
-[5/9]	Time 0.4642s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
-[6/9]	Time 0.4642s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
-[7/9]	Time 0.4763s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
-[8/9]	Time 0.7946s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
-[9/9]	Time 0.4674s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
+end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 39.1820s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4664s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
+[3/9]	Time 0.6511s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
+[4/9]	Time 0.4597s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
+[5/9]	Time 0.4650s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
+[6/9]	Time 0.4761s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
+[7/9]	Time 0.7648s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
+[8/9]	Time 0.4612s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4605s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
 
 model_stateful, ps_stateful, st_stateful = create_model(StatefulNeuralODE)
 
@@ -298,8 +298,8 @@
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.889 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.920 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    + \ No newline at end of file diff --git a/dev/tutorials/intermediate/2_BayesianNN.html b/dev/tutorials/intermediate/2_BayesianNN.html index 83c3c683c6..7b8843d64c 100644 --- a/dev/tutorials/intermediate/2_BayesianNN.html +++ b/dev/tutorials/intermediate/2_BayesianNN.html @@ -5,14 +5,14 @@ Bayesian Neural Network | Lux.jl Docs - - + + - + - + - + @@ -114,8 +114,8 @@ Iterations = 1:1:5000 Number of chains = 1 Samples per chain = 5000 -Wall duration = 20.89 seconds -Compute duration = 20.89 seconds +Wall duration = 21.82 seconds +Compute duration = 21.82 seconds parameters = parameters[1], parameters[2], parameters[3], parameters[4], parameters[5], parameters[6], parameters[7], parameters[8], parameters[9], parameters[10], parameters[11], parameters[12], parameters[13], parameters[14], parameters[15], parameters[16], parameters[17], parameters[18], parameters[19], parameters[20] internals = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, numerical_error, step_size, nom_step_size @@ -123,26 +123,26 @@ parameters mean std mcse ess_bulk ess_tail rhat ess_per_sec Symbol Float64 Float64 Float64 Float64 Float64 Float64 Float64 - parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.5787 - parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 3.8271 - parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.9285 - parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.6877 - parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.7213 - parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 1.0398 - parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.5957 - parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 1.2539 - parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.6770 - parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.6744 - parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.8775 - parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 1.1049 - parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.9310 - parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.6869 - parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.7805 - parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.5017 - parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.7504 - parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 2.1515 - parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.8464 - parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.4596 + parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.5541 + parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 3.6645 + parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.8890 + parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.6585 + parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.6907 + parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 0.9956 + parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.5703 + parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 1.2006 + parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.6482 + parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.6457 + parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.8402 + parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 1.0579 + parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.8915 + parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.6578 + parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.7474 + parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.4379 + parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.7186 + parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 2.0601 + parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.8104 + parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.3976 Quantiles parameters 2.5% 25.0% 50.0% 75.0% 97.5% @@ -233,7 +233,7 @@ JULIA_CUDA_HARD_MEMORY_LIMIT = 100% JULIA_PKG_PRECOMPILE_AUTO = 0 JULIA_DEBUG = Literate

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/dev/tutorials/intermediate/3_HyperNet.html b/dev/tutorials/intermediate/3_HyperNet.html index 03c319333e..8a3b447733 100644 --- a/dev/tutorials/intermediate/3_HyperNet.html +++ b/dev/tutorials/intermediate/3_HyperNet.html @@ -5,14 +5,14 @@ Training a HyperNetwork on MNIST and FashionMNIST | Lux.jl Docs - - + + - + - + - + @@ -136,109 +136,109 @@ return test_acc_list end -test_acc_list = train()
    [  1/ 50]	       MNIST	Time 67.83115s	Training Accuracy: 57.71%	Test Accuracy: 59.38%
-[  1/ 50]	FashionMNIST	Time 0.02742s	Training Accuracy: 51.17%	Test Accuracy: 53.12%
-[  2/ 50]	       MNIST	Time 0.02788s	Training Accuracy: 67.48%	Test Accuracy: 65.62%
-[  2/ 50]	FashionMNIST	Time 0.02874s	Training Accuracy: 63.87%	Test Accuracy: 68.75%
-[  3/ 50]	       MNIST	Time 0.02629s	Training Accuracy: 71.48%	Test Accuracy: 65.62%
-[  3/ 50]	FashionMNIST	Time 0.02589s	Training Accuracy: 68.85%	Test Accuracy: 59.38%
-[  4/ 50]	       MNIST	Time 0.02471s	Training Accuracy: 79.98%	Test Accuracy: 62.50%
-[  4/ 50]	FashionMNIST	Time 0.02508s	Training Accuracy: 71.09%	Test Accuracy: 71.88%
-[  5/ 50]	       MNIST	Time 0.02502s	Training Accuracy: 85.84%	Test Accuracy: 65.62%
-[  5/ 50]	FashionMNIST	Time 0.03813s	Training Accuracy: 76.17%	Test Accuracy: 68.75%
-[  6/ 50]	       MNIST	Time 0.02162s	Training Accuracy: 87.70%	Test Accuracy: 59.38%
-[  6/ 50]	FashionMNIST	Time 0.02053s	Training Accuracy: 79.88%	Test Accuracy: 75.00%
-[  7/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 91.11%	Test Accuracy: 71.88%
-[  7/ 50]	FashionMNIST	Time 0.02198s	Training Accuracy: 81.74%	Test Accuracy: 68.75%
-[  8/ 50]	       MNIST	Time 0.02085s	Training Accuracy: 93.95%	Test Accuracy: 71.88%
-[  8/ 50]	FashionMNIST	Time 0.02078s	Training Accuracy: 82.62%	Test Accuracy: 75.00%
-[  9/ 50]	       MNIST	Time 0.02036s	Training Accuracy: 95.41%	Test Accuracy: 71.88%
-[  9/ 50]	FashionMNIST	Time 0.02046s	Training Accuracy: 79.98%	Test Accuracy: 68.75%
-[ 10/ 50]	       MNIST	Time 0.03479s	Training Accuracy: 96.68%	Test Accuracy: 75.00%
-[ 10/ 50]	FashionMNIST	Time 0.02228s	Training Accuracy: 84.57%	Test Accuracy: 75.00%
-[ 11/ 50]	       MNIST	Time 0.02177s	Training Accuracy: 97.27%	Test Accuracy: 75.00%
-[ 11/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 84.38%	Test Accuracy: 78.12%
-[ 12/ 50]	       MNIST	Time 0.02053s	Training Accuracy: 97.85%	Test Accuracy: 71.88%
-[ 12/ 50]	FashionMNIST	Time 0.02042s	Training Accuracy: 85.74%	Test Accuracy: 75.00%
-[ 13/ 50]	       MNIST	Time 0.02046s	Training Accuracy: 97.75%	Test Accuracy: 68.75%
-[ 13/ 50]	FashionMNIST	Time 0.03820s	Training Accuracy: 85.84%	Test Accuracy: 75.00%
-[ 14/ 50]	       MNIST	Time 0.04972s	Training Accuracy: 98.54%	Test Accuracy: 75.00%
-[ 14/ 50]	FashionMNIST	Time 0.02482s	Training Accuracy: 87.30%	Test Accuracy: 75.00%
-[ 15/ 50]	       MNIST	Time 0.06350s	Training Accuracy: 98.34%	Test Accuracy: 71.88%
-[ 15/ 50]	FashionMNIST	Time 0.04444s	Training Accuracy: 85.16%	Test Accuracy: 78.12%
-[ 16/ 50]	       MNIST	Time 0.03805s	Training Accuracy: 99.32%	Test Accuracy: 75.00%
-[ 16/ 50]	FashionMNIST	Time 0.02524s	Training Accuracy: 84.96%	Test Accuracy: 68.75%
-[ 17/ 50]	       MNIST	Time 0.02049s	Training Accuracy: 99.32%	Test Accuracy: 75.00%
-[ 17/ 50]	FashionMNIST	Time 0.02056s	Training Accuracy: 86.82%	Test Accuracy: 75.00%
-[ 18/ 50]	       MNIST	Time 0.02162s	Training Accuracy: 99.61%	Test Accuracy: 75.00%
-[ 18/ 50]	FashionMNIST	Time 0.02066s	Training Accuracy: 84.47%	Test Accuracy: 68.75%
-[ 19/ 50]	       MNIST	Time 0.02218s	Training Accuracy: 99.80%	Test Accuracy: 75.00%
-[ 19/ 50]	FashionMNIST	Time 0.02053s	Training Accuracy: 86.62%	Test Accuracy: 71.88%
-[ 20/ 50]	       MNIST	Time 0.02384s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 20/ 50]	FashionMNIST	Time 0.02117s	Training Accuracy: 86.13%	Test Accuracy: 71.88%
-[ 21/ 50]	       MNIST	Time 0.02141s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 21/ 50]	FashionMNIST	Time 0.02030s	Training Accuracy: 85.35%	Test Accuracy: 78.12%
-[ 22/ 50]	       MNIST	Time 0.02055s	Training Accuracy: 99.90%	Test Accuracy: 75.00%
-[ 22/ 50]	FashionMNIST	Time 0.02162s	Training Accuracy: 87.30%	Test Accuracy: 71.88%
-[ 23/ 50]	       MNIST	Time 0.03429s	Training Accuracy: 99.71%	Test Accuracy: 75.00%
-[ 23/ 50]	FashionMNIST	Time 0.02119s	Training Accuracy: 87.40%	Test Accuracy: 78.12%
-[ 24/ 50]	       MNIST	Time 0.02052s	Training Accuracy: 99.71%	Test Accuracy: 75.00%
-[ 24/ 50]	FashionMNIST	Time 0.02107s	Training Accuracy: 87.70%	Test Accuracy: 71.88%
-[ 25/ 50]	       MNIST	Time 0.02111s	Training Accuracy: 99.90%	Test Accuracy: 75.00%
-[ 25/ 50]	FashionMNIST	Time 0.02145s	Training Accuracy: 87.99%	Test Accuracy: 81.25%
-[ 26/ 50]	       MNIST	Time 0.02167s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 26/ 50]	FashionMNIST	Time 0.02121s	Training Accuracy: 88.67%	Test Accuracy: 78.12%
-[ 27/ 50]	       MNIST	Time 0.02237s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 27/ 50]	FashionMNIST	Time 0.03032s	Training Accuracy: 89.45%	Test Accuracy: 78.12%
-[ 28/ 50]	       MNIST	Time 0.02097s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 28/ 50]	FashionMNIST	Time 0.02132s	Training Accuracy: 90.43%	Test Accuracy: 81.25%
-[ 29/ 50]	       MNIST	Time 0.02005s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 29/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 91.50%	Test Accuracy: 81.25%
-[ 30/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 30/ 50]	FashionMNIST	Time 0.02047s	Training Accuracy: 91.99%	Test Accuracy: 78.12%
-[ 31/ 50]	       MNIST	Time 0.02050s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 31/ 50]	FashionMNIST	Time 0.02051s	Training Accuracy: 92.19%	Test Accuracy: 81.25%
-[ 32/ 50]	       MNIST	Time 0.02583s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 32/ 50]	FashionMNIST	Time 0.02268s	Training Accuracy: 93.36%	Test Accuracy: 78.12%
-[ 33/ 50]	       MNIST	Time 0.02040s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 33/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 92.77%	Test Accuracy: 75.00%
-[ 34/ 50]	       MNIST	Time 0.02076s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 34/ 50]	FashionMNIST	Time 0.02091s	Training Accuracy: 94.04%	Test Accuracy: 78.12%
-[ 35/ 50]	       MNIST	Time 0.02073s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 35/ 50]	FashionMNIST	Time 0.02027s	Training Accuracy: 94.04%	Test Accuracy: 78.12%
-[ 36/ 50]	       MNIST	Time 0.02058s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 36/ 50]	FashionMNIST	Time 0.01989s	Training Accuracy: 94.73%	Test Accuracy: 81.25%
-[ 37/ 50]	       MNIST	Time 0.02057s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 37/ 50]	FashionMNIST	Time 0.02052s	Training Accuracy: 95.02%	Test Accuracy: 81.25%
-[ 38/ 50]	       MNIST	Time 0.02024s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 38/ 50]	FashionMNIST	Time 0.02059s	Training Accuracy: 95.61%	Test Accuracy: 78.12%
-[ 39/ 50]	       MNIST	Time 0.02044s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 39/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 95.51%	Test Accuracy: 81.25%
-[ 40/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 40/ 50]	FashionMNIST	Time 0.02694s	Training Accuracy: 95.70%	Test Accuracy: 84.38%
-[ 41/ 50]	       MNIST	Time 0.02009s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 41/ 50]	FashionMNIST	Time 0.02056s	Training Accuracy: 96.19%	Test Accuracy: 84.38%
-[ 42/ 50]	       MNIST	Time 0.02053s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 42/ 50]	FashionMNIST	Time 0.02058s	Training Accuracy: 96.19%	Test Accuracy: 84.38%
-[ 43/ 50]	       MNIST	Time 0.02069s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 43/ 50]	FashionMNIST	Time 0.02049s	Training Accuracy: 96.58%	Test Accuracy: 84.38%
-[ 44/ 50]	       MNIST	Time 0.02184s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 44/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 96.39%	Test Accuracy: 84.38%
-[ 45/ 50]	       MNIST	Time 0.02675s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 45/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 96.88%	Test Accuracy: 84.38%
-[ 46/ 50]	       MNIST	Time 0.02074s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 46/ 50]	FashionMNIST	Time 0.02048s	Training Accuracy: 96.97%	Test Accuracy: 81.25%
-[ 47/ 50]	       MNIST	Time 0.02057s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 47/ 50]	FashionMNIST	Time 0.01975s	Training Accuracy: 96.88%	Test Accuracy: 84.38%
-[ 48/ 50]	       MNIST	Time 0.02605s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 48/ 50]	FashionMNIST	Time 0.02151s	Training Accuracy: 97.17%	Test Accuracy: 81.25%
-[ 49/ 50]	       MNIST	Time 0.02068s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 49/ 50]	FashionMNIST	Time 0.04558s	Training Accuracy: 96.97%	Test Accuracy: 84.38%
-[ 50/ 50]	       MNIST	Time 0.04517s	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[ 50/ 50]	FashionMNIST	Time 0.05537s	Training Accuracy: 97.36%	Test Accuracy: 84.38%
+test_acc_list = train()
    [  1/ 50]	       MNIST	Time 69.55185s	Training Accuracy: 62.70%	Test Accuracy: 56.25%
+[  1/ 50]	FashionMNIST	Time 0.01251s	Training Accuracy: 45.90%	Test Accuracy: 37.50%
+[  2/ 50]	       MNIST	Time 0.01233s	Training Accuracy: 72.85%	Test Accuracy: 65.62%
+[  2/ 50]	FashionMNIST	Time 0.01215s	Training Accuracy: 56.84%	Test Accuracy: 50.00%
+[  3/ 50]	       MNIST	Time 0.01216s	Training Accuracy: 74.80%	Test Accuracy: 56.25%
+[  3/ 50]	FashionMNIST	Time 0.01212s	Training Accuracy: 63.18%	Test Accuracy: 62.50%
+[  4/ 50]	       MNIST	Time 0.01212s	Training Accuracy: 83.59%	Test Accuracy: 68.75%
+[  4/ 50]	FashionMNIST	Time 0.01207s	Training Accuracy: 68.26%	Test Accuracy: 59.38%
+[  5/ 50]	       MNIST	Time 0.01219s	Training Accuracy: 83.98%	Test Accuracy: 75.00%
+[  5/ 50]	FashionMNIST	Time 0.01210s	Training Accuracy: 71.39%	Test Accuracy: 53.12%
+[  6/ 50]	       MNIST	Time 0.01214s	Training Accuracy: 87.01%	Test Accuracy: 75.00%
+[  6/ 50]	FashionMNIST	Time 0.03375s	Training Accuracy: 77.73%	Test Accuracy: 65.62%
+[  7/ 50]	       MNIST	Time 0.01088s	Training Accuracy: 88.09%	Test Accuracy: 71.88%
+[  7/ 50]	FashionMNIST	Time 0.01087s	Training Accuracy: 76.86%	Test Accuracy: 68.75%
+[  8/ 50]	       MNIST	Time 0.01083s	Training Accuracy: 92.48%	Test Accuracy: 75.00%
+[  8/ 50]	FashionMNIST	Time 0.01082s	Training Accuracy: 78.61%	Test Accuracy: 71.88%
+[  9/ 50]	       MNIST	Time 0.01074s	Training Accuracy: 93.26%	Test Accuracy: 78.12%
+[  9/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 81.25%	Test Accuracy: 71.88%
+[ 10/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 95.51%	Test Accuracy: 75.00%
+[ 10/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 82.71%	Test Accuracy: 68.75%
+[ 11/ 50]	       MNIST	Time 0.01072s	Training Accuracy: 96.68%	Test Accuracy: 71.88%
+[ 11/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 84.57%	Test Accuracy: 75.00%
+[ 12/ 50]	       MNIST	Time 0.01218s	Training Accuracy: 98.05%	Test Accuracy: 75.00%
+[ 12/ 50]	FashionMNIST	Time 0.03001s	Training Accuracy: 85.35%	Test Accuracy: 78.12%
+[ 13/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 98.73%	Test Accuracy: 75.00%
+[ 13/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 85.35%	Test Accuracy: 75.00%
+[ 14/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 98.73%	Test Accuracy: 81.25%
+[ 14/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 87.79%	Test Accuracy: 78.12%
+[ 15/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 99.12%	Test Accuracy: 75.00%
+[ 15/ 50]	FashionMNIST	Time 0.01060s	Training Accuracy: 88.96%	Test Accuracy: 75.00%
+[ 16/ 50]	       MNIST	Time 0.01060s	Training Accuracy: 99.12%	Test Accuracy: 81.25%
+[ 16/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 83.11%	Test Accuracy: 62.50%
+[ 17/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 99.22%	Test Accuracy: 75.00%
+[ 17/ 50]	FashionMNIST	Time 0.01063s	Training Accuracy: 84.28%	Test Accuracy: 78.12%
+[ 18/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 99.22%	Test Accuracy: 84.38%
+[ 18/ 50]	FashionMNIST	Time 0.02054s	Training Accuracy: 85.74%	Test Accuracy: 78.12%
+[ 19/ 50]	       MNIST	Time 0.01081s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 19/ 50]	FashionMNIST	Time 0.01073s	Training Accuracy: 87.60%	Test Accuracy: 78.12%
+[ 20/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 20/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 85.64%	Test Accuracy: 71.88%
+[ 21/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
+[ 21/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 87.79%	Test Accuracy: 75.00%
+[ 22/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 22/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 89.16%	Test Accuracy: 78.12%
+[ 23/ 50]	       MNIST	Time 0.01064s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 23/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 90.82%	Test Accuracy: 75.00%
+[ 24/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 24/ 50]	FashionMNIST	Time 0.02139s	Training Accuracy: 91.89%	Test Accuracy: 75.00%
+[ 25/ 50]	       MNIST	Time 0.01081s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 25/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 92.58%	Test Accuracy: 81.25%
+[ 26/ 50]	       MNIST	Time 0.01069s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 26/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 92.97%	Test Accuracy: 78.12%
+[ 27/ 50]	       MNIST	Time 0.01063s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 27/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 93.85%	Test Accuracy: 78.12%
+[ 28/ 50]	       MNIST	Time 0.01063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 28/ 50]	FashionMNIST	Time 0.01061s	Training Accuracy: 93.36%	Test Accuracy: 84.38%
+[ 29/ 50]	       MNIST	Time 0.01062s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 29/ 50]	FashionMNIST	Time 0.01066s	Training Accuracy: 91.80%	Test Accuracy: 75.00%
+[ 30/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 30/ 50]	FashionMNIST	Time 0.01107s	Training Accuracy: 92.38%	Test Accuracy: 84.38%
+[ 31/ 50]	       MNIST	Time 0.01085s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 31/ 50]	FashionMNIST	Time 0.01073s	Training Accuracy: 94.14%	Test Accuracy: 81.25%
+[ 32/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 32/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 94.63%	Test Accuracy: 84.38%
+[ 33/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 33/ 50]	FashionMNIST	Time 0.01070s	Training Accuracy: 95.51%	Test Accuracy: 84.38%
+[ 34/ 50]	       MNIST	Time 0.01071s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 34/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 96.19%	Test Accuracy: 81.25%
+[ 35/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 35/ 50]	FashionMNIST	Time 0.01067s	Training Accuracy: 96.09%	Test Accuracy: 81.25%
+[ 36/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 36/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 97.07%	Test Accuracy: 87.50%
+[ 37/ 50]	       MNIST	Time 0.01079s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 37/ 50]	FashionMNIST	Time 0.01177s	Training Accuracy: 97.46%	Test Accuracy: 81.25%
+[ 38/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 38/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 97.17%	Test Accuracy: 84.38%
+[ 39/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 39/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.56%	Test Accuracy: 84.38%
+[ 40/ 50]	       MNIST	Time 0.01066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 40/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.36%	Test Accuracy: 87.50%
+[ 41/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 41/ 50]	FashionMNIST	Time 0.01065s	Training Accuracy: 97.36%	Test Accuracy: 84.38%
+[ 42/ 50]	       MNIST	Time 0.01067s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 42/ 50]	FashionMNIST	Time 0.01064s	Training Accuracy: 97.66%	Test Accuracy: 84.38%
+[ 43/ 50]	       MNIST	Time 0.01086s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 43/ 50]	FashionMNIST	Time 0.01189s	Training Accuracy: 97.46%	Test Accuracy: 84.38%
+[ 44/ 50]	       MNIST	Time 0.01077s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 44/ 50]	FashionMNIST	Time 0.01074s	Training Accuracy: 97.85%	Test Accuracy: 84.38%
+[ 45/ 50]	       MNIST	Time 0.01072s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 45/ 50]	FashionMNIST	Time 0.01070s	Training Accuracy: 98.05%	Test Accuracy: 84.38%
+[ 46/ 50]	       MNIST	Time 0.01070s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 46/ 50]	FashionMNIST	Time 0.01071s	Training Accuracy: 97.75%	Test Accuracy: 87.50%
+[ 47/ 50]	       MNIST	Time 0.01068s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 47/ 50]	FashionMNIST	Time 0.01069s	Training Accuracy: 98.54%	Test Accuracy: 81.25%
+[ 48/ 50]	       MNIST	Time 0.01065s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 48/ 50]	FashionMNIST	Time 0.01068s	Training Accuracy: 98.34%	Test Accuracy: 84.38%
+[ 49/ 50]	       MNIST	Time 0.02206s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 49/ 50]	FashionMNIST	Time 0.01183s	Training Accuracy: 98.54%	Test Accuracy: 84.38%
+[ 50/ 50]	       MNIST	Time 0.01088s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 50/ 50]	FashionMNIST	Time 0.01084s	Training Accuracy: 98.63%	Test Accuracy: 84.38%
 
-[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 75.00%
-[FINAL]	FashionMNIST	Training Accuracy: 97.36%	Test Accuracy: 84.38%

    Appendix ​

    julia
    using InteractiveUtils
+[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[FINAL]	FashionMNIST	Training Accuracy: 98.63%	Test Accuracy: 84.38%

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -297,9 +297,10 @@
 Environment:
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
-1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 2.170 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - +2 devices: + 0: Quadro RTX 5000 (sm_75, 12.100 GiB / 16.000 GiB available) + 1: Quadro RTX 5000 (sm_75, 15.549 GiB / 16.000 GiB available)

    This page was generated using Literate.jl.

    + \ No newline at end of file diff --git a/dev/tutorials/intermediate/4_PINN2DPDE.html b/dev/tutorials/intermediate/4_PINN2DPDE.html index d35ba28276..2e190bcd81 100644 --- a/dev/tutorials/intermediate/4_PINN2DPDE.html +++ b/dev/tutorials/intermediate/4_PINN2DPDE.html @@ -5,14 +5,14 @@ Training a PINN on 2D PDE | Lux.jl Docs - - + + - + - + - + @@ -176,116 +176,116 @@ trained_u = Lux.testmode(StatefulLuxLayer{true}( trained_model.model.u, trained_model.ps.u, trained_model.st.u))
    ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
-Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162476 (1.982162476) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505080 (0.598505080)
-Iteration: [  501 / 50000] 	 Loss: 0.040918160 (0.025583776) 	 Physics Loss: 0.000391877 (0.000269295) 	 Data Loss: 0.014243508 (0.009196416) 	 BC Loss: 0.026282774 (0.016118063)
-Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281426) 	 Physics Loss: 0.000071670 (0.000163182) 	 Data Loss: 0.007905648 (0.010876314) 	 BC Loss: 0.007363341 (0.014241929)
-Iteration: [ 1501 / 50000] 	 Loss: 0.019567011 (0.026170366) 	 Physics Loss: 0.001279382 (0.001009728) 	 Data Loss: 0.003071612 (0.010452257) 	 BC Loss: 0.015216017 (0.014708381)
-Iteration: [ 2001 / 50000] 	 Loss: 0.035556547 (0.027273500) 	 Physics Loss: 0.004061943 (0.001871931) 	 Data Loss: 0.013011228 (0.010586374) 	 BC Loss: 0.018483378 (0.014815190)
-Iteration: [ 2501 / 50000] 	 Loss: 0.011505678 (0.022150228) 	 Physics Loss: 0.002304791 (0.001940841) 	 Data Loss: 0.005615299 (0.007885863) 	 BC Loss: 0.003585588 (0.012323526)
-Iteration: [ 3001 / 50000] 	 Loss: 0.031768262 (0.029164422) 	 Physics Loss: 0.008404830 (0.003920683) 	 Data Loss: 0.008808360 (0.010936455) 	 BC Loss: 0.014555071 (0.014307282)
-Iteration: [ 3501 / 50000] 	 Loss: 0.017645847 (0.042499166) 	 Physics Loss: 0.001848093 (0.001690981) 	 Data Loss: 0.005461216 (0.018073166) 	 BC Loss: 0.010336538 (0.022735020)
-Iteration: [ 4001 / 50000] 	 Loss: 0.028128654 (0.027620461) 	 Physics Loss: 0.005112350 (0.002448601) 	 Data Loss: 0.013499700 (0.011073254) 	 BC Loss: 0.009516605 (0.014098606)
-Iteration: [ 4501 / 50000] 	 Loss: 0.014340003 (0.033320315) 	 Physics Loss: 0.001292084 (0.004329988) 	 Data Loss: 0.008556721 (0.012002973) 	 BC Loss: 0.004491198 (0.016987354)
-Iteration: [ 5001 / 50000] 	 Loss: 0.030331207 (0.041541956) 	 Physics Loss: 0.000723805 (0.002695386) 	 Data Loss: 0.004466736 (0.016228525) 	 BC Loss: 0.025140665 (0.022618050)
-Iteration: [ 5501 / 50000] 	 Loss: 0.022293953 (0.021166507) 	 Physics Loss: 0.000554349 (0.000707158) 	 Data Loss: 0.004542338 (0.007950513) 	 BC Loss: 0.017197266 (0.012508835)
-Iteration: [ 6001 / 50000] 	 Loss: 0.018723227 (0.020389127) 	 Physics Loss: 0.000442930 (0.000975201) 	 Data Loss: 0.006906096 (0.007273634) 	 BC Loss: 0.011374202 (0.012140292)
-Iteration: [ 6501 / 50000] 	 Loss: 0.028305896 (0.020493284) 	 Physics Loss: 0.000753467 (0.001849154) 	 Data Loss: 0.011062279 (0.008014920) 	 BC Loss: 0.016490150 (0.010629211)
-Iteration: [ 7001 / 50000] 	 Loss: 0.015494239 (0.020624701) 	 Physics Loss: 0.007333768 (0.001517879) 	 Data Loss: 0.004043682 (0.008185850) 	 BC Loss: 0.004116789 (0.010920972)
-Iteration: [ 7501 / 50000] 	 Loss: 0.019155467 (0.018008159) 	 Physics Loss: 0.002511769 (0.001448493) 	 Data Loss: 0.009514628 (0.006572613) 	 BC Loss: 0.007129070 (0.009987052)
-Iteration: [ 8001 / 50000] 	 Loss: 0.018706074 (0.017215939) 	 Physics Loss: 0.001792779 (0.001894138) 	 Data Loss: 0.005628760 (0.005433898) 	 BC Loss: 0.011284535 (0.009887908)
-Iteration: [ 8501 / 50000] 	 Loss: 0.017605182 (0.019859709) 	 Physics Loss: 0.002696904 (0.003125851) 	 Data Loss: 0.007372384 (0.005978104) 	 BC Loss: 0.007535893 (0.010755754)
-Iteration: [ 9001 / 50000] 	 Loss: 0.017614847 (0.016386209) 	 Physics Loss: 0.003162773 (0.002853032) 	 Data Loss: 0.003553676 (0.004607514) 	 BC Loss: 0.010898398 (0.008925664)
-Iteration: [ 9501 / 50000] 	 Loss: 0.008732248 (0.014530762) 	 Physics Loss: 0.001913586 (0.002123826) 	 Data Loss: 0.003329928 (0.004253434) 	 BC Loss: 0.003488734 (0.008153505)
-Iteration: [10001 / 50000] 	 Loss: 0.017750096 (0.017799046) 	 Physics Loss: 0.003630795 (0.003061282) 	 Data Loss: 0.003031955 (0.005044522) 	 BC Loss: 0.011087346 (0.009693242)
-Iteration: [10501 / 50000] 	 Loss: 0.007509941 (0.011342090) 	 Physics Loss: 0.001571818 (0.001135202) 	 Data Loss: 0.001611185 (0.002613829) 	 BC Loss: 0.004326937 (0.007593059)
-Iteration: [11001 / 50000] 	 Loss: 0.022739200 (0.011915382) 	 Physics Loss: 0.000807529 (0.001103305) 	 Data Loss: 0.002606506 (0.003132161) 	 BC Loss: 0.019325165 (0.007679918)
-Iteration: [11501 / 50000] 	 Loss: 0.019484218 (0.011457845) 	 Physics Loss: 0.001845157 (0.001325290) 	 Data Loss: 0.002358936 (0.002806861) 	 BC Loss: 0.015280124 (0.007325693)
-Iteration: [12001 / 50000] 	 Loss: 0.019427970 (0.011598296) 	 Physics Loss: 0.004434146 (0.001386037) 	 Data Loss: 0.008772802 (0.003136263) 	 BC Loss: 0.006221022 (0.007075997)
-Iteration: [12501 / 50000] 	 Loss: 0.012775686 (0.011906523) 	 Physics Loss: 0.001197650 (0.001402911) 	 Data Loss: 0.001059842 (0.002556076) 	 BC Loss: 0.010518193 (0.007947534)
-Iteration: [13001 / 50000] 	 Loss: 0.006105572 (0.011037273) 	 Physics Loss: 0.000987124 (0.001454655) 	 Data Loss: 0.001305370 (0.002395016) 	 BC Loss: 0.003813077 (0.007187600)
-Iteration: [13501 / 50000] 	 Loss: 0.010004668 (0.011247103) 	 Physics Loss: 0.001224264 (0.001615586) 	 Data Loss: 0.002474443 (0.002668936) 	 BC Loss: 0.006305961 (0.006962581)
-Iteration: [14001 / 50000] 	 Loss: 0.009895653 (0.009313912) 	 Physics Loss: 0.001215764 (0.001694928) 	 Data Loss: 0.002037087 (0.002155375) 	 BC Loss: 0.006642802 (0.005463609)
-Iteration: [14501 / 50000] 	 Loss: 0.014400685 (0.008724037) 	 Physics Loss: 0.001658659 (0.001564218) 	 Data Loss: 0.003658900 (0.002201537) 	 BC Loss: 0.009083126 (0.004958282)
-Iteration: [15001 / 50000] 	 Loss: 0.007676640 (0.008405063) 	 Physics Loss: 0.001566568 (0.001608545) 	 Data Loss: 0.002262217 (0.001837625) 	 BC Loss: 0.003847855 (0.004958895)
-Iteration: [15501 / 50000] 	 Loss: 0.004365115 (0.009256126) 	 Physics Loss: 0.001047856 (0.002247394) 	 Data Loss: 0.000648518 (0.002218451) 	 BC Loss: 0.002668741 (0.004790282)
-Iteration: [16001 / 50000] 	 Loss: 0.004880759 (0.007501942) 	 Physics Loss: 0.001649067 (0.001671217) 	 Data Loss: 0.000296087 (0.001492234) 	 BC Loss: 0.002935605 (0.004338491)
-Iteration: [16501 / 50000] 	 Loss: 0.008074892 (0.007220342) 	 Physics Loss: 0.001825325 (0.001970405) 	 Data Loss: 0.001302087 (0.001730468) 	 BC Loss: 0.004947479 (0.003519468)
-Iteration: [17001 / 50000] 	 Loss: 0.005824474 (0.005835793) 	 Physics Loss: 0.002019164 (0.001728887) 	 Data Loss: 0.000624455 (0.001011056) 	 BC Loss: 0.003180855 (0.003095849)
-Iteration: [17501 / 50000] 	 Loss: 0.006616294 (0.005807751) 	 Physics Loss: 0.002489866 (0.001884541) 	 Data Loss: 0.001154157 (0.001245214) 	 BC Loss: 0.002972272 (0.002677995)
-Iteration: [18001 / 50000] 	 Loss: 0.004335414 (0.005200472) 	 Physics Loss: 0.001764537 (0.002015869) 	 Data Loss: 0.000705982 (0.001058994) 	 BC Loss: 0.001864895 (0.002125610)
-Iteration: [18501 / 50000] 	 Loss: 0.004978007 (0.005351806) 	 Physics Loss: 0.002553018 (0.002129085) 	 Data Loss: 0.000752965 (0.001336156) 	 BC Loss: 0.001672024 (0.001886566)
-Iteration: [19001 / 50000] 	 Loss: 0.004518208 (0.004657542) 	 Physics Loss: 0.001975382 (0.001959185) 	 Data Loss: 0.000239237 (0.000983244) 	 BC Loss: 0.002303589 (0.001715113)
-Iteration: [19501 / 50000] 	 Loss: 0.006942283 (0.004119421) 	 Physics Loss: 0.004590358 (0.001714717) 	 Data Loss: 0.001148389 (0.000894285) 	 BC Loss: 0.001203537 (0.001510418)
-Iteration: [20001 / 50000] 	 Loss: 0.008026988 (0.003463188) 	 Physics Loss: 0.005315070 (0.001485110) 	 Data Loss: 0.001919697 (0.000688489) 	 BC Loss: 0.000792221 (0.001289588)
-Iteration: [20501 / 50000] 	 Loss: 0.004855067 (0.003504427) 	 Physics Loss: 0.001560361 (0.001747428) 	 Data Loss: 0.000460401 (0.000618138) 	 BC Loss: 0.002834306 (0.001138862)
-Iteration: [21001 / 50000] 	 Loss: 0.002417301 (0.003299790) 	 Physics Loss: 0.000924424 (0.001513747) 	 Data Loss: 0.000300984 (0.000659882) 	 BC Loss: 0.001191893 (0.001126162)
-Iteration: [21501 / 50000] 	 Loss: 0.003911218 (0.002670718) 	 Physics Loss: 0.002752088 (0.001218763) 	 Data Loss: 0.000835131 (0.000637475) 	 BC Loss: 0.000323999 (0.000814481)
-Iteration: [22001 / 50000] 	 Loss: 0.002318957 (0.002511334) 	 Physics Loss: 0.001409405 (0.001309370) 	 Data Loss: 0.000344849 (0.000535839) 	 BC Loss: 0.000564703 (0.000666125)
-Iteration: [22501 / 50000] 	 Loss: 0.001994215 (0.002343247) 	 Physics Loss: 0.000746255 (0.001113559) 	 Data Loss: 0.000431564 (0.000577084) 	 BC Loss: 0.000816396 (0.000652603)
-Iteration: [23001 / 50000] 	 Loss: 0.002989073 (0.002301548) 	 Physics Loss: 0.001930058 (0.001243673) 	 Data Loss: 0.000805100 (0.000536315) 	 BC Loss: 0.000253915 (0.000521560)
-Iteration: [23501 / 50000] 	 Loss: 0.002210422 (0.002371583) 	 Physics Loss: 0.001325711 (0.001179150) 	 Data Loss: 0.000160725 (0.000516864) 	 BC Loss: 0.000723986 (0.000675569)
-Iteration: [24001 / 50000] 	 Loss: 0.002023818 (0.002276814) 	 Physics Loss: 0.000857553 (0.001135292) 	 Data Loss: 0.000525994 (0.000580056) 	 BC Loss: 0.000640270 (0.000561465)
-Iteration: [24501 / 50000] 	 Loss: 0.002234935 (0.002236427) 	 Physics Loss: 0.001041825 (0.001241080) 	 Data Loss: 0.000286281 (0.000510494) 	 BC Loss: 0.000906830 (0.000484853)
-Iteration: [25001 / 50000] 	 Loss: 0.001972227 (0.001982885) 	 Physics Loss: 0.000893347 (0.000971555) 	 Data Loss: 0.000666251 (0.000537486) 	 BC Loss: 0.000412629 (0.000473843)
-Iteration: [25501 / 50000] 	 Loss: 0.001659834 (0.002081697) 	 Physics Loss: 0.001085884 (0.001258897) 	 Data Loss: 0.000359855 (0.000474958) 	 BC Loss: 0.000214095 (0.000347842)
-Iteration: [26001 / 50000] 	 Loss: 0.002059042 (0.001770784) 	 Physics Loss: 0.001171167 (0.000890729) 	 Data Loss: 0.000426463 (0.000447986) 	 BC Loss: 0.000461412 (0.000432069)
-Iteration: [26501 / 50000] 	 Loss: 0.002089650 (0.002068657) 	 Physics Loss: 0.001594031 (0.001356683) 	 Data Loss: 0.000330550 (0.000420685) 	 BC Loss: 0.000165069 (0.000291289)
-Iteration: [27001 / 50000] 	 Loss: 0.001346401 (0.001879478) 	 Physics Loss: 0.000798861 (0.001050717) 	 Data Loss: 0.000333685 (0.000522715) 	 BC Loss: 0.000213855 (0.000306047)
-Iteration: [27501 / 50000] 	 Loss: 0.001717428 (0.001494603) 	 Physics Loss: 0.000754971 (0.000796866) 	 Data Loss: 0.000408040 (0.000351864) 	 BC Loss: 0.000554417 (0.000345872)
-Iteration: [28001 / 50000] 	 Loss: 0.001498525 (0.001789054) 	 Physics Loss: 0.000881248 (0.001008533) 	 Data Loss: 0.000310122 (0.000482373) 	 BC Loss: 0.000307154 (0.000298148)
-Iteration: [28501 / 50000] 	 Loss: 0.001300987 (0.001541014) 	 Physics Loss: 0.000685782 (0.000873007) 	 Data Loss: 0.000372513 (0.000378884) 	 BC Loss: 0.000242692 (0.000289123)
-Iteration: [29001 / 50000] 	 Loss: 0.001302390 (0.001531350) 	 Physics Loss: 0.000764329 (0.000809421) 	 Data Loss: 0.000326938 (0.000448536) 	 BC Loss: 0.000211123 (0.000273393)
-Iteration: [29501 / 50000] 	 Loss: 0.001228882 (0.001639404) 	 Physics Loss: 0.000864926 (0.000892691) 	 Data Loss: 0.000257633 (0.000503808) 	 BC Loss: 0.000106323 (0.000242905)
-Iteration: [30001 / 50000] 	 Loss: 0.001362466 (0.001636085) 	 Physics Loss: 0.000635906 (0.000994050) 	 Data Loss: 0.000649345 (0.000412151) 	 BC Loss: 0.000077215 (0.000229884)
-Iteration: [30501 / 50000] 	 Loss: 0.000972152 (0.001479216) 	 Physics Loss: 0.000619220 (0.000883626) 	 Data Loss: 0.000230181 (0.000379169) 	 BC Loss: 0.000122752 (0.000216420)
-Iteration: [31001 / 50000] 	 Loss: 0.005065940 (0.001380907) 	 Physics Loss: 0.004397592 (0.000831873) 	 Data Loss: 0.000477057 (0.000344165) 	 BC Loss: 0.000191290 (0.000204869)
-Iteration: [31501 / 50000] 	 Loss: 0.001720798 (0.001439294) 	 Physics Loss: 0.001153235 (0.000778960) 	 Data Loss: 0.000366380 (0.000444564) 	 BC Loss: 0.000201183 (0.000215771)
-Iteration: [32001 / 50000] 	 Loss: 0.001272172 (0.001182971) 	 Physics Loss: 0.000547293 (0.000645338) 	 Data Loss: 0.000612554 (0.000330688) 	 BC Loss: 0.000112325 (0.000206944)
-Iteration: [32501 / 50000] 	 Loss: 0.001592739 (0.001303701) 	 Physics Loss: 0.001089040 (0.000732894) 	 Data Loss: 0.000368751 (0.000377450) 	 BC Loss: 0.000134948 (0.000193357)
-Iteration: [33001 / 50000] 	 Loss: 0.001257007 (0.001731674) 	 Physics Loss: 0.000964463 (0.001143823) 	 Data Loss: 0.000217459 (0.000395994) 	 BC Loss: 0.000075086 (0.000191858)
-Iteration: [33501 / 50000] 	 Loss: 0.001404967 (0.001307026) 	 Physics Loss: 0.000708074 (0.000685339) 	 Data Loss: 0.000530431 (0.000435325) 	 BC Loss: 0.000166462 (0.000186362)
-Iteration: [34001 / 50000] 	 Loss: 0.000798481 (0.001060399) 	 Physics Loss: 0.000366831 (0.000566683) 	 Data Loss: 0.000218087 (0.000317301) 	 BC Loss: 0.000213564 (0.000176415)
-Iteration: [34501 / 50000] 	 Loss: 0.001798573 (0.001364744) 	 Physics Loss: 0.001362987 (0.000847362) 	 Data Loss: 0.000308547 (0.000354412) 	 BC Loss: 0.000127040 (0.000162970)
-Iteration: [35001 / 50000] 	 Loss: 0.000781053 (0.001014936) 	 Physics Loss: 0.000422994 (0.000539345) 	 Data Loss: 0.000270824 (0.000311768) 	 BC Loss: 0.000087234 (0.000163823)
-Iteration: [35501 / 50000] 	 Loss: 0.001209691 (0.001329915) 	 Physics Loss: 0.000784697 (0.000694873) 	 Data Loss: 0.000264404 (0.000425117) 	 BC Loss: 0.000160591 (0.000209925)
-Iteration: [36001 / 50000] 	 Loss: 0.001398915 (0.001136703) 	 Physics Loss: 0.000490567 (0.000583094) 	 Data Loss: 0.000744279 (0.000401171) 	 BC Loss: 0.000164069 (0.000152438)
-Iteration: [36501 / 50000] 	 Loss: 0.000835373 (0.001367094) 	 Physics Loss: 0.000595709 (0.000792289) 	 Data Loss: 0.000132676 (0.000426714) 	 BC Loss: 0.000106987 (0.000148092)
-Iteration: [37001 / 50000] 	 Loss: 0.000952036 (0.001044580) 	 Physics Loss: 0.000709375 (0.000580873) 	 Data Loss: 0.000161545 (0.000334467) 	 BC Loss: 0.000081115 (0.000129240)
-Iteration: [37501 / 50000] 	 Loss: 0.000579479 (0.001085730) 	 Physics Loss: 0.000270049 (0.000599332) 	 Data Loss: 0.000242183 (0.000348117) 	 BC Loss: 0.000067247 (0.000138281)
-Iteration: [38001 / 50000] 	 Loss: 0.001053359 (0.001110448) 	 Physics Loss: 0.000445312 (0.000595116) 	 Data Loss: 0.000482833 (0.000353724) 	 BC Loss: 0.000125214 (0.000161607)
-Iteration: [38501 / 50000] 	 Loss: 0.000667115 (0.001049175) 	 Physics Loss: 0.000326662 (0.000543516) 	 Data Loss: 0.000173648 (0.000387762) 	 BC Loss: 0.000166804 (0.000117897)
-Iteration: [39001 / 50000] 	 Loss: 0.000687853 (0.001085206) 	 Physics Loss: 0.000313576 (0.000590442) 	 Data Loss: 0.000321921 (0.000367264) 	 BC Loss: 0.000052356 (0.000127500)
-Iteration: [39501 / 50000] 	 Loss: 0.000673423 (0.001160439) 	 Physics Loss: 0.000421422 (0.000714949) 	 Data Loss: 0.000192454 (0.000300986) 	 BC Loss: 0.000059547 (0.000144504)
-Iteration: [40001 / 50000] 	 Loss: 0.000938448 (0.001243982) 	 Physics Loss: 0.000547043 (0.000710499) 	 Data Loss: 0.000287677 (0.000374305) 	 BC Loss: 0.000103729 (0.000159178)
-Iteration: [40501 / 50000] 	 Loss: 0.001798752 (0.001093546) 	 Physics Loss: 0.000866074 (0.000650020) 	 Data Loss: 0.000780730 (0.000305183) 	 BC Loss: 0.000151947 (0.000138344)
-Iteration: [41001 / 50000] 	 Loss: 0.001101400 (0.001448896) 	 Physics Loss: 0.000600817 (0.000901216) 	 Data Loss: 0.000405329 (0.000389668) 	 BC Loss: 0.000095254 (0.000158011)
-Iteration: [41501 / 50000] 	 Loss: 0.000776893 (0.000824789) 	 Physics Loss: 0.000455096 (0.000410463) 	 Data Loss: 0.000206055 (0.000317277) 	 BC Loss: 0.000115742 (0.000097049)
-Iteration: [42001 / 50000] 	 Loss: 0.001060384 (0.001240770) 	 Physics Loss: 0.000583868 (0.000778712) 	 Data Loss: 0.000378937 (0.000335865) 	 BC Loss: 0.000097578 (0.000126193)
-Iteration: [42501 / 50000] 	 Loss: 0.000691815 (0.000944194) 	 Physics Loss: 0.000381213 (0.000516831) 	 Data Loss: 0.000162085 (0.000307154) 	 BC Loss: 0.000148517 (0.000120209)
-Iteration: [43001 / 50000] 	 Loss: 0.000545245 (0.000868335) 	 Physics Loss: 0.000363892 (0.000449657) 	 Data Loss: 0.000124517 (0.000324281) 	 BC Loss: 0.000056836 (0.000094398)
-Iteration: [43501 / 50000] 	 Loss: 0.001068081 (0.001141232) 	 Physics Loss: 0.000613936 (0.000622923) 	 Data Loss: 0.000369106 (0.000392804) 	 BC Loss: 0.000085040 (0.000125504)
-Iteration: [44001 / 50000] 	 Loss: 0.001168622 (0.000963809) 	 Physics Loss: 0.000486189 (0.000540564) 	 Data Loss: 0.000549527 (0.000306839) 	 BC Loss: 0.000132906 (0.000116407)
-Iteration: [44501 / 50000] 	 Loss: 0.000939374 (0.000756769) 	 Physics Loss: 0.000379668 (0.000386392) 	 Data Loss: 0.000454265 (0.000277635) 	 BC Loss: 0.000105440 (0.000092742)
-Iteration: [45001 / 50000] 	 Loss: 0.001202180 (0.000892342) 	 Physics Loss: 0.000726597 (0.000506370) 	 Data Loss: 0.000367492 (0.000280093) 	 BC Loss: 0.000108091 (0.000105879)
-Iteration: [45501 / 50000] 	 Loss: 0.000887047 (0.000908168) 	 Physics Loss: 0.000641108 (0.000560744) 	 Data Loss: 0.000148591 (0.000263541) 	 BC Loss: 0.000097348 (0.000083883)
-Iteration: [46001 / 50000] 	 Loss: 0.000525872 (0.000734937) 	 Physics Loss: 0.000322635 (0.000362759) 	 Data Loss: 0.000158824 (0.000288360) 	 BC Loss: 0.000044412 (0.000083818)
-Iteration: [46501 / 50000] 	 Loss: 0.000545968 (0.000907963) 	 Physics Loss: 0.000350560 (0.000489585) 	 Data Loss: 0.000136768 (0.000312205) 	 BC Loss: 0.000058640 (0.000106173)
-Iteration: [47001 / 50000] 	 Loss: 0.000826687 (0.000924496) 	 Physics Loss: 0.000575209 (0.000511645) 	 Data Loss: 0.000205284 (0.000314716) 	 BC Loss: 0.000046194 (0.000098134)
-Iteration: [47501 / 50000] 	 Loss: 0.000620959 (0.001185708) 	 Physics Loss: 0.000382126 (0.000788119) 	 Data Loss: 0.000148993 (0.000271556) 	 BC Loss: 0.000089840 (0.000126034)
-Iteration: [48001 / 50000] 	 Loss: 0.000664698 (0.000798627) 	 Physics Loss: 0.000412207 (0.000461797) 	 Data Loss: 0.000182339 (0.000250679) 	 BC Loss: 0.000070152 (0.000086152)
-Iteration: [48501 / 50000] 	 Loss: 0.000826713 (0.000925605) 	 Physics Loss: 0.000485339 (0.000512687) 	 Data Loss: 0.000170909 (0.000314013) 	 BC Loss: 0.000170466 (0.000098905)
-Iteration: [49001 / 50000] 	 Loss: 0.000559556 (0.000846881) 	 Physics Loss: 0.000350503 (0.000421408) 	 Data Loss: 0.000165743 (0.000346362) 	 BC Loss: 0.000043311 (0.000079111)
-Iteration: [49501 / 50000] 	 Loss: 0.000652327 (0.000740137) 	 Physics Loss: 0.000301691 (0.000376061) 	 Data Loss: 0.000195276 (0.000275985) 	 BC Loss: 0.000155360 (0.000088090)

    Visualizing the Results ​

    julia
    ts, xs, ys = 0.0f0:0.05f0:2.0f0, 0.0f0:0.02f0:2.0f0, 0.0f0:0.02f0:2.0f0
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-16/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162595 (1.982162595) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505020 (0.598505020)
+Iteration: [  501 / 50000] 	 Loss: 0.040918142 (0.025583766) 	 Physics Loss: 0.000391871 (0.000269293) 	 Data Loss: 0.014243502 (0.009196415) 	 BC Loss: 0.026282769 (0.016118057)
+Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281416) 	 Physics Loss: 0.000071671 (0.000163183) 	 Data Loss: 0.007905646 (0.010876307) 	 BC Loss: 0.007363342 (0.014241925)
+Iteration: [ 1501 / 50000] 	 Loss: 0.019566942 (0.026170360) 	 Physics Loss: 0.001279285 (0.001009776) 	 Data Loss: 0.003071634 (0.010452234) 	 BC Loss: 0.015216023 (0.014708348)
+Iteration: [ 2001 / 50000] 	 Loss: 0.035557814 (0.027273539) 	 Physics Loss: 0.004062099 (0.001872000) 	 Data Loss: 0.013011657 (0.010586375) 	 BC Loss: 0.018484056 (0.014815168)
+Iteration: [ 2501 / 50000] 	 Loss: 0.011523969 (0.022175739) 	 Physics Loss: 0.002366644 (0.001909967) 	 Data Loss: 0.005568597 (0.007926281) 	 BC Loss: 0.003588729 (0.012339489)
+Iteration: [ 3001 / 50000] 	 Loss: 0.026762877 (0.031860039) 	 Physics Loss: 0.003192307 (0.006136029) 	 Data Loss: 0.009067146 (0.010883353) 	 BC Loss: 0.014503423 (0.014840655)
+Iteration: [ 3501 / 50000] 	 Loss: 0.025398642 (0.045591548) 	 Physics Loss: 0.003200589 (0.001476229) 	 Data Loss: 0.007934441 (0.019488331) 	 BC Loss: 0.014263613 (0.024626993)
+Iteration: [ 4001 / 50000] 	 Loss: 0.024822138 (0.029854555) 	 Physics Loss: 0.001797515 (0.003666145) 	 Data Loss: 0.012387147 (0.011416912) 	 BC Loss: 0.010637476 (0.014771502)
+Iteration: [ 4501 / 50000] 	 Loss: 0.029747024 (0.031501673) 	 Physics Loss: 0.005927891 (0.004604227) 	 Data Loss: 0.018316794 (0.011011195) 	 BC Loss: 0.005502339 (0.015886255)
+Iteration: [ 5001 / 50000] 	 Loss: 0.025349757 (0.043140102) 	 Physics Loss: 0.000728280 (0.002826607) 	 Data Loss: 0.003646539 (0.016781844) 	 BC Loss: 0.020974938 (0.023531647)
+Iteration: [ 5501 / 50000] 	 Loss: 0.026193798 (0.022408014) 	 Physics Loss: 0.000325338 (0.000346806) 	 Data Loss: 0.004838886 (0.008093885) 	 BC Loss: 0.021029573 (0.013967324)
+Iteration: [ 6001 / 50000] 	 Loss: 0.021179065 (0.022343241) 	 Physics Loss: 0.000527887 (0.000702699) 	 Data Loss: 0.007729640 (0.007971080) 	 BC Loss: 0.012921538 (0.013669461)
+Iteration: [ 6501 / 50000] 	 Loss: 0.030424416 (0.021639718) 	 Physics Loss: 0.000403142 (0.000825630) 	 Data Loss: 0.013241904 (0.009147485) 	 BC Loss: 0.016779371 (0.011666608)
+Iteration: [ 7001 / 50000] 	 Loss: 0.012885604 (0.022515845) 	 Physics Loss: 0.000715418 (0.000757549) 	 Data Loss: 0.006018532 (0.009810397) 	 BC Loss: 0.006151655 (0.011947901)
+Iteration: [ 7501 / 50000] 	 Loss: 0.021707822 (0.020194370) 	 Physics Loss: 0.000589236 (0.000854950) 	 Data Loss: 0.011797804 (0.008223927) 	 BC Loss: 0.009320782 (0.011115490)
+Iteration: [ 8001 / 50000] 	 Loss: 0.021062549 (0.019913949) 	 Physics Loss: 0.000528065 (0.000852256) 	 Data Loss: 0.008882683 (0.007675199) 	 BC Loss: 0.011651803 (0.011386497)
+Iteration: [ 8501 / 50000] 	 Loss: 0.022159226 (0.022652669) 	 Physics Loss: 0.001293960 (0.001621564) 	 Data Loss: 0.009795113 (0.008712809) 	 BC Loss: 0.011070153 (0.012318298)
+Iteration: [ 9001 / 50000] 	 Loss: 0.022167893 (0.021835698) 	 Physics Loss: 0.001893832 (0.001330249) 	 Data Loss: 0.006999766 (0.008149015) 	 BC Loss: 0.013274295 (0.012356437)
+Iteration: [ 9501 / 50000] 	 Loss: 0.016295999 (0.020470668) 	 Physics Loss: 0.002220259 (0.000820911) 	 Data Loss: 0.006721330 (0.008737788) 	 BC Loss: 0.007354409 (0.010911969)
+Iteration: [10001 / 50000] 	 Loss: 0.020919006 (0.022420531) 	 Physics Loss: 0.001985265 (0.000919413) 	 Data Loss: 0.003050850 (0.008777075) 	 BC Loss: 0.015882893 (0.012724041)
+Iteration: [10501 / 50000] 	 Loss: 0.015559800 (0.017435743) 	 Physics Loss: 0.000461346 (0.000464835) 	 Data Loss: 0.005280917 (0.006347454) 	 BC Loss: 0.009817538 (0.010623454)
+Iteration: [11001 / 50000] 	 Loss: 0.022515260 (0.018052436) 	 Physics Loss: 0.000395792 (0.000500864) 	 Data Loss: 0.004308904 (0.006823588) 	 BC Loss: 0.017810564 (0.010727987)
+Iteration: [11501 / 50000] 	 Loss: 0.021128871 (0.017566014) 	 Physics Loss: 0.000394875 (0.000523516) 	 Data Loss: 0.005251979 (0.006506131) 	 BC Loss: 0.015482016 (0.010536365)
+Iteration: [12001 / 50000] 	 Loss: 0.030411797 (0.018228574) 	 Physics Loss: 0.000672969 (0.000554123) 	 Data Loss: 0.019357491 (0.007100587) 	 BC Loss: 0.010381337 (0.010573865)
+Iteration: [12501 / 50000] 	 Loss: 0.018811548 (0.018401809) 	 Physics Loss: 0.000697236 (0.000691837) 	 Data Loss: 0.002697282 (0.006085245) 	 BC Loss: 0.015417031 (0.011624727)
+Iteration: [13001 / 50000] 	 Loss: 0.011868341 (0.017906483) 	 Physics Loss: 0.000776913 (0.000821749) 	 Data Loss: 0.003124564 (0.006143386) 	 BC Loss: 0.007966863 (0.010941349)
+Iteration: [13501 / 50000] 	 Loss: 0.014784457 (0.018100940) 	 Physics Loss: 0.000598925 (0.000848341) 	 Data Loss: 0.006280850 (0.006679880) 	 BC Loss: 0.007904682 (0.010572717)
+Iteration: [14001 / 50000] 	 Loss: 0.015592407 (0.016142424) 	 Physics Loss: 0.000597963 (0.000956681) 	 Data Loss: 0.006269322 (0.005398072) 	 BC Loss: 0.008725122 (0.009787671)
+Iteration: [14501 / 50000] 	 Loss: 0.025311429 (0.015725624) 	 Physics Loss: 0.001073368 (0.001058255) 	 Data Loss: 0.009163789 (0.005325366) 	 BC Loss: 0.015074271 (0.009342002)
+Iteration: [15001 / 50000] 	 Loss: 0.016914256 (0.015927596) 	 Physics Loss: 0.000966280 (0.001092387) 	 Data Loss: 0.005907979 (0.004847473) 	 BC Loss: 0.010039996 (0.009987737)
+Iteration: [15501 / 50000] 	 Loss: 0.010531792 (0.017309623) 	 Physics Loss: 0.001679290 (0.001229975) 	 Data Loss: 0.002361344 (0.005612177) 	 BC Loss: 0.006491158 (0.010467471)
+Iteration: [16001 / 50000] 	 Loss: 0.012041669 (0.015774786) 	 Physics Loss: 0.000827484 (0.001132011) 	 Data Loss: 0.001785598 (0.004045264) 	 BC Loss: 0.009428588 (0.010597511)
+Iteration: [16501 / 50000] 	 Loss: 0.017424010 (0.015305851) 	 Physics Loss: 0.001297660 (0.001166611) 	 Data Loss: 0.004497696 (0.004839269) 	 BC Loss: 0.011628653 (0.009299970)
+Iteration: [17001 / 50000] 	 Loss: 0.012141772 (0.013891050) 	 Physics Loss: 0.001339099 (0.001353682) 	 Data Loss: 0.001933840 (0.003089970) 	 BC Loss: 0.008868832 (0.009447398)
+Iteration: [17501 / 50000] 	 Loss: 0.015801162 (0.014529660) 	 Physics Loss: 0.001557242 (0.001193460) 	 Data Loss: 0.004102683 (0.004099627) 	 BC Loss: 0.010141237 (0.009236576)
+Iteration: [18001 / 50000] 	 Loss: 0.011515658 (0.013194922) 	 Physics Loss: 0.001411696 (0.001229187) 	 Data Loss: 0.003250526 (0.003880639) 	 BC Loss: 0.006853437 (0.008085096)
+Iteration: [18501 / 50000] 	 Loss: 0.014405347 (0.014845728) 	 Physics Loss: 0.000946846 (0.001311332) 	 Data Loss: 0.005034053 (0.004776356) 	 BC Loss: 0.008424449 (0.008758042)
+Iteration: [19001 / 50000] 	 Loss: 0.015056589 (0.013655247) 	 Physics Loss: 0.001184456 (0.001292421) 	 Data Loss: 0.003681621 (0.003807114) 	 BC Loss: 0.010190512 (0.008555712)
+Iteration: [19501 / 50000] 	 Loss: 0.013299890 (0.013724705) 	 Physics Loss: 0.001885105 (0.001415296) 	 Data Loss: 0.005226072 (0.003555858) 	 BC Loss: 0.006188713 (0.008753551)
+Iteration: [20001 / 50000] 	 Loss: 0.017522790 (0.012518023) 	 Physics Loss: 0.001983398 (0.001058186) 	 Data Loss: 0.009327068 (0.002995419) 	 BC Loss: 0.006212323 (0.008464416)
+Iteration: [20501 / 50000] 	 Loss: 0.021622123 (0.012733568) 	 Physics Loss: 0.001156326 (0.001213073) 	 Data Loss: 0.002187656 (0.003051734) 	 BC Loss: 0.018278142 (0.008468761)
+Iteration: [21001 / 50000] 	 Loss: 0.012820827 (0.013354947) 	 Physics Loss: 0.001192615 (0.001322284) 	 Data Loss: 0.000899577 (0.003003312) 	 BC Loss: 0.010728635 (0.009029351)
+Iteration: [21501 / 50000] 	 Loss: 0.016243555 (0.012213146) 	 Physics Loss: 0.002533023 (0.001458879) 	 Data Loss: 0.008658717 (0.003036225) 	 BC Loss: 0.005051813 (0.007718043)
+Iteration: [22001 / 50000] 	 Loss: 0.009802144 (0.011197914) 	 Physics Loss: 0.000975115 (0.001180587) 	 Data Loss: 0.002004704 (0.002875937) 	 BC Loss: 0.006822325 (0.007141389)
+Iteration: [22501 / 50000] 	 Loss: 0.010460960 (0.011299610) 	 Physics Loss: 0.001939239 (0.001350272) 	 Data Loss: 0.003044792 (0.002758041) 	 BC Loss: 0.005476930 (0.007191297)
+Iteration: [23001 / 50000] 	 Loss: 0.007303775 (0.010617163) 	 Physics Loss: 0.001568149 (0.001255589) 	 Data Loss: 0.002208042 (0.002901576) 	 BC Loss: 0.003527584 (0.006459999)
+Iteration: [23501 / 50000] 	 Loss: 0.013054269 (0.012929341) 	 Physics Loss: 0.001233365 (0.001229342) 	 Data Loss: 0.002502190 (0.002853266) 	 BC Loss: 0.009318714 (0.008846736)
+Iteration: [24001 / 50000] 	 Loss: 0.015850905 (0.011500452) 	 Physics Loss: 0.001933446 (0.001243522) 	 Data Loss: 0.003105919 (0.002514284) 	 BC Loss: 0.010811541 (0.007742645)
+Iteration: [24501 / 50000] 	 Loss: 0.014612451 (0.011235875) 	 Physics Loss: 0.000830096 (0.001126977) 	 Data Loss: 0.001665982 (0.002516578) 	 BC Loss: 0.012116373 (0.007592318)
+Iteration: [25001 / 50000] 	 Loss: 0.012446284 (0.012136834) 	 Physics Loss: 0.001868680 (0.001571323) 	 Data Loss: 0.002885855 (0.002738171) 	 BC Loss: 0.007691749 (0.007827340)
+Iteration: [25501 / 50000] 	 Loss: 0.006147437 (0.010866777) 	 Physics Loss: 0.001199473 (0.001432847) 	 Data Loss: 0.000741202 (0.003181246) 	 BC Loss: 0.004206762 (0.006252684)
+Iteration: [26001 / 50000] 	 Loss: 0.009382382 (0.012447415) 	 Physics Loss: 0.002055192 (0.001882480) 	 Data Loss: 0.001872860 (0.002723655) 	 BC Loss: 0.005454330 (0.007841279)
+Iteration: [26501 / 50000] 	 Loss: 0.005613507 (0.009200627) 	 Physics Loss: 0.001204579 (0.001562393) 	 Data Loss: 0.001970439 (0.002349454) 	 BC Loss: 0.002438489 (0.005288779)
+Iteration: [27001 / 50000] 	 Loss: 0.008868203 (0.010649174) 	 Physics Loss: 0.001465238 (0.001703513) 	 Data Loss: 0.002235665 (0.002514324) 	 BC Loss: 0.005167301 (0.006431336)
+Iteration: [27501 / 50000] 	 Loss: 0.016564976 (0.011443286) 	 Physics Loss: 0.001060150 (0.001400374) 	 Data Loss: 0.003114783 (0.002061526) 	 BC Loss: 0.012390045 (0.007981384)
+Iteration: [28001 / 50000] 	 Loss: 0.008029928 (0.011309550) 	 Physics Loss: 0.001292446 (0.001715263) 	 Data Loss: 0.001651582 (0.003145006) 	 BC Loss: 0.005085900 (0.006449282)
+Iteration: [28501 / 50000] 	 Loss: 0.009412916 (0.010573968) 	 Physics Loss: 0.002310942 (0.001589268) 	 Data Loss: 0.000935075 (0.002185026) 	 BC Loss: 0.006166900 (0.006799675)
+Iteration: [29001 / 50000] 	 Loss: 0.008956734 (0.010948162) 	 Physics Loss: 0.001730394 (0.001973851) 	 Data Loss: 0.002910380 (0.002606902) 	 BC Loss: 0.004315960 (0.006367408)
+Iteration: [29501 / 50000] 	 Loss: 0.004513775 (0.010220452) 	 Physics Loss: 0.000894471 (0.001687118) 	 Data Loss: 0.001609863 (0.002537781) 	 BC Loss: 0.002009441 (0.005995554)
+Iteration: [30001 / 50000] 	 Loss: 0.004987885 (0.009731424) 	 Physics Loss: 0.001069190 (0.001562893) 	 Data Loss: 0.001401214 (0.002276014) 	 BC Loss: 0.002517481 (0.005892518)
+Iteration: [30501 / 50000] 	 Loss: 0.002595518 (0.008276105) 	 Physics Loss: 0.000917391 (0.001500407) 	 Data Loss: 0.000658372 (0.001976204) 	 BC Loss: 0.001019756 (0.004799493)
+Iteration: [31001 / 50000] 	 Loss: 0.011266185 (0.010293764) 	 Physics Loss: 0.001391491 (0.001473510) 	 Data Loss: 0.003635545 (0.002334290) 	 BC Loss: 0.006239149 (0.006485965)
+Iteration: [31501 / 50000] 	 Loss: 0.013991628 (0.010204735) 	 Physics Loss: 0.002423544 (0.001848977) 	 Data Loss: 0.004265617 (0.002343330) 	 BC Loss: 0.007302467 (0.006012429)
+Iteration: [32001 / 50000] 	 Loss: 0.007164373 (0.009639673) 	 Physics Loss: 0.001379599 (0.001698582) 	 Data Loss: 0.001701293 (0.001939833) 	 BC Loss: 0.004083481 (0.006001258)
+Iteration: [32501 / 50000] 	 Loss: 0.005502027 (0.009115184) 	 Physics Loss: 0.001436674 (0.001596168) 	 Data Loss: 0.001871673 (0.001895187) 	 BC Loss: 0.002193680 (0.005623828)
+Iteration: [33001 / 50000] 	 Loss: 0.005276884 (0.009546057) 	 Physics Loss: 0.002079992 (0.001929016) 	 Data Loss: 0.001436205 (0.001975889) 	 BC Loss: 0.001760687 (0.005641153)
+Iteration: [33501 / 50000] 	 Loss: 0.009125081 (0.008468037) 	 Physics Loss: 0.002191632 (0.001853661) 	 Data Loss: 0.002888216 (0.001945538) 	 BC Loss: 0.004045234 (0.004668837)
+Iteration: [34001 / 50000] 	 Loss: 0.008431979 (0.009071054) 	 Physics Loss: 0.001222603 (0.002027335) 	 Data Loss: 0.001151307 (0.001467187) 	 BC Loss: 0.006058069 (0.005576532)
+Iteration: [34501 / 50000] 	 Loss: 0.009580643 (0.008130345) 	 Physics Loss: 0.002242853 (0.001757617) 	 Data Loss: 0.005897780 (0.002249273) 	 BC Loss: 0.001440010 (0.004123454)
+Iteration: [35001 / 50000] 	 Loss: 0.005147196 (0.007530828) 	 Physics Loss: 0.001256183 (0.001518356) 	 Data Loss: 0.001415687 (0.001431392) 	 BC Loss: 0.002475326 (0.004581080)
+Iteration: [35501 / 50000] 	 Loss: 0.006185614 (0.008019534) 	 Physics Loss: 0.001828601 (0.001882976) 	 Data Loss: 0.001599119 (0.001704919) 	 BC Loss: 0.002757893 (0.004431641)
+Iteration: [36001 / 50000] 	 Loss: 0.009109061 (0.007414200) 	 Physics Loss: 0.001948264 (0.001928137) 	 Data Loss: 0.001738493 (0.001558672) 	 BC Loss: 0.005422304 (0.003927392)
+Iteration: [36501 / 50000] 	 Loss: 0.007278774 (0.007622246) 	 Physics Loss: 0.002025729 (0.001955235) 	 Data Loss: 0.002425219 (0.001680354) 	 BC Loss: 0.002827825 (0.003986657)
+Iteration: [37001 / 50000] 	 Loss: 0.003918901 (0.005813450) 	 Physics Loss: 0.000957004 (0.001656469) 	 Data Loss: 0.001278843 (0.001385545) 	 BC Loss: 0.001683053 (0.002771435)
+Iteration: [37501 / 50000] 	 Loss: 0.004814619 (0.006178515) 	 Physics Loss: 0.001686519 (0.001814278) 	 Data Loss: 0.001063806 (0.001266953) 	 BC Loss: 0.002064294 (0.003097285)
+Iteration: [38001 / 50000] 	 Loss: 0.004643182 (0.004791833) 	 Physics Loss: 0.001702445 (0.001350533) 	 Data Loss: 0.000782973 (0.001124672) 	 BC Loss: 0.002157764 (0.002316626)
+Iteration: [38501 / 50000] 	 Loss: 0.003810477 (0.004910624) 	 Physics Loss: 0.001390965 (0.001866840) 	 Data Loss: 0.000597971 (0.000985127) 	 BC Loss: 0.001821541 (0.002058655)
+Iteration: [39001 / 50000] 	 Loss: 0.003878577 (0.005460986) 	 Physics Loss: 0.002414002 (0.002075948) 	 Data Loss: 0.000412366 (0.001095256) 	 BC Loss: 0.001052209 (0.002289784)
+Iteration: [39501 / 50000] 	 Loss: 0.004065084 (0.004690929) 	 Physics Loss: 0.002393598 (0.001760620) 	 Data Loss: 0.000800843 (0.000796935) 	 BC Loss: 0.000870643 (0.002133374)
+Iteration: [40001 / 50000] 	 Loss: 0.004619529 (0.004839203) 	 Physics Loss: 0.001658047 (0.001965609) 	 Data Loss: 0.001588614 (0.000951834) 	 BC Loss: 0.001372868 (0.001921760)
+Iteration: [40501 / 50000] 	 Loss: 0.004326439 (0.004265270) 	 Physics Loss: 0.001820831 (0.001626283) 	 Data Loss: 0.001034181 (0.000920903) 	 BC Loss: 0.001471427 (0.001718084)
+Iteration: [41001 / 50000] 	 Loss: 0.002906828 (0.003618096) 	 Physics Loss: 0.001272737 (0.001407278) 	 Data Loss: 0.000809125 (0.000837591) 	 BC Loss: 0.000824967 (0.001373227)
+Iteration: [41501 / 50000] 	 Loss: 0.004206825 (0.003857500) 	 Physics Loss: 0.001842899 (0.001706076) 	 Data Loss: 0.000595867 (0.000833410) 	 BC Loss: 0.001768059 (0.001318014)
+Iteration: [42001 / 50000] 	 Loss: 0.003873884 (0.003361913) 	 Physics Loss: 0.002234953 (0.001453047) 	 Data Loss: 0.000641468 (0.000736817) 	 BC Loss: 0.000997462 (0.001172049)
+Iteration: [42501 / 50000] 	 Loss: 0.003796777 (0.003109351) 	 Physics Loss: 0.001149042 (0.001176956) 	 Data Loss: 0.000481490 (0.000653678) 	 BC Loss: 0.002166245 (0.001278718)
+Iteration: [43001 / 50000] 	 Loss: 0.001652764 (0.003025806) 	 Physics Loss: 0.000779939 (0.001280978) 	 Data Loss: 0.000271947 (0.000640416) 	 BC Loss: 0.000600878 (0.001104411)
+Iteration: [43501 / 50000] 	 Loss: 0.002625272 (0.003282005) 	 Physics Loss: 0.001812232 (0.001467459) 	 Data Loss: 0.000510654 (0.000728400) 	 BC Loss: 0.000302386 (0.001086145)
+Iteration: [44001 / 50000] 	 Loss: 0.002716297 (0.002971325) 	 Physics Loss: 0.001654499 (0.001524955) 	 Data Loss: 0.000789657 (0.000616715) 	 BC Loss: 0.000272142 (0.000829655)
+Iteration: [44501 / 50000] 	 Loss: 0.003345200 (0.002695851) 	 Physics Loss: 0.001435974 (0.001263972) 	 Data Loss: 0.001142784 (0.000586666) 	 BC Loss: 0.000766442 (0.000845213)
+Iteration: [45001 / 50000] 	 Loss: 0.002160525 (0.002570425) 	 Physics Loss: 0.001014665 (0.001206274) 	 Data Loss: 0.000531116 (0.000582542) 	 BC Loss: 0.000614744 (0.000781609)
+Iteration: [45501 / 50000] 	 Loss: 0.002704483 (0.002375467) 	 Physics Loss: 0.001256422 (0.001126713) 	 Data Loss: 0.000432294 (0.000537340) 	 BC Loss: 0.001015767 (0.000711415)
+Iteration: [46001 / 50000] 	 Loss: 0.002083311 (0.002229830) 	 Physics Loss: 0.001028772 (0.001128200) 	 Data Loss: 0.000365198 (0.000481077) 	 BC Loss: 0.000689341 (0.000620552)
+Iteration: [46501 / 50000] 	 Loss: 0.001869006 (0.002387334) 	 Physics Loss: 0.001049295 (0.001213058) 	 Data Loss: 0.000542785 (0.000525414) 	 BC Loss: 0.000276926 (0.000648862)
+Iteration: [47001 / 50000] 	 Loss: 0.002161729 (0.002562924) 	 Physics Loss: 0.001238901 (0.001363050) 	 Data Loss: 0.000660004 (0.000578991) 	 BC Loss: 0.000262824 (0.000620883)
+Iteration: [47501 / 50000] 	 Loss: 0.003218761 (0.002436990) 	 Physics Loss: 0.001678471 (0.001291234) 	 Data Loss: 0.000470345 (0.000499937) 	 BC Loss: 0.001069946 (0.000645818)
+Iteration: [48001 / 50000] 	 Loss: 0.001741661 (0.001888746) 	 Physics Loss: 0.000994589 (0.000895934) 	 Data Loss: 0.000492043 (0.000409009) 	 BC Loss: 0.000255029 (0.000583803)
+Iteration: [48501 / 50000] 	 Loss: 0.001460486 (0.002372291) 	 Physics Loss: 0.000619964 (0.001287611) 	 Data Loss: 0.000285411 (0.000508584) 	 BC Loss: 0.000555111 (0.000576096)
+Iteration: [49001 / 50000] 	 Loss: 0.001497735 (0.002357919) 	 Physics Loss: 0.000912286 (0.001282274) 	 Data Loss: 0.000287049 (0.000532408) 	 BC Loss: 0.000298400 (0.000543235)
+Iteration: [49501 / 50000] 	 Loss: 0.003226456 (0.001985326) 	 Physics Loss: 0.001385620 (0.000979689) 	 Data Loss: 0.000554629 (0.000451327) 	 BC Loss: 0.001286207 (0.000554309)

    Visualizing the Results ​

    julia
    ts, xs, ys = 0.0f0:0.05f0:2.0f0, 0.0f0:0.02f0:2.0f0, 0.0f0:0.02f0:2.0f0
 grid = stack([[elem...] for elem in vec(collect(Iterators.product(xs, ys, ts)))])
 
 u_real = reshape(analytical_solution(grid), length(xs), length(ys), length(ts))
@@ -309,7 +309,7 @@
     end
 
     fig
-end

    Appendix ​

    julia
    using InteractiveUtils
+end

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -368,9 +368,10 @@
 Environment:
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
-1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - +2 devices: + 0: Quadro RTX 5000 (sm_75, 15.258 GiB / 16.000 GiB available) + 1: Quadro RTX 5000 (sm_75, 15.549 GiB / 16.000 GiB available)

    This page was generated using Literate.jl.

    + \ No newline at end of file