diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 12a2a422..96cced30 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-02-06T05:14:01","documenter_version":"1.2.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.1","generation_timestamp":"2024-02-18T08:26:51","documenter_version":"1.2.1"}} \ No newline at end of file diff --git a/dev/api/index.html b/dev/api/index.html index 043cff72..c8509f66 100644 --- a/dev/api/index.html +++ b/dev/api/index.html @@ -1,6 +1,6 @@ -API Reference · Sophon.jl
Sophon.AdaptiveTrainingType
AdaptiveTraining(pde_weights, bcs_weights)

Adaptive weights for the loss functions. Here pde_weights and bcs_weights are functions that take in (phi, x, θ) and return the point-wise weights. Note that bcs_weights can be real numbers but they will be converted to functions that return the same numbers.

source
Sophon.BetaRandomSamplerType
BetaRandomSampler(pde_points, bcs_points=pde_points; sampling_alg=SobolSample(),
-                  resample::Bool=false, α=0.4, β=1.0)

Same as QuasiRandomSampler, but use Beta distribution along time on the domain.

source
Sophon.ChainStateType
ChainState(model, rng::AbstractRNG=Random.default_rng())

It this similar to Lux.Chain but wraps it in a stateful container.

Fields

  • model: The neural network.
  • states: The states of the neural network.

Input

  • x: The input to the neural network.
  • ps: The parameters of the neural network.

Arguments

  • model: AbstractExplicitLayer, or a named tuple of them, which will be treated as a Chain.
  • rng: AbstractRNG to use for initialising the neural network.
source
Sophon.ConstantFunctionType
ConstantFunction()

A conatiner for scalar parameter. This is useful for the case that you want a dummy layer that returns the scalar parameter for any input.

source
Sophon.DeepONetType
DeepONet(branch_net, trunk_net;
+API Reference · Sophon.jl
Sophon.AdaptiveTrainingType
AdaptiveTraining(pde_weights, bcs_weights)

Adaptive weights for the loss functions. Here pde_weights and bcs_weights are functions that take in (phi, x, θ) and return the point-wise weights. Note that bcs_weights can be real numbers but they will be converted to functions that return the same numbers.

source
Sophon.BetaRandomSamplerType
BetaRandomSampler(pde_points, bcs_points=pde_points; sampling_alg=SobolSample(),
+                  resample::Bool=false, α=0.4, β=1.0)

Same as QuasiRandomSampler, but use Beta distribution along time on the domain.

source
Sophon.ChainStateType
ChainState(model, rng::AbstractRNG=Random.default_rng())

It this similar to Lux.Chain but wraps it in a stateful container.

Fields

  • model: The neural network.
  • states: The states of the neural network.

Input

  • x: The input to the neural network.
  • ps: The parameters of the neural network.

Arguments

  • model: AbstractExplicitLayer, or a named tuple of them, which will be treated as a Chain.
  • rng: AbstractRNG to use for initialising the neural network.
source
Sophon.ConstantFunctionType
ConstantFunction()

A conatiner for scalar parameter. This is useful for the case that you want a dummy layer that returns the scalar parameter for any input.

source
Sophon.DeepONetType
DeepONet(branch_net, trunk_net;
          flatten_layer=FlattenLayer(),
          linear_layer=NoOpLayer(),
          bias=ScalarLayer())
@@ -26,9 +26,9 @@
     linear_layer = NoOpLayer(),
     bias = ScalarLayer(),                    # 1 parameters
 )         # Total: 111 parameters,
-          #        plus 0 states, summarysize 80 bytes.

Reference

[5]

source
Sophon.DiscreteFourierFeatureType
DiscreteFourierFeature(in_dims::Int, out_dims::Int, N::Int, period::Real)

The discrete Fourier filter proposed in [6]. For a periodic function with period $P$, the Fourier series in amplitude-phase form is

\[s_N(x)=\frac{a_0}{2}+\sum_{n=1}^N{a_n}\cdot \sin \left( \frac{2\pi}{P}nx+\varphi _n \right)\]

The output is guaranteed to be periodic.

Arguments

  • in_dims: Number of the input dimensions.
  • out_dims: Number of the output dimensions.
  • N: $N$ in the formula.
  • period: $P$ in the formula.
source
Sophon.FactorizedDenseType
FactorizedDense(in_dims::Int, out_dims::Int, activation=identity;
+          #        plus 0 states, summarysize 80 bytes.

Reference

[5]

source
Sophon.DiscreteFourierFeatureType
DiscreteFourierFeature(in_dims::Int, out_dims::Int, N::Int, period::Real)

The discrete Fourier filter proposed in [6]. For a periodic function with period $P$, the Fourier series in amplitude-phase form is

\[s_N(x)=\frac{a_0}{2}+\sum_{n=1}^N{a_n}\cdot \sin \left( \frac{2\pi}{P}nx+\varphi _n \right)\]

The output is guaranteed to be periodic.

Arguments

  • in_dims: Number of the input dimensions.
  • out_dims: Number of the output dimensions.
  • N: $N$ in the formula.
  • period: $P$ in the formula.
source
Sophon.FactorizedDenseType
FactorizedDense(in_dims::Int, out_dims::Int, activation=identity;
                      mean::AbstractFloat=1.0f0, std::AbstractFloat=0.1f0,
-                     init_weight=kaiming_uniform(activation), init_bias=zeros32)

Create a Dense layer where the weight is factorized into twa parts, the scaling factors for each row and the weight matrix.

Arguments

  • in_dims: number of input dimensions
  • out_dims: number of output dimensions
  • activation: activation function

Keyword Arguments

  • mean: mean of the scaling factors
  • std: standard deviation of the scaling factors
  • init_weight: weight initialization function
  • init_bias: bias initialization function

Input

  • x: input vector or matrix

Returns

  • y = activation.(scale * weight * x+ bias).
  • Empty NamedTuple().

Parameters

  • scale: scaling factors. Shape: (out_dims, 1)
  • weight: Weight Matrix of size (out_dims, in_dims).
  • bias: Bias of size (out_dims, 1).

References

[7]

source
Sophon.FourierFeatureType
FourierFeature(in_dims::Int, std::NTuple{N,Pair{S,T}}) where {N,S,T<:Int}
+                     init_weight=kaiming_uniform(activation), init_bias=zeros32)

Create a Dense layer where the weight is factorized into twa parts, the scaling factors for each row and the weight matrix.

Arguments

  • in_dims: number of input dimensions
  • out_dims: number of output dimensions
  • activation: activation function

Keyword Arguments

  • mean: mean of the scaling factors
  • std: standard deviation of the scaling factors
  • init_weight: weight initialization function
  • init_bias: bias initialization function

Input

  • x: input vector or matrix

Returns

  • y = activation.(scale * weight * x+ bias).
  • Empty NamedTuple().

Parameters

  • scale: scaling factors. Shape: (out_dims, 1)
  • weight: Weight Matrix of size (out_dims, in_dims).
  • bias: Bias of size (out_dims, 1).

References

[7]

source
Sophon.FourierFeatureType
FourierFeature(in_dims::Int, std::NTuple{N,Pair{S,T}}) where {N,S,T<:Int}
 FourierFeature(in_dims::Int, frequencies::NTuple{N, T}) where {N, T <: Real}
 FourierFeature(in_dims::Int, out_dims::Int, std::Real)

Fourier Feature Network.

Arguments

  • in_dims: Number of the input dimensions.
  • std: A tuple of pairs of sigma => out_dims, where sigma is the standard deviation of the Gaussian distribution.

\[\phi^{(i)}(x)=\left[\sin \left(2 \pi W^{(i)} x\right) ; \cos 2 \pi W^{(i)} x\right],\ W^{(i)} \sim \mathcal{N}\left(0, \sigma^{(i)}\right),\ i\in 1, \dots, D\]

  • frequencies: A tuple of frequencies $(f1,f2,...,fn)$.

\[\phi^{(i)}(x)=\left[\sin \left(2 \pi f_i x\right) ; \cos 2 \pi f_i x\right]\]

Parameters

If std is used, then parameters are Ws in the formula.

Inputs

  • x: AbstractArray with size(x, 1) == in_dims.

Returns

  • $[\phi^{(1)}, \phi^{(2)}, ... ,\phi^{(D)}]$ with size(y, 1) == sum(last(modes) * 2).

Examples

julia> f = FourierFeature(2,10,1) # Random Fourier Feature
 FourierFeature(2 => 10)
@@ -37,7 +37,7 @@
 FourierFeature(2 => 14)
 
 julia>  f = FourierFeature(2, (1,2,3,4)) # Predefined frequencies
-FourierFeature(2 => 16)

References

[8]

[9]

[4]

source
Sophon.NonAdaptiveTrainingType
NonAdaptiveTraining(pde_weights=1, bcs_weights=pde_weights)

Fixed weights for the loss functions.

Arguments

  • pde_weights: weights for the PDE loss functions. If a single number is given, it is used for all PDE loss functions.
  • bcs_weights: weights for the boundary conditions loss functions. If a single number is given, it is used for all boundary conditions loss functions.
source
Sophon.PINNType
PINN(chain, rng::AbstractRNG=Random.default_rng())
+FourierFeature(2 => 16)

References

[8]

[9]

[4]

source
Sophon.NonAdaptiveTrainingType
NonAdaptiveTraining(pde_weights=1, bcs_weights=pde_weights)

Fixed weights for the loss functions.

Arguments

  • pde_weights: weights for the PDE loss functions. If a single number is given, it is used for all PDE loss functions.
  • bcs_weights: weights for the boundary conditions loss functions. If a single number is given, it is used for all boundary conditions loss functions.
source
Sophon.PINNType
PINN(chain, rng::AbstractRNG=Random.default_rng())
 PINN(rng::AbstractRNG=Random.default_rng(); kwargs...)

A container for a neural network, its states and its initial parameters. The default element type of the parameters is Float64.

Fields

  • phi: ChainState if there is only one neural network, or an named tuple of ChainStates if there are multiple neural networks. The names are the same as the dependent variables in the PDE.
  • init_params: The initial parameters of the neural network.

Arguments

  • chain: AbstractExplicitLayer or a named tuple of AbstractExplicitLayers.
  • rng: AbstractRNG to use for initialising the neural network. If yout want to set the seed, write
using Random
 rng = Random.default_rng()
 Random.seed!(rng, 0)d

and pass rng to PINN as

using Sophon
@@ -48,15 +48,15 @@
 pinn = PINN(chain, rng);
 
 # multiple dependent varibales
-pinn = PINN(rng; a=chain, b=chain);
source
Sophon.PINNAttentionType
PINNAttention(H_net, U_net, V_net, fusion_layers)
 PINNAttention(in_dims::Int, out_dims::Int, activation::Function=sin;
               hidden_dims::Int, num_layers::Int)

The output dimesion of H_net and the input dimension of fusion_layers must be the same. For the second and the third constructor, Dense layers is used for H_net, U_net, and V_net. Note that the first constructer does not contain the output layer, but the second one does.

                 x → U_net → u                           u
                                ↘                           ↘
 x → H_net →  h1 → fusionlayer1 → connection → fusionlayer2 → connection
                                ↗                           ↗
-                 x → V_net → v                           v

Arguments

  • H_net: AbstractExplicitLayer.
  • U_net: AbstractExplicitLayer.
  • V_net: AbstractExplicitLayer.
  • fusion_layers: Chain.

Keyword Arguments

  • num_layers: The number of hidden layers.
  • hidden_dims: The number of hidden dimensions of each hidden layer.

Reference

[10]

source
Sophon.QuasiRandomSamplerType
QuasiRandomSampler(pde_points, bcs_points=pde_points;
+                 x → V_net → v                           v

Arguments

  • H_net: AbstractExplicitLayer.
  • U_net: AbstractExplicitLayer.
  • V_net: AbstractExplicitLayer.
  • fusion_layers: Chain.

Keyword Arguments

  • num_layers: The number of hidden layers.
  • hidden_dims: The number of hidden dimensions of each hidden layer.

Reference

[10]

source
Sophon.QuasiRandomSamplerType
QuasiRandomSampler(pde_points, bcs_points=pde_points;
                    sampling_alg=SobolSample(),
-                   resample = false))

Sampler to generate the datasets for PDE and boundary conditions using a quisa-random sampling algorithm. You can call sample(pde, sampler, strategy) on it to generate all the datasets. See QuasiMonteCarlo.jl for available sampling algorithms. The default element type of the sampled data is Float64. The initial sampled data lives on GPU if PINN is. You will need manually move the data to GPU if you want to resample.

Arguments

  • pde_points: The number of points to sample for each PDE. If a single number is given, the same number of points will be sampled for each PDE. If a tuple of numbers is given, the number of points for each PDE will be the corresponding element in the tuple. The default is 100.
  • bcs_points: The number of points to sample for each boundary condition. If a single number is given, the same number of points will be sampled for each boundary condition. If a tuple of numbers is given, the number of points for each boundary condition will be the corresponding element in the tuple. The default is pde_points.

Keyword Arguments

  • sampling_alg: The sampling algorithm to use. The default is SobolSample().

  • resample: Whether to resample the data for each equation. The default is false, which can save a lot of memory if you are solving a large number of PDEs. In this case, pde_points has to be a integer. If you want to resample the data, you will need to manually move the data to GPU if you want to use GPU to solve the PDEs.

source
Sophon.RBFType
RBF(in_dims::Int, out_dims::Int, num_centers::Int=out_dims; sigma::AbstractFloat=0.2f0)

Normalized Radial Basis Fuction Network.

source
Sophon.TriplewiseFusionType

TriplewiseFusion(connection, layers...)

     u1                    u2
+                   resample = false))

Sampler to generate the datasets for PDE and boundary conditions using a quisa-random sampling algorithm. You can call sample(pde, sampler, strategy) on it to generate all the datasets. See QuasiMonteCarlo.jl for available sampling algorithms. The default element type of the sampled data is Float64. The initial sampled data lives on GPU if PINN is. You will need manually move the data to GPU if you want to resample.

Arguments

  • pde_points: The number of points to sample for each PDE. If a single number is given, the same number of points will be sampled for each PDE. If a tuple of numbers is given, the number of points for each PDE will be the corresponding element in the tuple. The default is 100.
  • bcs_points: The number of points to sample for each boundary condition. If a single number is given, the same number of points will be sampled for each boundary condition. If a tuple of numbers is given, the number of points for each boundary condition will be the corresponding element in the tuple. The default is pde_points.

Keyword Arguments

  • sampling_alg: The sampling algorithm to use. The default is SobolSample().

  • resample: Whether to resample the data for each equation. The default is false, which can save a lot of memory if you are solving a large number of PDEs. In this case, pde_points has to be a integer. If you want to resample the data, you will need to manually move the data to GPU if you want to use GPU to solve the PDEs.

source
Sophon.RBFType
RBF(in_dims::Int, out_dims::Int, num_centers::Int=out_dims; sigma::AbstractFloat=0.2f0)

Normalized Radial Basis Fuction Network.

source
Sophon.TriplewiseFusionType

TriplewiseFusion(connection, layers...)

     u1                    u2
         ↘                     ↘
 h1 → layer1 → connection → layer2 → connection
         ↗                     ↗
@@ -64,7 +64,7 @@
     h = connection(layers[i](h), u[i], v[i])
 end
  1. A triple of (h, u, v), where u and v are AbstractArrays.
for i in 1:N
     h = connection(layers[i](h), u, v)
-end

Parameters

  • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N

States

  • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N
source
Sophon.BACONMethod
BACON(in_dims::Int, out_dims::Int, N::Int, period::Real; hidden_dims::Int, num_layers::Int)

Band-limited Coordinate Networks (BACON) from [6]. Similar to FourierFilterNet but the frequcies are dicrete and nontrainable.

Tips: It is recommended to set period to be 1,2,π or for better performance.

source
Sophon.FourierAttentionMethod
FourierAttention(in_dims::Int, out_dims::Int, activation::Function, std;
+end

Parameters

  • Parameters of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N

States

  • States of each layer wrapped in a NamedTuple with fields = layer_1, layer_2, ..., layer_N
source
Sophon.BACONMethod
BACON(in_dims::Int, out_dims::Int, N::Int, period::Real; hidden_dims::Int, num_layers::Int)

Band-limited Coordinate Networks (BACON) from [6]. Similar to FourierFilterNet but the frequcies are dicrete and nontrainable.

Tips: It is recommended to set period to be 1,2,π or for better performance.

source
Sophon.FourierAttentionMethod
FourierAttention(in_dims::Int, out_dims::Int, activation::Function, std;
                  hidden_dims::Int=512, num_layers::Int=6, modes::NTuple)
 FourierAttention(in_dims::Int, out_dims::Int, activation::Function, frequencies;
                  hidden_dims::Int=512, num_layers::Int=6, modes::NTuple)

A model that combines FourierFeature and PINNAttention.

x → [FourierFeature(x); x] → PINNAttention

Arguments

  • in_dims: The input dimension.

Keyword Arguments

  • hidden_dim: The hidden dimension of each hidden layer.
  • num_layers: The number of hidden layers.

Examples

julia> FourierAttention(3, 1, sin, (1 => 10, 10 => 10, 50 => 10); hidden_dims=10, num_layers=3)
@@ -83,12 +83,12 @@
     ),
     layer_3 = Dense(10 => 1),           # 11 parameters
 )         # Total: 2_371 parameters,
-          #        plus 90 states, summarysize 192 bytes
source
Sophon.FourierFilterNetMethod
FourierFilterNet(in_dims::Int, out_dims::Int; hidden_dims::Int, num_layers::Int,
+          #        plus 90 states, summarysize 192 bytes
source
Sophon.FourierFilterNetMethod
FourierFilterNet(in_dims::Int, out_dims::Int; hidden_dims::Int, num_layers::Int,
                  bandwidth::Real)

Multiplicative filter network defined by

\[\begin{aligned} z^{(1)} &=g\left(x ; \theta^{(1)}\right) \\ z^{(i+1)} &=\left(W^{(i)} z^{(i)}+b^{(i)}\right) \circ \sin \left(\omega^{(i)} x+\phi^{(i)}\right)\right) \\ f(x) &=W^{(k)} z^{(k)}+b^{(k)} -\end{aligned}\]

Keyword Arguments

  • bandwidth: The maximum bandwidth of the network. The bandwidth is the sum of each filter's bandwidth.

Parameters

  • Parameters of the filters:

\[ W\sim \mathcal{U}(-\frac{ω}{n}, \frac{ω}{n}), \quad b\sim \mathcal{U}(-\pi, \pi),\]

where n is the number of filters.

For a periodic function with period $P$, the Fourier series in amplitude-phase form is

\[s_N(x)=\frac{a_0}{2}+\sum_{n=1}^N{a_n}\cdot \sin \left( \frac{2\pi}{P}nx+\varphi _n \right)\]

We have the following relation between the banthwidth and the parameters of the model:

\[ω = 2πB=\frac{2πN}{P}.\]

where $B$ is the bandwidth of the network.

References

[11]

[6]

source
Sophon.FourierNetMethod
FourierNet(ayer_sizes::NTuple, activation, modes::NTuple)

A model that combines FourierFeature and FullyConnected.

x → FourierFeature → FullyConnected → y

Arguments

  • in_dims: The number of input dimensions.
  • layer_sizes: A tuple of hidden dimensions used to construct FullyConnected.
  • activation: The activation function used to construct FullyConnected.
  • modes: A tuple of modes used to construct FourierFeature.

Examples

julia> FourierNet((2, 30, 30, 1), sin, (1 => 10, 10 => 10, 50 => 10))
+\end{aligned}\]

Keyword Arguments

  • bandwidth: The maximum bandwidth of the network. The bandwidth is the sum of each filter's bandwidth.

Parameters

  • Parameters of the filters:

\[ W\sim \mathcal{U}(-\frac{ω}{n}, \frac{ω}{n}), \quad b\sim \mathcal{U}(-\pi, \pi),\]

where n is the number of filters.

For a periodic function with period $P$, the Fourier series in amplitude-phase form is

\[s_N(x)=\frac{a_0}{2}+\sum_{n=1}^N{a_n}\cdot \sin \left( \frac{2\pi}{P}nx+\varphi _n \right)\]

We have the following relation between the banthwidth and the parameters of the model:

\[ω = 2πB=\frac{2πN}{P}.\]

where $B$ is the bandwidth of the network.

References

[11]

[6]

source
Sophon.FourierNetMethod
FourierNet(ayer_sizes::NTuple, activation, modes::NTuple)

A model that combines FourierFeature and FullyConnected.

x → FourierFeature → FullyConnected → y

Arguments

  • in_dims: The number of input dimensions.
  • layer_sizes: A tuple of hidden dimensions used to construct FullyConnected.
  • activation: The activation function used to construct FullyConnected.
  • modes: A tuple of modes used to construct FourierFeature.

Examples

julia> FourierNet((2, 30, 30, 1), sin, (1 => 10, 10 => 10, 50 => 10))
 Chain(
     layer_1 = FourierFeature(2 => 60),
     layer_2 = Dense(60 => 30, sin),     # 1_830 parameters
@@ -104,7 +104,7 @@
     layer_3 = Dense(30 => 30, sin),     # 930 parameters
     layer_4 = Dense(30 => 1),           # 31 parameters
 )         # Total: 1_471 parameters,
-          #        plus 4 states, summarysize 96 bytes.
source
Sophon.FullyConnectedMethod
FullyConnected(layer_sizes::NTuple{N, Int}, activation; outermost = true,
+          #        plus 4 states, summarysize 96 bytes.
source
Sophon.FullyConnectedMethod
FullyConnected(layer_sizes::NTuple{N, Int}, activation; outermost = true,
                init_weight=kaiming_uniform(activation),
                init_bias=zeros32,
                allow_fast_activation=false)
@@ -127,7 +127,7 @@
     layer_3 = Dense(20 => 20, relu),    # 420 parameters
     layer_4 = Dense(20 => 10),          # 210 parameters
 )         # Total: 1_090 parameters,
-          #        plus 0 states, summarysize 64 bytes.
source
Sophon.ResNetMethod
ResNet(layer_sizes::NTuple{N, Int}, activation; outermost=true,
+          #        plus 0 states, summarysize 64 bytes.
source
Sophon.ResNetMethod
ResNet(layer_sizes::NTuple{N, Int}, activation; outermost=true,
                init_weight=kaiming_uniform(activation),
                init_bias=zeros32,
                allow_fast_activation=false)
@@ -159,8 +159,8 @@
     ),
     layer_4 = Dense(20 => 10),          # 210 parameters
 )         # Total: 1_090 parameters,
-          #        plus 0 states, summarysize 64 bytes.
source
Sophon.SineMethod
Sine(in_dims::Int, out_dims::Int; omega::Real)

Sinusoidal layer.

Example

s = Sine(2, 2; omega=30.0f0) # first layer
-s = Sine(2, 2) # hidden layer
source
Sophon.SirenMethod
Siren(in_dims::Int, out_dims::Int; hidden_dims::Int, num_layers::Int, omega=30.0f0,
+          #        plus 0 states, summarysize 64 bytes.
source
Sophon.SineMethod
Sine(in_dims::Int, out_dims::Int; omega::Real)

Sinusoidal layer.

Example

s = Sine(2, 2; omega=30.0f0) # first layer
+s = Sine(2, 2) # hidden layer
source
Sophon.SirenMethod
Siren(in_dims::Int, out_dims::Int; hidden_dims::Int, num_layers::Int, omega=30.0f0,
       init_weight=nothing))
 Siren(layer_sizes::Int...; omega=30.0f0, init_weight=nothing)

Sinusoidal Representation Network.

Keyword Arguments

  • omega: The ω₀ used for the first layer.
  • init_weight: The initialization algorithm for the weights of the input layer. Note that all hidden layers use kaiming_uniform as the initialization algorithm. The default is

    \[ W\sim \mathcal{U}\left(-\frac{\omega}{fan_{in}}, \frac{\omega}{fan_{in}}\right)\]

Examples

julia> Siren(2, 32, 32, 1; omega=5.0f0)
 Chain(
@@ -181,6 +181,6 @@
 
 # Use your own initialization algorithm for the input layer.
 julia> init_weight(rng::AbstractRNG, out_dims::Int, in_dims::Int) = randn(rng, Float32, out_dims, in_dims) .* 2.5f0
-julia> chain = Siren(2, 1; num_layers = 4, hidden_dims = 50, init_weight = init_weight)

Reference

[1]

source
Sophon.discretizeMethod
 discretize(pde_system::PDESystem, pinn::PINN, sampler::PINNSampler,
+julia> chain = Siren(2, 1; num_layers = 4, hidden_dims = 50, init_weight = init_weight)

Reference

[1]

source
Sophon.discretizeMethod
 discretize(pde_system::PDESystem, pinn::PINN, sampler::PINNSampler,
                 strategy::AbstractTrainingAlg; derivative=finitediff,
-                additional_loss)

Convert the PDESystem into an OptimizationProblem. You will have access to each loss function Sophon.residual_function_1, Sophon.residual_function_2... after calling this function.

source
Sophon.gaussianFunction
gaussian(x, a=0.2)

The Gaussian activation function.

\[e^{\frac{- x^{2}}{2a^{2}}}\]

Reference

[2]

source
Sophon.stanFunction
stan()

Self-scalable Tanh.

\[\sigma(x^i) = tanh(x^i) + \beta^i * x^i * tanh(x^i)\]

Reference

[12]

source
+ additional_loss)

Convert the PDESystem into an OptimizationProblem. You will have access to each loss function Sophon.residual_function_1, Sophon.residual_function_2... after calling this function.

source
Sophon.gaussianFunction
gaussian(x, a=0.2)

The Gaussian activation function.

\[e^{\frac{- x^{2}}{2a^{2}}}\]

Reference

[2]

source
Sophon.stanFunction
stan()

Self-scalable Tanh.

\[\sigma(x^i) = tanh(x^i) + \beta^i * x^i * tanh(x^i)\]

Reference

[12]

source
diff --git a/dev/index.html b/dev/index.html index c7582be7..02b08cef 100644 --- a/dev/index.html +++ b/dev/index.html @@ -17,4 +17,4 @@ t ∈ Interval(0.0, π/2)] @named pde_system = PDESystem(eqs, bcs, domains, [x,t], [u(x,t),v(x,t)])

The @parameters macro defines the parameters of the PDE system, and the @variables macro defines the dependent variables. We use Differential to define the derivatives with respect to time and space. The eqs array defines the equations in the PDE system. The bcs array defines the boundary conditions. The domains array defines the spatial and temporal domains of the PDE system. Finally, we use the @named macro to give a name to the PDE system.

Step 3: Define the neural network architecture

Next, we define the physics-informed neural network (PINN) using Sophon.jl. In this example, we will use a Siren network with 2 sine layers and 1 cosine layer for each variable, and 16 hidden dimensions per layer. We will use 4 layers for both variables, and set the frequency parameter $\omega$ to 1.0.

pinn = PINN(u = Siren(2,1; hidden_dims=16,num_layers=4, omega = 1.0),
-            v = Siren(2,1; hidden_dims=16,num_layers=4, omega = 1.0))            

We define a physics-informed neural network (PINN) with the pinn variable. The PINN macro takes a dictionary that maps the dependent variables to their corresponding neural network architecture. In this case, we use the Siren architecture for both u and v with 2 input dimensions, 1 output dimension, 16 hidden dimensions, and 4 layers. We also set the frequency of the sine activation functions to 1.0.

Step 4: Create a QuasiRandomSampler object

Here, we create a QuasiRandomSampler object with 500 sample points, where the first argument corresponds to the number of data points for each equation, and the second argument corresponds to the number of data points for each boundary condition.

sampler = QuasiRandomSampler(500, (200,200,20,20))

Step 5: Define a training strategy

Here, we use a NonAdaptiveTraining strategy with 1 as the weight of all equations, and (10,10,1,1) for the four boundary conditions.

strategy = NonAdaptiveTraining(1,(10,10,1,1))

Step 6: Discretize the PDE system using Sophon

prob = Sophon.discretize(pde_system, pinn, sampler, strategy)

Step 7: Solve the optimization problem

res = Optimization.solve(prob, BFGS(); maxiters=2000)
+ v = Siren(2,1; hidden_dims=16,num_layers=4, omega = 1.0))

We define a physics-informed neural network (PINN) with the pinn variable. The PINN macro takes a dictionary that maps the dependent variables to their corresponding neural network architecture. In this case, we use the Siren architecture for both u and v with 2 input dimensions, 1 output dimension, 16 hidden dimensions, and 4 layers. We also set the frequency of the sine activation functions to 1.0.

Step 4: Create a QuasiRandomSampler object

Here, we create a QuasiRandomSampler object with 500 sample points, where the first argument corresponds to the number of data points for each equation, and the second argument corresponds to the number of data points for each boundary condition.

sampler = QuasiRandomSampler(500, (200,200,20,20))

Step 5: Define a training strategy

Here, we use a NonAdaptiveTraining strategy with 1 as the weight of all equations, and (10,10,1,1) for the four boundary conditions.

strategy = NonAdaptiveTraining(1,(10,10,1,1))

Step 6: Discretize the PDE system using Sophon

prob = Sophon.discretize(pde_system, pinn, sampler, strategy)

Step 7: Solve the optimization problem

res = Optimization.solve(prob, BFGS(); maxiters=2000)
diff --git a/dev/qa/index.html b/dev/qa/index.html index f451785a..9dc98f5a 100644 --- a/dev/qa/index.html +++ b/dev/qa/index.html @@ -5,4 +5,4 @@ loss_2 = sum(abs2, Sophon.residual_function_1(prob.p[2], p)) println("loss: $loss_1, loss_2") return false -end

Finally, pass the callback function to Optimization.solve to monitor the loss as the training progresses.

Q: How can I inspect the generated symbolic loss function?

A: Simply replace Sophon.discretize by Sophon.symbolic_discretize.

+end

Finally, pass the callback function to Optimization.solve to monitor the loss as the training progresses.

Q: How can I inspect the generated symbolic loss function?

A: Simply replace Sophon.discretize by Sophon.symbolic_discretize.

diff --git a/dev/references/index.html b/dev/references/index.html index 347431ec..8e871c3b 100644 --- a/dev/references/index.html +++ b/dev/references/index.html @@ -1,2 +1,2 @@ -References · Sophon.jl

References

[1]
V. Sitzmann, J. Martel, A. Bergman, D. Lindell and G. Wetzstein. Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems 33, 7462–7473 (2020).
[2]
S. Ramasinghe and S. Lucey. Beyond periodicity: Towards a unifying framework for activations in coordinate-mlps, arXiv preprint arXiv:2111.15135 (2021).
[3]
S. Ramasinghe, L. MacDonald and S. Lucey. On Regularizing Coordinate-MLPs, arXiv preprint arXiv:2202.00790 (2022).
[4]
S. Wang, H. Wang and P. Perdikaris. On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering 384, 113938 (2021).
[5]
L. Lu, P. Jin, G. Pang, Z. Zhang and G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence 3, 218–229 (2021).
[6]
D. B. Lindell, D. Van Veen, J. J. Park and G. Wetzstein. BACON: Band-limited coordinate networks for multiscale scene representation, arXiv preprint arXiv:0000.00000 (2021).
[7]
S. Wang, H. Wang, J. H. Seidman and P. Perdikaris. Random Weight Factorization Improves the Training of Continuous Neural Representations, arXiv preprint arXiv:2210.01274 (2022).
[8]
A. Rahimi and B. Recht. Random features for large-scale kernel machines. Advances in neural information processing systems 20 (2007).
[9]
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron and R. Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems 33, 7537–7547 (2020).
[10]
S. Wang, Y. Teng and P. Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing 43, A3055–A3081 (2021).
[11]
R. Fathony, A. K. Sahu, D. Willmott and J. Z. Kolter. Multiplicative Filter Networks. In: International Conference on Learning Representations (2021).
[12]
R. Gnanasambandam, B. Shen, J. Chung, X. Yue and others. Self-scalable tanh (stan): Faster convergence and better generalization in physics-informed neural networks, arXiv preprint arXiv:2204.12589 (2022).
+References · Sophon.jl

References

[1]
V. Sitzmann, J. Martel, A. Bergman, D. Lindell and G. Wetzstein. Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems 33, 7462–7473 (2020).
[2]
S. Ramasinghe and S. Lucey. Beyond periodicity: Towards a unifying framework for activations in coordinate-mlps, arXiv preprint arXiv:2111.15135 (2021).
[3]
S. Ramasinghe, L. MacDonald and S. Lucey. On Regularizing Coordinate-MLPs, arXiv preprint arXiv:2202.00790 (2022).
[4]
S. Wang, H. Wang and P. Perdikaris. On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering 384, 113938 (2021).
[5]
L. Lu, P. Jin, G. Pang, Z. Zhang and G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence 3, 218–229 (2021).
[6]
D. B. Lindell, D. Van Veen, J. J. Park and G. Wetzstein. BACON: Band-limited coordinate networks for multiscale scene representation, arXiv preprint arXiv:0000.00000 (2021).
[7]
S. Wang, H. Wang, J. H. Seidman and P. Perdikaris. Random Weight Factorization Improves the Training of Continuous Neural Representations, arXiv preprint arXiv:2210.01274 (2022).
[8]
A. Rahimi and B. Recht. Random features for large-scale kernel machines. Advances in neural information processing systems 20 (2007).
[9]
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron and R. Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems 33, 7537–7547 (2020).
[10]
S. Wang, Y. Teng and P. Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing 43, A3055–A3081 (2021).
[11]
R. Fathony, A. K. Sahu, D. Willmott and J. Z. Kolter. Multiplicative Filter Networks. In: International Conference on Learning Representations (2021).
[12]
R. Gnanasambandam, B. Shen, J. Chung, X. Yue and others. Self-scalable tanh (stan): Faster convergence and better generalization in physics-informed neural networks, arXiv preprint arXiv:2204.12589 (2022).
diff --git a/dev/tutorials/L_shape/index.html b/dev/tutorials/L_shape/index.html index 2e3f4ad2..2dadcb6a 100644 --- a/dev/tutorials/L_shape/index.html +++ b/dev/tutorials/L_shape/index.html @@ -54,430 +54,362 @@ u_pred = [ifelse(x>0.0 && y>0.0, NaN, pinn.phi([x,y], res.u)[1]) for x in xs, y in ys] fig, ax, hm = heatmap(xs, ys, u_pred, colormap=:jet) Colorbar(fig[:, end+1], hm) -fig

0.1%┣                                         ┫ 1/1.0k [01:00<Inf:Inf, InfGs/it]
-
3.353620e+02 0.2%┣                             ┫ 2/1.0k [01:00<16:38:58, 60s/it]
-
2.862752e+02 0.3%┣                             ┫ 3/1.0k [01:00<08:19:32, 30s/it]
-
1.584824e+02 0.5%┣▏                            ┫ 5/1.0k [01:00<04:09:44, 15s/it]
-
8.688561e+01 0.7%┣▏                            ┫ 7/1.0k [01:00<02:46:21, 10s/it]
-
3.094053e+01 0.9%┣▎                             ┫ 9/1.0k [01:00<02:04:39, 8s/it]
-
1.759503e+01 1.1%┣▎                            ┫ 11/1.0k [01:00<01:39:38, 6s/it]
-
6.372584e+00 1.3%┣▍                            ┫ 13/1.0k [01:00<01:22:57, 5s/it]
-
3.804188e+00 1.5%┣▍                            ┫ 15/1.0k [01:01<01:11:02, 4s/it]
-
2.338239e+00 1.7%┣▌                            ┫ 17/1.0k [01:01<01:02:05, 4s/it]
-
1.617707e+00 1.9%┣▋                               ┫ 19/1.0k [01:01<55:08, 3s/it]
-
1.146932e+00 2.1%┣▊                               ┫ 21/1.0k [01:01<49:34, 3s/it]
-
9.366290e-01 2.3%┣▊                               ┫ 23/1.0k [01:01<45:00, 3s/it]
-
7.620831e-01 2.6%┣▉                               ┫ 26/1.0k [01:01<39:32, 2s/it]
-
4.501904e-01 2.8%┣█                               ┫ 28/1.0k [01:01<36:34, 2s/it]
-
3.805184e-01 3.0%┣█                               ┫ 30/1.0k [01:01<34:00, 2s/it]
-
2.705196e-01 3.2%┣█                               ┫ 32/1.0k [01:01<31:47, 2s/it]
-
2.081409e-01 3.5%┣█▏                              ┫ 35/1.0k [01:01<28:55, 2s/it]
-
1.563658e-01 3.8%┣█▏                              ┫ 38/1.0k [01:01<26:31, 2s/it]
-
1.333699e-01 4.1%┣█▎                              ┫ 41/1.0k [01:01<24:29, 2s/it]
-
1.079300e-01 4.4%┣█▍                              ┫ 44/1.0k [01:01<22:44, 1s/it]
-
9.059360e-02 4.6%┣█▌                              ┫ 46/1.0k [01:01<21:41, 1s/it]
-
7.733614e-02 4.9%┣█▋                              ┫ 49/1.0k [01:01<20:18, 1s/it]
-
6.898208e-02 5.2%┣█▋                              ┫ 52/1.0k [01:01<19:03, 1s/it]
-
5.715830e-02 5.5%┣█▊                              ┫ 55/1.0k [01:02<17:58, 1s/it]
-
4.946172e-02 5.8%┣█▉                              ┫ 58/1.0k [01:02<16:59, 1s/it]
-
4.463075e-02 6.0%┣██                              ┫ 60/1.0k [01:02<16:23, 1s/it]
-
3.652041e-02 6.3%┣██                              ┫ 63/1.0k [01:02<15:33, 1it/s]
-
2.831844e-02 6.6%┣██                              ┫ 66/1.0k [01:02<14:48, 1it/s]
-
2.299712e-02 6.9%┣██▏                             ┫ 69/1.0k [01:02<14:07, 1it/s]
-
1.920728e-02 7.2%┣██▎                             ┫ 72/1.0k [01:02<13:30, 1it/s]
-
1.668054e-02 7.5%┣██▍                             ┫ 75/1.0k [01:02<12:55, 1it/s]
-
1.472526e-02 7.8%┣██▌                             ┫ 78/1.0k [01:02<12:23, 1it/s]
-
1.324911e-02 8.1%┣██▋                             ┫ 81/1.0k [01:02<11:54, 1it/s]
-
1.134558e-02 8.4%┣██▊                             ┫ 84/1.0k [01:02<11:27, 1it/s]
-
1.026686e-02 8.7%┣██▉                             ┫ 87/1.0k [01:02<11:01, 1it/s]
-
9.062615e-03 9.0%┣██▉                             ┫ 90/1.0k [01:02<10:37, 1it/s]
-
8.274057e-03 9.3%┣███                             ┫ 93/1.0k [01:02<10:15, 1it/s]
-
7.852569e-03 9.6%┣███                             ┫ 96/1.0k [01:02<09:54, 2it/s]
-
7.488457e-03 9.9%┣███▏                            ┫ 99/1.0k [01:02<09:35, 2it/s]
-
7.194180e-03 10.2%┣███                           ┫ 102/1.0k [01:03<09:17, 2it/s]
-
6.788896e-03 10.5%┣███▏                          ┫ 105/1.0k [01:03<08:59, 2it/s]
-
6.238394e-03 10.8%┣███▎                          ┫ 108/1.0k [01:03<08:43, 2it/s]
-
5.957860e-03 11.1%┣███▎                          ┫ 111/1.0k [01:03<08:27, 2it/s]
-
5.774090e-03 11.4%┣███▍                          ┫ 114/1.0k [01:03<08:13, 2it/s]
-
5.526326e-03 11.7%┣███▌                          ┫ 117/1.0k [01:03<07:59, 2it/s]
-
5.229096e-03 12.0%┣███▋                          ┫ 120/1.0k [01:03<07:46, 2it/s]
-
4.913240e-03 12.3%┣███▊                          ┫ 123/1.0k [01:03<07:33, 2it/s]
-
4.559521e-03 12.6%┣███▊                          ┫ 126/1.0k [01:03<07:21, 2it/s]
-
4.339523e-03 12.9%┣███▉                          ┫ 129/1.0k [01:03<07:10, 2it/s]
-
4.076924e-03 13.2%┣████                          ┫ 132/1.0k [01:03<06:59, 2it/s]
-
3.821564e-03 13.5%┣████                          ┫ 135/1.0k [01:03<06:49, 2it/s]
-
3.646844e-03 13.8%┣████▏                         ┫ 138/1.0k [01:03<06:39, 2it/s]
-
3.490606e-03 14.1%┣████▎                         ┫ 141/1.0k [01:03<06:29, 2it/s]
-
3.356760e-03 14.4%┣████▎                         ┫ 144/1.0k [01:03<06:20, 2it/s]
-
3.214260e-03 14.7%┣████▍                         ┫ 147/1.0k [01:03<06:11, 2it/s]
-
3.116750e-03 14.9%┣████▌                         ┫ 149/1.0k [01:04<06:06, 2it/s]
-
2.906998e-03 15.2%┣████▌                         ┫ 152/1.0k [01:04<05:58, 2it/s]
-
2.777977e-03 15.5%┣████▋                         ┫ 155/1.0k [01:04<05:50, 2it/s]
-
2.613999e-03 15.8%┣████▊                         ┫ 158/1.0k [01:04<05:42, 2it/s]
-
2.435325e-03 16.1%┣████▉                         ┫ 161/1.0k [01:04<05:35, 3it/s]
-
2.260365e-03 16.4%┣█████                         ┫ 164/1.0k [01:04<05:28, 3it/s]
-
2.133030e-03 16.7%┣█████                         ┫ 167/1.0k [01:04<05:21, 3it/s]
-
2.062952e-03 17.0%┣█████                         ┫ 170/1.0k [01:04<05:15, 3it/s]
-
1.957221e-03 17.3%┣█████▏                        ┫ 173/1.0k [01:04<05:08, 3it/s]
-
1.910034e-03 17.6%┣█████▎                        ┫ 176/1.0k [01:04<05:02, 3it/s]
-
1.882042e-03 17.8%┣█████▍                        ┫ 178/1.0k [01:04<04:58, 3it/s]
-
1.809470e-03 18.1%┣█████▍                        ┫ 181/1.0k [01:04<04:53, 3it/s]
-
1.765840e-03 18.4%┣█████▌                        ┫ 184/1.0k [01:04<04:47, 3it/s]
-
1.741301e-03 18.6%┣█████▋                        ┫ 186/1.0k [01:04<04:43, 3it/s]
-
1.700670e-03 18.9%┣█████▋                        ┫ 189/1.0k [01:04<04:38, 3it/s]
-
1.659234e-03 19.2%┣█████▊                        ┫ 192/1.0k [01:04<04:33, 3it/s]
-
1.608088e-03 19.5%┣█████▉                        ┫ 195/1.0k [01:05<04:28, 3it/s]
-
1.559223e-03 19.8%┣██████                        ┫ 198/1.0k [01:05<04:23, 3it/s]
-
1.522948e-03 20.0%┣██████                        ┫ 200/1.0k [01:05<04:20, 3it/s]
-
1.490467e-03 20.3%┣██████                        ┫ 203/1.0k [01:05<04:16, 3it/s]
-
1.469209e-03 20.6%┣██████▏                       ┫ 206/1.0k [01:05<04:11, 3it/s]
-
1.455920e-03 20.8%┣██████▎                       ┫ 208/1.0k [01:05<04:08, 3it/s]
-
1.429418e-03 21.1%┣██████▎                       ┫ 211/1.0k [01:05<04:04, 3it/s]
-
1.415654e-03 21.3%┣██████▍                       ┫ 213/1.0k [01:05<04:02, 3it/s]
-
1.401613e-03 21.5%┣██████▍                       ┫ 215/1.0k [01:05<03:59, 3it/s]
-
1.391111e-03 21.7%┣██████▌                       ┫ 217/1.0k [01:05<03:56, 3it/s]
-
1.378822e-03 21.9%┣██████▋                       ┫ 219/1.0k [01:05<03:54, 3it/s]
-
1.362865e-03 22.1%┣██████▋                       ┫ 221/1.0k [01:05<03:51, 3it/s]
-
1.353496e-03 22.3%┣██████▊                       ┫ 223/1.0k [01:05<03:49, 3it/s]
-
1.341148e-03 22.5%┣██████▊                       ┫ 225/1.0k [01:05<03:46, 3it/s]
-
1.329214e-03 22.7%┣██████▉                       ┫ 227/1.0k [01:05<03:44, 3it/s]
-
1.319105e-03 22.9%┣██████▉                       ┫ 229/1.0k [01:05<03:42, 3it/s]
-
1.303707e-03 23.2%┣███████                       ┫ 232/1.0k [01:06<03:38, 4it/s]
-
1.288635e-03 23.4%┣███████                       ┫ 234/1.0k [01:06<03:36, 4it/s]
-
1.277298e-03 23.6%┣███████                       ┫ 236/1.0k [01:06<03:34, 4it/s]
-
1.262239e-03 23.9%┣███████▏                      ┫ 239/1.0k [01:06<03:30, 4it/s]
-
1.251989e-03 24.1%┣███████▎                      ┫ 241/1.0k [01:06<03:28, 4it/s]
-
1.231423e-03 24.3%┣███████▎                      ┫ 243/1.0k [01:06<03:26, 4it/s]
-
1.208421e-03 24.6%┣███████▍                      ┫ 246/1.0k [01:06<03:23, 4it/s]
-
1.202055e-03 24.8%┣███████▍                      ┫ 248/1.0k [01:06<03:21, 4it/s]
-
1.193991e-03 25.0%┣███████▌                      ┫ 250/1.0k [01:06<03:19, 4it/s]
-
1.183817e-03 25.2%┣███████▌                      ┫ 252/1.0k [01:06<03:17, 4it/s]
-
1.178927e-03 25.4%┣███████▋                      ┫ 254/1.0k [01:06<03:15, 4it/s]
-
1.168781e-03 25.6%┣███████▊                      ┫ 256/1.0k [01:06<03:13, 4it/s]
-
1.155631e-03 25.9%┣███████▊                      ┫ 259/1.0k [01:06<03:10, 4it/s]
-
1.142346e-03 26.2%┣███████▉                      ┫ 262/1.0k [01:06<03:08, 4it/s]
-
1.134296e-03 26.4%┣████████                      ┫ 264/1.0k [01:06<03:06, 4it/s]
-
1.123206e-03 26.7%┣████████                      ┫ 267/1.0k [01:06<03:03, 4it/s]
-
1.115909e-03 26.9%┣████████                      ┫ 269/1.0k [01:06<03:02, 4it/s]
-
1.104784e-03 27.2%┣████████▏                     ┫ 272/1.0k [01:07<02:59, 4it/s]
-
1.094947e-03 27.5%┣████████▎                     ┫ 275/1.0k [01:07<02:56, 4it/s]
-
1.090292e-03 27.7%┣████████▎                     ┫ 277/1.0k [01:07<02:55, 4it/s]
-
1.081351e-03 27.9%┣████████▍                     ┫ 279/1.0k [01:07<02:53, 4it/s]
-
1.074899e-03 28.2%┣████████▌                     ┫ 282/1.0k [01:07<02:51, 4it/s]
-
1.069867e-03 28.4%┣████████▌                     ┫ 284/1.0k [01:07<02:49, 4it/s]
-
1.062240e-03 28.6%┣████████▋                     ┫ 286/1.0k [01:07<02:48, 4it/s]
-
1.056581e-03 28.8%┣████████▋                     ┫ 288/1.0k [01:07<02:46, 4it/s]
-
1.052462e-03 29.0%┣████████▊                     ┫ 290/1.0k [01:07<02:45, 4it/s]
-
1.044219e-03 29.2%┣████████▊                     ┫ 292/1.0k [01:07<02:43, 4it/s]
-
1.032175e-03 29.5%┣████████▉                     ┫ 295/1.0k [01:07<02:41, 4it/s]
-
1.026176e-03 29.7%┣█████████                     ┫ 297/1.0k [01:07<02:40, 4it/s]
-
1.014148e-03 30.0%┣█████████                     ┫ 300/1.0k [01:07<02:38, 4it/s]
-
1.004853e-03 30.2%┣█████████                     ┫ 302/1.0k [01:07<02:36, 4it/s]
-
9.984130e-04 30.4%┣█████████                     ┫ 304/1.0k [01:07<02:35, 4it/s]
-
9.918338e-04 30.6%┣█████████▏                    ┫ 306/1.0k [01:07<02:34, 5it/s]
-
9.829494e-04 30.8%┣█████████▎                    ┫ 308/1.0k [01:08<02:32, 5it/s]
-
9.691419e-04 31.1%┣█████████▎                    ┫ 311/1.0k [01:08<02:30, 5it/s]
-
9.618324e-04 31.3%┣█████████▍                    ┫ 313/1.0k [01:08<02:29, 5it/s]
-
9.528702e-04 31.5%┣█████████▍                    ┫ 315/1.0k [01:08<02:28, 5it/s]
-
9.376882e-04 31.7%┣█████████▌                    ┫ 317/1.0k [01:08<02:27, 5it/s]
-
9.177192e-04 32.0%┣█████████▋                    ┫ 320/1.0k [01:08<02:25, 5it/s]
-
9.100313e-04 32.3%┣█████████▊                    ┫ 323/1.0k [01:08<02:23, 5it/s]
-
9.033554e-04 32.6%┣█████████▊                    ┫ 326/1.0k [01:08<02:21, 5it/s]
-
8.920675e-04 32.9%┣█████████▉                    ┫ 329/1.0k [01:08<02:19, 5it/s]
-
8.844443e-04 33.2%┣██████████                    ┫ 332/1.0k [01:08<02:18, 5it/s]
-
8.785308e-04 33.4%┣██████████                    ┫ 334/1.0k [01:08<02:16, 5it/s]
-
8.707407e-04 33.6%┣██████████                    ┫ 336/1.0k [01:08<02:15, 5it/s]
-
8.617842e-04 33.9%┣██████████▏                   ┫ 339/1.0k [01:08<02:14, 5it/s]
-
8.560114e-04 34.1%┣██████████▏                   ┫ 341/1.0k [01:08<02:13, 5it/s]
-
8.491081e-04 34.3%┣██████████▎                   ┫ 343/1.0k [01:08<02:12, 5it/s]
-
8.404099e-04 34.6%┣██████████▍                   ┫ 346/1.0k [01:08<02:10, 5it/s]
-
8.263324e-04 34.9%┣██████████▌                   ┫ 349/1.0k [01:09<02:08, 5it/s]
-
8.205439e-04 35.1%┣██████████▌                   ┫ 351/1.0k [01:09<02:07, 5it/s]
-
8.077064e-04 35.4%┣██████████▋                   ┫ 354/1.0k [01:09<02:06, 5it/s]
-
8.009866e-04 35.7%┣██████████▊                   ┫ 357/1.0k [01:09<02:04, 5it/s]
-
7.960660e-04 35.9%┣██████████▊                   ┫ 359/1.0k [01:09<02:03, 5it/s]
-
7.895254e-04 36.1%┣██████████▉                   ┫ 361/1.0k [01:09<02:02, 5it/s]
-
7.834084e-04 36.3%┣██████████▉                   ┫ 363/1.0k [01:09<02:01, 5it/s]
-
7.760828e-04 36.5%┣███████████                   ┫ 365/1.0k [01:09<02:00, 5it/s]
-
7.690768e-04 36.8%┣███████████                   ┫ 368/1.0k [01:09<01:59, 5it/s]
-
7.599633e-04 37.0%┣███████████                   ┫ 370/1.0k [01:09<01:58, 5it/s]
-
7.512999e-04 37.2%┣███████████▏                  ┫ 372/1.0k [01:09<01:57, 5it/s]
-
7.450606e-04 37.4%┣███████████▏                  ┫ 374/1.0k [01:09<01:56, 5it/s]
-
7.329791e-04 37.7%┣███████████▎                  ┫ 377/1.0k [01:09<01:55, 5it/s]
-
7.239803e-04 37.9%┣███████████▍                  ┫ 379/1.0k [01:09<01:54, 5it/s]
-
7.159544e-04 38.2%┣███████████▌                  ┫ 382/1.0k [01:09<01:53, 5it/s]
-
7.112233e-04 38.4%┣███████████▌                  ┫ 384/1.0k [01:09<01:52, 6it/s]
-
7.062233e-04 38.6%┣███████████▋                  ┫ 386/1.0k [01:09<01:51, 6it/s]
-
7.007991e-04 38.8%┣███████████▋                  ┫ 388/1.0k [01:09<01:50, 6it/s]
-
6.960482e-04 39.0%┣███████████▊                  ┫ 390/1.0k [01:10<01:49, 6it/s]
-
6.853868e-04 39.3%┣███████████▉                  ┫ 393/1.0k [01:10<01:48, 6it/s]
-
6.800140e-04 39.5%┣███████████▉                  ┫ 395/1.0k [01:10<01:47, 6it/s]
-
6.735989e-04 39.7%┣████████████                  ┫ 397/1.0k [01:10<01:46, 6it/s]
-
6.643462e-04 40.0%┣████████████                  ┫ 400/1.0k [01:10<01:45, 6it/s]
-
6.525757e-04 40.2%┣████████████                  ┫ 402/1.0k [01:10<01:44, 6it/s]
-
6.431165e-04 40.4%┣████████████                  ┫ 404/1.0k [01:10<01:43, 6it/s]
-
6.379063e-04 40.6%┣████████████▏                 ┫ 406/1.0k [01:10<01:43, 6it/s]
-
6.320425e-04 40.8%┣████████████▎                 ┫ 408/1.0k [01:10<01:42, 6it/s]
-
6.223340e-04 41.1%┣████████████▎                 ┫ 411/1.0k [01:10<01:41, 6it/s]
-
6.150668e-04 41.4%┣████████████▍                 ┫ 414/1.0k [01:10<01:40, 6it/s]
-
6.060925e-04 41.7%┣████████████▌                 ┫ 417/1.0k [01:10<01:39, 6it/s]
-
5.993832e-04 42.0%┣████████████▋                 ┫ 420/1.0k [01:10<01:37, 6it/s]
-
5.962235e-04 42.2%┣████████████▋                 ┫ 422/1.0k [01:10<01:37, 6it/s]
-
5.917593e-04 42.5%┣████████████▊                 ┫ 425/1.0k [01:10<01:36, 6it/s]
-
5.867794e-04 42.8%┣████████████▉                 ┫ 428/1.0k [01:10<01:35, 6it/s]
-
5.838672e-04 43.0%┣████████████▉                 ┫ 430/1.0k [01:10<01:34, 6it/s]
-
5.814145e-04 43.2%┣█████████████                 ┫ 432/1.0k [01:11<01:33, 6it/s]
-
5.783157e-04 43.4%┣█████████████                 ┫ 434/1.0k [01:11<01:32, 6it/s]
-
5.748093e-04 43.6%┣█████████████                 ┫ 436/1.0k [01:11<01:32, 6it/s]
-
5.693458e-04 43.8%┣█████████████▏                ┫ 438/1.0k [01:11<01:31, 6it/s]
-
5.663012e-04 44.0%┣█████████████▏                ┫ 440/1.0k [01:11<01:30, 6it/s]
-
5.638448e-04 44.2%┣█████████████▎                ┫ 442/1.0k [01:11<01:30, 6it/s]
-
5.607913e-04 44.4%┣█████████████▎                ┫ 444/1.0k [01:11<01:29, 6it/s]
-
5.552544e-04 44.6%┣█████████████▍                ┫ 446/1.0k [01:11<01:29, 6it/s]
-
5.496969e-04 44.8%┣█████████████▍                ┫ 448/1.0k [01:11<01:28, 6it/s]
-
5.459163e-04 45.0%┣█████████████▌                ┫ 450/1.0k [01:11<01:27, 6it/s]
-
5.415573e-04 45.2%┣█████████████▌                ┫ 452/1.0k [01:11<01:27, 6it/s]
-
5.353353e-04 45.4%┣█████████████▋                ┫ 454/1.0k [01:11<01:26, 6it/s]
-
5.303557e-04 45.6%┣█████████████▋                ┫ 456/1.0k [01:11<01:25, 6it/s]
-
5.268950e-04 45.8%┣█████████████▊                ┫ 458/1.0k [01:11<01:25, 6it/s]
-
5.244228e-04 46.0%┣█████████████▉                ┫ 460/1.0k [01:11<01:24, 6it/s]
-
5.209591e-04 46.2%┣█████████████▉                ┫ 462/1.0k [01:11<01:24, 6it/s]
-
5.174179e-04 46.4%┣██████████████                ┫ 464/1.0k [01:11<01:23, 6it/s]
-
5.136990e-04 46.7%┣██████████████                ┫ 467/1.0k [01:12<01:22, 7it/s]
-
5.104873e-04 46.9%┣██████████████                ┫ 469/1.0k [01:12<01:21, 7it/s]
-
5.070487e-04 47.1%┣██████████████▏               ┫ 471/1.0k [01:12<01:21, 7it/s]
-
5.008912e-04 47.4%┣██████████████▏               ┫ 474/1.0k [01:12<01:20, 7it/s]
-
4.954768e-04 47.7%┣██████████████▎               ┫ 477/1.0k [01:12<01:19, 7it/s]
-
4.906942e-04 47.9%┣██████████████▍               ┫ 479/1.0k [01:12<01:18, 7it/s]
-
4.832220e-04 48.2%┣██████████████▌               ┫ 482/1.0k [01:12<01:18, 7it/s]
-
4.750460e-04 48.5%┣██████████████▌               ┫ 485/1.0k [01:12<01:17, 7it/s]
-
4.709073e-04 48.8%┣██████████████▋               ┫ 488/1.0k [01:12<01:16, 7it/s]
-
4.682238e-04 49.0%┣██████████████▊               ┫ 490/1.0k [01:12<01:15, 7it/s]
-
4.646649e-04 49.2%┣██████████████▊               ┫ 492/1.0k [01:12<01:15, 7it/s]
-
4.611414e-04 49.4%┣██████████████▉               ┫ 494/1.0k [01:12<01:14, 7it/s]
-
4.566801e-04 49.7%┣███████████████               ┫ 497/1.0k [01:12<01:13, 7it/s]
-
4.530857e-04 49.9%┣███████████████               ┫ 499/1.0k [01:12<01:13, 7it/s]
-
4.496869e-04 50.0%┣███████████████               ┫ 501/1.0k [01:12<01:12, 7it/s]
-
4.457215e-04 50.2%┣███████████████               ┫ 503/1.0k [01:12<01:12, 7it/s]
-
4.394494e-04 50.4%┣███████████████▏              ┫ 505/1.0k [01:13<01:11, 7it/s]
-
4.330296e-04 50.7%┣███████████████▎              ┫ 508/1.0k [01:13<01:11, 7it/s]
-
4.316960e-04 50.9%┣███████████████▎              ┫ 510/1.0k [01:13<01:10, 7it/s]
-
4.298618e-04 51.1%┣███████████████▍              ┫ 512/1.0k [01:13<01:10, 7it/s]
-
4.262656e-04 51.4%┣███████████████▍              ┫ 515/1.0k [01:13<01:09, 7it/s]
-
4.208883e-04 51.7%┣███████████████▌              ┫ 518/1.0k [01:13<01:08, 7it/s]
-
4.180467e-04 51.9%┣███████████████▋              ┫ 520/1.0k [01:13<01:08, 7it/s]
-
4.143122e-04 52.1%┣███████████████▋              ┫ 522/1.0k [01:13<01:07, 7it/s]
-
4.094821e-04 52.3%┣███████████████▊              ┫ 524/1.0k [01:13<01:07, 7it/s]
-
4.047974e-04 52.6%┣███████████████▉              ┫ 527/1.0k [01:13<01:06, 7it/s]
-
4.017097e-04 52.8%┣███████████████▉              ┫ 529/1.0k [01:13<01:05, 7it/s]
-
3.977850e-04 53.0%┣████████████████              ┫ 531/1.0k [01:13<01:05, 7it/s]
-
3.936699e-04 53.2%┣████████████████              ┫ 533/1.0k [01:13<01:04, 7it/s]
-
3.914701e-04 53.4%┣████████████████              ┫ 535/1.0k [01:13<01:04, 7it/s]
-
3.873990e-04 53.6%┣████████████████              ┫ 537/1.0k [01:13<01:04, 7it/s]
-
3.835506e-04 53.9%┣████████████████▏             ┫ 540/1.0k [01:13<01:03, 7it/s]
-
3.813744e-04 54.1%┣████████████████▎             ┫ 542/1.0k [01:13<01:02, 7it/s]
-
3.782632e-04 54.3%┣████████████████▎             ┫ 544/1.0k [01:14<01:02, 7it/s]
-
3.717977e-04 54.6%┣████████████████▍             ┫ 547/1.0k [01:14<01:01, 7it/s]
-
3.693352e-04 54.8%┣████████████████▌             ┫ 549/1.0k [01:14<01:01, 7it/s]
-
3.668764e-04 55.1%┣████████████████▌             ┫ 552/1.0k [01:14<01:00, 7it/s]
-
3.653237e-04 55.4%┣████████████████▋             ┫ 555/1.0k [01:14<00:59, 8it/s]
-
3.631468e-04 55.6%┣████████████████▊             ┫ 557/1.0k [01:14<00:59, 8it/s]
-
3.601086e-04 55.8%┣████████████████▊             ┫ 559/1.0k [01:14<00:59, 8it/s]
-
3.572064e-04 56.0%┣████████████████▉             ┫ 561/1.0k [01:14<00:58, 8it/s]
-
3.541459e-04 56.2%┣████████████████▉             ┫ 563/1.0k [01:14<00:58, 8it/s]
-
3.503545e-04 56.5%┣█████████████████             ┫ 566/1.0k [01:14<00:57, 8it/s]
-
3.478622e-04 56.7%┣█████████████████             ┫ 568/1.0k [01:14<00:57, 8it/s]
-
3.435254e-04 57.0%┣█████████████████▏            ┫ 571/1.0k [01:14<00:56, 8it/s]
-
3.413669e-04 57.2%┣█████████████████▏            ┫ 573/1.0k [01:14<00:56, 8it/s]
-
3.378498e-04 57.4%┣█████████████████▎            ┫ 575/1.0k [01:14<00:55, 8it/s]
-
3.361546e-04 57.6%┣█████████████████▎            ┫ 577/1.0k [01:14<00:55, 8it/s]
-
3.332255e-04 57.9%┣█████████████████▍            ┫ 580/1.0k [01:15<00:54, 8it/s]
-
3.314775e-04 58.2%┣█████████████████▌            ┫ 583/1.0k [01:15<00:54, 8it/s]
-
3.301391e-04 58.4%┣█████████████████▌            ┫ 585/1.0k [01:15<00:53, 8it/s]
-
3.281049e-04 58.6%┣█████████████████▋            ┫ 587/1.0k [01:15<00:53, 8it/s]
-
3.249095e-04 58.9%┣█████████████████▊            ┫ 590/1.0k [01:15<00:52, 8it/s]
-
3.224021e-04 59.1%┣█████████████████▊            ┫ 592/1.0k [01:15<00:52, 8it/s]
-
3.187947e-04 59.4%┣█████████████████▉            ┫ 595/1.0k [01:15<00:51, 8it/s]
-
3.162909e-04 59.6%┣██████████████████            ┫ 597/1.0k [01:15<00:51, 8it/s]
-
3.130435e-04 59.8%┣██████████████████            ┫ 599/1.0k [01:15<00:50, 8it/s]
-
3.071982e-04 60.1%┣██████████████████            ┫ 602/1.0k [01:15<00:50, 8it/s]
-
3.053460e-04 60.3%┣██████████████████            ┫ 604/1.0k [01:15<00:49, 8it/s]
-
3.035575e-04 60.5%┣██████████████████▏           ┫ 606/1.0k [01:15<00:49, 8it/s]
-
2.998633e-04 60.7%┣██████████████████▏           ┫ 608/1.0k [01:15<00:49, 8it/s]
-
2.944393e-04 61.0%┣██████████████████▎           ┫ 611/1.0k [01:15<00:48, 8it/s]
-
2.913783e-04 61.3%┣██████████████████▍           ┫ 614/1.0k [01:15<00:48, 8it/s]
-
2.888287e-04 61.6%┣██████████████████▌           ┫ 617/1.0k [01:15<00:47, 8it/s]
-
2.855880e-04 61.9%┣██████████████████▋           ┫ 620/1.0k [01:15<00:46, 8it/s]
-
2.829773e-04 62.2%┣██████████████████▊           ┫ 623/1.0k [01:16<00:46, 8it/s]
-
2.806563e-04 62.5%┣██████████████████▊           ┫ 626/1.0k [01:16<00:45, 8it/s]
-
2.773991e-04 62.8%┣██████████████████▉           ┫ 629/1.0k [01:16<00:45, 8it/s]
-
2.739563e-04 63.1%┣███████████████████           ┫ 632/1.0k [01:16<00:44, 8it/s]
-
2.720634e-04 63.3%┣███████████████████           ┫ 634/1.0k [01:16<00:44, 8it/s]
-
2.702395e-04 63.5%┣███████████████████           ┫ 636/1.0k [01:16<00:44, 8it/s]
-
2.688961e-04 63.7%┣███████████████████▏          ┫ 638/1.0k [01:16<00:43, 8it/s]
-
2.672620e-04 63.9%┣███████████████████▏          ┫ 640/1.0k [01:16<00:43, 8it/s]
-
2.656907e-04 64.1%┣███████████████████▎          ┫ 642/1.0k [01:16<00:43, 8it/s]
-
2.641670e-04 64.3%┣███████████████████▎          ┫ 644/1.0k [01:16<00:42, 8it/s]
-
2.621320e-04 64.5%┣███████████████████▍          ┫ 646/1.0k [01:16<00:42, 8it/s]
-
2.604791e-04 64.8%┣███████████████████▌          ┫ 649/1.0k [01:16<00:41, 9it/s]
-
2.593369e-04 65.0%┣███████████████████▌          ┫ 651/1.0k [01:16<00:41, 9it/s]
-
2.582117e-04 65.2%┣███████████████████▋          ┫ 653/1.0k [01:16<00:41, 9it/s]
-
2.567020e-04 65.4%┣███████████████████▋          ┫ 655/1.0k [01:16<00:40, 9it/s]
-
2.555457e-04 65.6%┣███████████████████▊          ┫ 657/1.0k [01:16<00:40, 9it/s]
-
2.543599e-04 65.8%┣███████████████████▊          ┫ 659/1.0k [01:17<00:40, 9it/s]
-
2.531858e-04 66.0%┣███████████████████▉          ┫ 661/1.0k [01:17<00:39, 9it/s]
-
2.522472e-04 66.2%┣███████████████████▉          ┫ 663/1.0k [01:17<00:39, 9it/s]
-
2.511479e-04 66.4%┣████████████████████          ┫ 665/1.0k [01:17<00:39, 9it/s]
-
2.493217e-04 66.6%┣████████████████████          ┫ 667/1.0k [01:17<00:38, 9it/s]
-
2.475964e-04 66.9%┣████████████████████          ┫ 670/1.0k [01:17<00:38, 9it/s]
-
2.467428e-04 67.1%┣████████████████████▏         ┫ 672/1.0k [01:17<00:38, 9it/s]
-
2.454921e-04 67.3%┣████████████████████▏         ┫ 674/1.0k [01:17<00:37, 9it/s]
-
2.448745e-04 67.5%┣████████████████████▎         ┫ 676/1.0k [01:17<00:37, 9it/s]
-
2.438153e-04 67.7%┣████████████████████▎         ┫ 678/1.0k [01:17<00:37, 9it/s]
-
2.426388e-04 67.9%┣████████████████████▍         ┫ 680/1.0k [01:17<00:36, 9it/s]
-
2.415426e-04 68.1%┣████████████████████▍         ┫ 682/1.0k [01:17<00:36, 9it/s]
-
2.405574e-04 68.4%┣████████████████████▌         ┫ 685/1.0k [01:17<00:36, 9it/s]
-
2.397665e-04 68.6%┣████████████████████▋         ┫ 687/1.0k [01:17<00:35, 9it/s]
-
2.389815e-04 68.8%┣████████████████████▋         ┫ 689/1.0k [01:17<00:35, 9it/s]
-
2.374824e-04 69.0%┣████████████████████▊         ┫ 691/1.0k [01:17<00:35, 9it/s]
-
2.367565e-04 69.2%┣████████████████████▊         ┫ 693/1.0k [01:17<00:34, 9it/s]
-
2.362184e-04 69.4%┣████████████████████▉         ┫ 695/1.0k [01:17<00:34, 9it/s]
-
2.352693e-04 69.6%┣█████████████████████         ┫ 697/1.0k [01:18<00:34, 9it/s]
-
2.343886e-04 69.8%┣█████████████████████         ┫ 699/1.0k [01:18<00:34, 9it/s]
-
2.334759e-04 70.0%┣█████████████████████         ┫ 701/1.0k [01:18<00:33, 9it/s]
-
2.321317e-04 70.2%┣█████████████████████         ┫ 703/1.0k [01:18<00:33, 9it/s]
-
2.309833e-04 70.4%┣█████████████████████▏        ┫ 705/1.0k [01:18<00:33, 9it/s]
-
2.299639e-04 70.6%┣█████████████████████▏        ┫ 707/1.0k [01:18<00:32, 9it/s]
-
2.286682e-04 70.8%┣█████████████████████▎        ┫ 709/1.0k [01:18<00:32, 9it/s]
-
2.266541e-04 71.1%┣█████████████████████▍        ┫ 712/1.0k [01:18<00:32, 9it/s]
-
2.258841e-04 71.3%┣█████████████████████▍        ┫ 714/1.0k [01:18<00:31, 9it/s]
-
2.251062e-04 71.5%┣█████████████████████▌        ┫ 716/1.0k [01:18<00:31, 9it/s]
-
2.244289e-04 71.7%┣█████████████████████▌        ┫ 718/1.0k [01:18<00:31, 9it/s]
-
2.234397e-04 71.9%┣█████████████████████▋        ┫ 720/1.0k [01:18<00:31, 9it/s]
-
2.225749e-04 72.1%┣█████████████████████▋        ┫ 722/1.0k [01:18<00:30, 9it/s]
-
2.216594e-04 72.3%┣█████████████████████▊        ┫ 724/1.0k [01:18<00:30, 9it/s]
-
2.205517e-04 72.6%┣█████████████████████▉        ┫ 727/1.0k [01:18<00:30, 9it/s]
-
2.200095e-04 72.8%┣█████████████████████▉        ┫ 729/1.0k [01:18<00:29, 9it/s]
-
2.191683e-04 73.0%┣██████████████████████        ┫ 731/1.0k [01:19<00:29, 9it/s]
-
2.185053e-04 73.2%┣██████████████████████        ┫ 733/1.0k [01:19<00:29, 9it/s]
-
2.173565e-04 73.4%┣██████████████████████        ┫ 735/1.0k [01:19<00:28, 9it/s]
-
2.161869e-04 73.6%┣██████████████████████        ┫ 737/1.0k [01:19<00:28, 9it/s]
-
2.154514e-04 73.9%┣██████████████████████▏       ┫ 740/1.0k [01:19<00:28, 9it/s]
-
2.150113e-04 74.1%┣██████████████████████▎       ┫ 742/1.0k [01:19<00:28, 9it/s]
-
2.144559e-04 74.3%┣██████████████████████▎       ┫ 744/1.0k [01:19<00:27, 9it/s]
-
2.137773e-04 74.5%┣██████████████████████▍       ┫ 746/1.0k [01:19<00:27, 9it/s]
-
2.129452e-04 74.7%┣██████████████████████▍       ┫ 748/1.0k [01:19<00:27, 9it/s]
-
2.121851e-04 74.9%┣██████████████████████▌       ┫ 750/1.0k [01:19<00:26, 9it/s]
-
2.114434e-04 75.1%┣██████████████████████▌       ┫ 752/1.0k [01:19<00:26, 9it/s]
-
2.103982e-04 75.3%┣█████████████████████▉       ┫ 754/1.0k [01:19<00:26, 10it/s]
-
2.092470e-04 75.5%┣██████████████████████       ┫ 756/1.0k [01:19<00:26, 10it/s]
-
2.079654e-04 75.7%┣██████████████████████       ┫ 758/1.0k [01:19<00:25, 10it/s]
-
2.067708e-04 76.0%┣██████████████████████       ┫ 761/1.0k [01:19<00:25, 10it/s]
-
2.059972e-04 76.3%┣██████████████████████▏      ┫ 764/1.0k [01:19<00:25, 10it/s]
-
2.054426e-04 76.5%┣██████████████████████▏      ┫ 766/1.0k [01:20<00:24, 10it/s]
-
2.044409e-04 76.7%┣██████████████████████▎      ┫ 768/1.0k [01:20<00:24, 10it/s]
-
2.032992e-04 76.9%┣██████████████████████▎      ┫ 770/1.0k [01:20<00:24, 10it/s]
-
2.019500e-04 77.1%┣██████████████████████▍      ┫ 772/1.0k [01:20<00:24, 10it/s]
-
2.008197e-04 77.3%┣██████████████████████▍      ┫ 774/1.0k [01:20<00:23, 10it/s]
-
1.998014e-04 77.6%┣██████████████████████▌      ┫ 777/1.0k [01:20<00:23, 10it/s]
-
1.986508e-04 77.8%┣██████████████████████▋      ┫ 779/1.0k [01:20<00:23, 10it/s]
-
1.978787e-04 78.0%┣██████████████████████▋      ┫ 781/1.0k [01:20<00:23, 10it/s]
-
1.970807e-04 78.2%┣██████████████████████▊      ┫ 783/1.0k [01:20<00:22, 10it/s]
-
1.961179e-04 78.4%┣██████████████████████▊      ┫ 785/1.0k [01:20<00:22, 10it/s]
-
1.944373e-04 78.7%┣██████████████████████▉      ┫ 788/1.0k [01:20<00:22, 10it/s]
-
1.934498e-04 78.9%┣██████████████████████▉      ┫ 790/1.0k [01:20<00:21, 10it/s]
-
1.921643e-04 79.2%┣███████████████████████      ┫ 793/1.0k [01:20<00:21, 10it/s]
-
1.911937e-04 79.4%┣███████████████████████      ┫ 795/1.0k [01:20<00:21, 10it/s]
-
1.900830e-04 79.6%┣███████████████████████      ┫ 797/1.0k [01:20<00:21, 10it/s]
-
1.880063e-04 79.9%┣███████████████████████▏     ┫ 800/1.0k [01:20<00:20, 10it/s]
-
1.855283e-04 80.2%┣███████████████████████▎     ┫ 803/1.0k [01:20<00:20, 10it/s]
-
1.837304e-04 80.5%┣███████████████████████▍     ┫ 806/1.0k [01:21<00:20, 10it/s]
-
1.825574e-04 80.8%┣███████████████████████▍     ┫ 809/1.0k [01:21<00:19, 10it/s]
-
1.814801e-04 81.1%┣███████████████████████▌     ┫ 812/1.0k [01:21<00:19, 10it/s]
-
1.805289e-04 81.4%┣███████████████████████▋     ┫ 815/1.0k [01:21<00:18, 10it/s]
-
1.799763e-04 81.6%┣███████████████████████▊     ┫ 817/1.0k [01:21<00:18, 10it/s]
-
1.791603e-04 81.8%┣███████████████████████▊     ┫ 819/1.0k [01:21<00:18, 10it/s]
-
1.784035e-04 82.0%┣███████████████████████▉     ┫ 821/1.0k [01:21<00:18, 10it/s]
-
1.776861e-04 82.2%┣███████████████████████▉     ┫ 823/1.0k [01:21<00:18, 10it/s]
-
1.766647e-04 82.5%┣████████████████████████     ┫ 826/1.0k [01:21<00:17, 10it/s]
-
1.760345e-04 82.7%┣████████████████████████     ┫ 828/1.0k [01:21<00:17, 10it/s]
-
1.754529e-04 82.9%┣████████████████████████     ┫ 830/1.0k [01:21<00:17, 10it/s]
-
1.746625e-04 83.2%┣████████████████████████▏    ┫ 833/1.0k [01:21<00:16, 10it/s]
-
1.739374e-04 83.5%┣████████████████████████▏    ┫ 836/1.0k [01:21<00:16, 10it/s]
-
1.732493e-04 83.7%┣████████████████████████▎    ┫ 838/1.0k [01:21<00:16, 10it/s]
-
1.723180e-04 83.9%┣████████████████████████▍    ┫ 840/1.0k [01:21<00:16, 10it/s]
-
1.713446e-04 84.1%┣████████████████████████▍    ┫ 842/1.0k [01:21<00:15, 10it/s]
-
1.707809e-04 84.3%┣████████████████████████▌    ┫ 844/1.0k [01:21<00:15, 10it/s]
-
1.701937e-04 84.5%┣████████████████████████▌    ┫ 846/1.0k [01:22<00:15, 10it/s]
-
1.696638e-04 84.7%┣████████████████████████▋    ┫ 848/1.0k [01:22<00:15, 10it/s]
-
1.688878e-04 84.9%┣████████████████████████▋    ┫ 850/1.0k [01:22<00:15, 10it/s]
-
1.681859e-04 85.1%┣████████████████████████▊    ┫ 852/1.0k [01:22<00:14, 10it/s]
-
1.677416e-04 85.3%┣████████████████████████▊    ┫ 854/1.0k [01:22<00:14, 10it/s]
-
1.672131e-04 85.5%┣████████████████████████▉    ┫ 856/1.0k [01:22<00:14, 10it/s]
-
1.665961e-04 85.7%┣████████████████████████▉    ┫ 858/1.0k [01:22<00:14, 10it/s]
-
1.657684e-04 85.9%┣█████████████████████████    ┫ 860/1.0k [01:22<00:13, 10it/s]
-
1.642508e-04 86.1%┣█████████████████████████    ┫ 862/1.0k [01:22<00:13, 10it/s]
-
1.632633e-04 86.3%┣█████████████████████████    ┫ 864/1.0k [01:22<00:13, 11it/s]
-
1.626629e-04 86.5%┣█████████████████████████    ┫ 866/1.0k [01:22<00:13, 11it/s]
-
1.621260e-04 86.7%┣█████████████████████████▏   ┫ 868/1.0k [01:22<00:13, 11it/s]
-
1.615468e-04 86.9%┣█████████████████████████▏   ┫ 870/1.0k [01:22<00:12, 11it/s]
-
1.609843e-04 87.1%┣█████████████████████████▎   ┫ 872/1.0k [01:22<00:12, 11it/s]
-
1.604832e-04 87.3%┣█████████████████████████▎   ┫ 874/1.0k [01:22<00:12, 11it/s]
-
1.599156e-04 87.5%┣█████████████████████████▍   ┫ 876/1.0k [01:22<00:12, 11it/s]
-
1.584867e-04 87.8%┣█████████████████████████▌   ┫ 879/1.0k [01:22<00:11, 11it/s]
-
1.573700e-04 88.0%┣█████████████████████████▌   ┫ 881/1.0k [01:23<00:11, 11it/s]
-
1.557890e-04 88.3%┣█████████████████████████▋   ┫ 884/1.0k [01:23<00:11, 11it/s]
-
1.550056e-04 88.6%┣█████████████████████████▊   ┫ 887/1.0k [01:23<00:11, 11it/s]
-
1.547515e-04 88.8%┣█████████████████████████▊   ┫ 889/1.0k [01:23<00:10, 11it/s]
-
1.542813e-04 89.0%┣█████████████████████████▉   ┫ 891/1.0k [01:23<00:10, 11it/s]
-
1.536939e-04 89.2%┣█████████████████████████▉   ┫ 893/1.0k [01:23<00:10, 11it/s]
-
1.531744e-04 89.4%┣██████████████████████████   ┫ 895/1.0k [01:23<00:10, 11it/s]
-
1.525314e-04 89.6%┣██████████████████████████   ┫ 897/1.0k [01:23<00:10, 11it/s]
-
1.518095e-04 89.8%┣██████████████████████████   ┫ 899/1.0k [01:23<00:09, 11it/s]
-
1.509766e-04 90.0%┣██████████████████████████   ┫ 901/1.0k [01:23<00:09, 11it/s]
-
1.504769e-04 90.2%┣██████████████████████████▏  ┫ 903/1.0k [01:23<00:09, 11it/s]
-
1.496763e-04 90.4%┣██████████████████████████▏  ┫ 905/1.0k [01:23<00:09, 11it/s]
-
1.491099e-04 90.6%┣██████████████████████████▎  ┫ 907/1.0k [01:23<00:09, 11it/s]
-
1.487084e-04 90.8%┣██████████████████████████▍  ┫ 909/1.0k [01:23<00:08, 11it/s]
-
1.483214e-04 91.0%┣██████████████████████████▍  ┫ 911/1.0k [01:23<00:08, 11it/s]
-
1.478666e-04 91.2%┣██████████████████████████▌  ┫ 913/1.0k [01:23<00:08, 11it/s]
-
1.473556e-04 91.4%┣██████████████████████████▌  ┫ 915/1.0k [01:23<00:08, 11it/s]
-
1.465242e-04 91.6%┣██████████████████████████▋  ┫ 917/1.0k [01:24<00:08, 11it/s]
-
1.460172e-04 91.8%┣██████████████████████████▋  ┫ 919/1.0k [01:24<00:07, 11it/s]
-
1.455952e-04 92.0%┣██████████████████████████▊  ┫ 921/1.0k [01:24<00:07, 11it/s]
-
1.451896e-04 92.2%┣██████████████████████████▊  ┫ 923/1.0k [01:24<00:07, 11it/s]
-
1.445827e-04 92.4%┣██████████████████████████▉  ┫ 925/1.0k [01:24<00:07, 11it/s]
-
1.440268e-04 92.6%┣██████████████████████████▉  ┫ 927/1.0k [01:24<00:07, 11it/s]
-
1.435430e-04 92.8%┣███████████████████████████  ┫ 929/1.0k [01:24<00:07, 11it/s]
-
1.430115e-04 93.0%┣███████████████████████████  ┫ 931/1.0k [01:24<00:06, 11it/s]
-
1.426247e-04 93.2%┣███████████████████████████  ┫ 933/1.0k [01:24<00:06, 11it/s]
-
1.421552e-04 93.4%┣███████████████████████████  ┫ 935/1.0k [01:24<00:06, 11it/s]
-
1.412139e-04 93.7%┣███████████████████████████▏ ┫ 938/1.0k [01:24<00:06, 11it/s]
-
1.403481e-04 93.9%┣███████████████████████████▎ ┫ 940/1.0k [01:24<00:05, 11it/s]
-
1.386473e-04 94.2%┣███████████████████████████▎ ┫ 943/1.0k [01:24<00:05, 11it/s]
-
1.377067e-04 94.4%┣███████████████████████████▍ ┫ 945/1.0k [01:24<00:05, 11it/s]
-
1.367993e-04 94.7%┣███████████████████████████▌ ┫ 948/1.0k [01:24<00:05, 11it/s]
-
1.364788e-04 94.9%┣███████████████████████████▌ ┫ 950/1.0k [01:24<00:05, 11it/s]
-
1.360804e-04 95.1%┣███████████████████████████▋ ┫ 952/1.0k [01:24<00:04, 11it/s]
-
1.355994e-04 95.3%┣███████████████████████████▋ ┫ 954/1.0k [01:25<00:04, 11it/s]
-
1.352329e-04 95.5%┣███████████████████████████▊ ┫ 956/1.0k [01:25<00:04, 11it/s]
-
1.346543e-04 95.7%┣███████████████████████████▊ ┫ 958/1.0k [01:25<00:04, 11it/s]
-
1.339780e-04 95.9%┣███████████████████████████▉ ┫ 960/1.0k [01:25<00:04, 11it/s]
-
1.334225e-04 96.1%┣███████████████████████████▉ ┫ 962/1.0k [01:25<00:03, 11it/s]
-
1.324024e-04 96.4%┣████████████████████████████ ┫ 965/1.0k [01:25<00:03, 11it/s]
-
1.316228e-04 96.6%┣████████████████████████████ ┫ 967/1.0k [01:25<00:03, 11it/s]
-
1.304832e-04 96.9%┣████████████████████████████ ┫ 970/1.0k [01:25<00:03, 11it/s]
-
1.300069e-04 97.1%┣████████████████████████████▏┫ 972/1.0k [01:25<00:03, 11it/s]
-
1.294757e-04 97.4%┣████████████████████████████▎┫ 975/1.0k [01:25<00:02, 11it/s]
-
1.290953e-04 97.6%┣████████████████████████████▎┫ 977/1.0k [01:25<00:02, 11it/s]
-
1.284962e-04 97.8%┣████████████████████████████▍┫ 979/1.0k [01:25<00:02, 11it/s]
-
1.280348e-04 98.0%┣████████████████████████████▍┫ 981/1.0k [01:25<00:02, 11it/s]
-
1.274089e-04 98.2%┣████████████████████████████▌┫ 983/1.0k [01:25<00:02, 12it/s]
-
1.266838e-04 98.4%┣████████████████████████████▌┫ 985/1.0k [01:25<00:01, 12it/s]
-
1.259390e-04 98.7%┣████████████████████████████▋┫ 988/1.0k [01:25<00:01, 12it/s]
-
1.256323e-04 98.9%┣████████████████████████████▊┫ 990/1.0k [01:25<00:01, 12it/s]
-
1.252614e-04 99.1%┣████████████████████████████▊┫ 992/1.0k [01:26<00:01, 12it/s]
-
1.248389e-04 99.3%┣████████████████████████████▉┫ 994/1.0k [01:26<00:01, 12it/s]
-
1.241301e-04 99.5%┣████████████████████████████▉┫ 996/1.0k [01:26<00:00, 12it/s]
-
1.228733e-04 99.8%┣█████████████████████████████┫ 999/1.0k [01:26<00:00, 12it/s]
-
1.221060e-04 100.0%┣███████████████████████████┫ 1.0k/1.0k [01:26<00:00, 12it/s]

+fig

0.1%┣                                         ┫ 1/1.0k [01:04<Inf:Inf, InfGs/it]
+
4.577653e+02 0.2%┣                             ┫ 2/1.0k [01:04<17:42:13, 64s/it]
+
1.259814e+02 0.3%┣                             ┫ 3/1.0k [01:04<08:51:22, 32s/it]
+
6.024231e+01 0.4%┣▏                            ┫ 4/1.0k [01:04<05:54:19, 21s/it]
+
2.221092e+01 0.6%┣▏                            ┫ 6/1.0k [01:04<03:32:21, 13s/it]
+
1.320887e+01 0.8%┣▎                             ┫ 8/1.0k [01:04<02:31:35, 9s/it]
+
7.304793e+00 1.0%┣▎                            ┫ 10/1.0k [01:04<01:57:49, 7s/it]
+
4.736495e+00 1.2%┣▍                            ┫ 12/1.0k [01:04<01:36:18, 6s/it]
+
3.389967e+00 1.4%┣▍                            ┫ 14/1.0k [01:04<01:21:24, 5s/it]
+
3.033198e+00 1.6%┣▌                            ┫ 16/1.0k [01:04<01:10:28, 4s/it]
+
2.448815e+00 1.9%┣▋                               ┫ 19/1.0k [01:04<58:36, 4s/it]
+
2.085824e+00 2.1%┣▊                               ┫ 21/1.0k [01:04<52:40, 3s/it]
+
1.612337e+00 2.3%┣▊                               ┫ 23/1.0k [01:05<47:50, 3s/it]
+
1.406260e+00 2.6%┣▉                               ┫ 26/1.0k [01:05<42:01, 3s/it]
+
1.161345e+00 2.9%┣█                               ┫ 29/1.0k [01:05<37:26, 2s/it]
+
1.047384e+00 3.1%┣█                               ┫ 31/1.0k [01:05<34:55, 2s/it]
+
9.297605e-01 3.3%┣█                               ┫ 33/1.0k [01:05<32:41, 2s/it]
+
8.527217e-01 3.5%┣█▏                              ┫ 35/1.0k [01:05<30:44, 2s/it]
+
7.548066e-01 3.7%┣█▏                              ┫ 37/1.0k [01:05<28:59, 2s/it]
+
7.201363e-01 3.9%┣█▎                              ┫ 39/1.0k [01:05<27:25, 2s/it]
+
6.592581e-01 4.1%┣█▎                              ┫ 41/1.0k [01:05<26:01, 2s/it]
+
5.451504e-01 4.3%┣█▍                              ┫ 43/1.0k [01:05<24:45, 2s/it]
+
4.586885e-01 4.5%┣█▍                              ┫ 45/1.0k [01:05<23:36, 1s/it]
+
3.950271e-01 4.7%┣█▌                              ┫ 47/1.0k [01:05<22:33, 1s/it]
+
3.190123e-01 4.9%┣█▋                              ┫ 49/1.0k [01:05<21:35, 1s/it]
+
2.538515e-01 5.1%┣█▋                              ┫ 51/1.0k [01:05<20:42, 1s/it]
+
2.152797e-01 5.4%┣█▊                              ┫ 54/1.0k [01:05<19:29, 1s/it]
+
1.892988e-01 5.7%┣█▉                              ┫ 57/1.0k [01:05<18:24, 1s/it]
+
1.757319e-01 6.0%┣██                              ┫ 60/1.0k [01:06<17:25, 1s/it]
+
1.539942e-01 6.2%┣██                              ┫ 62/1.0k [01:06<16:50, 1s/it]
+
1.346207e-01 6.4%┣██                              ┫ 64/1.0k [01:06<16:16, 1s/it]
+
1.203151e-01 6.7%┣██▏                             ┫ 67/1.0k [01:06<15:30, 1it/s]
+
1.134274e-01 7.0%┣██▎                             ┫ 70/1.0k [01:06<14:48, 1it/s]
+
9.171586e-02 7.2%┣██▎                             ┫ 72/1.0k [01:06<14:22, 1it/s]
+
8.281120e-02 7.5%┣██▍                             ┫ 75/1.0k [01:06<13:45, 1it/s]
+
7.706703e-02 7.7%┣██▌                             ┫ 77/1.0k [01:06<13:22, 1it/s]
+
6.736247e-02 8.0%┣██▋                             ┫ 80/1.0k [01:06<12:50, 1it/s]
+
5.858664e-02 8.2%┣██▋                             ┫ 82/1.0k [01:06<12:30, 1it/s]
+
5.533483e-02 8.5%┣██▊                             ┫ 85/1.0k [01:06<12:01, 1it/s]
+
5.031098e-02 8.8%┣██▉                             ┫ 88/1.0k [01:06<11:35, 1it/s]
+
4.437209e-02 9.1%┣███                             ┫ 91/1.0k [01:06<11:10, 1it/s]
+
3.981425e-02 9.4%┣███                             ┫ 94/1.0k [01:06<10:47, 1it/s]
+
3.638520e-02 9.7%┣███                             ┫ 97/1.0k [01:06<10:25, 1it/s]
+
3.194753e-02 10.0%┣███                           ┫ 100/1.0k [01:06<10:05, 1it/s]
+
2.946877e-02 10.3%┣███                           ┫ 103/1.0k [01:07<09:46, 2it/s]
+
2.668139e-02 10.6%┣███▏                          ┫ 106/1.0k [01:07<09:28, 2it/s]
+
2.470283e-02 10.9%┣███▎                          ┫ 109/1.0k [01:07<09:11, 2it/s]
+
2.298622e-02 11.2%┣███▍                          ┫ 112/1.0k [01:07<08:54, 2it/s]
+
1.954464e-02 11.5%┣███▌                          ┫ 115/1.0k [01:07<08:39, 2it/s]
+
1.763914e-02 11.8%┣███▌                          ┫ 118/1.0k [01:07<08:25, 2it/s]
+
1.555628e-02 12.1%┣███▋                          ┫ 121/1.0k [01:07<08:11, 2it/s]
+
1.431829e-02 12.4%┣███▊                          ┫ 124/1.0k [01:07<07:58, 2it/s]
+
1.304293e-02 12.7%┣███▉                          ┫ 127/1.0k [01:07<07:45, 2it/s]
+
1.195425e-02 13.0%┣████                          ┫ 130/1.0k [01:07<07:33, 2it/s]
+
1.093359e-02 13.3%┣████                          ┫ 133/1.0k [01:07<07:22, 2it/s]
+
9.952296e-03 13.6%┣████                          ┫ 136/1.0k [01:07<07:11, 2it/s]
+
9.160037e-03 13.9%┣████▏                         ┫ 139/1.0k [01:07<07:00, 2it/s]
+
8.522577e-03 14.2%┣████▎                         ┫ 142/1.0k [01:07<06:50, 2it/s]
+
8.087275e-03 14.5%┣████▍                         ┫ 145/1.0k [01:07<06:41, 2it/s]
+
7.717920e-03 14.8%┣████▍                         ┫ 148/1.0k [01:07<06:32, 2it/s]
+
7.316644e-03 15.1%┣████▌                         ┫ 151/1.0k [01:08<06:23, 2it/s]
+
6.916761e-03 15.4%┣████▋                         ┫ 154/1.0k [01:08<06:14, 2it/s]
+
6.546048e-03 15.7%┣████▊                         ┫ 157/1.0k [01:08<06:06, 2it/s]
+
6.199308e-03 16.0%┣████▉                         ┫ 160/1.0k [01:08<05:58, 2it/s]
+
5.830124e-03 16.3%┣████▉                         ┫ 163/1.0k [01:08<05:51, 2it/s]
+
5.584419e-03 16.6%┣█████                         ┫ 166/1.0k [01:08<05:43, 2it/s]
+
5.312865e-03 16.9%┣█████                         ┫ 169/1.0k [01:08<05:36, 2it/s]
+
5.096707e-03 17.2%┣█████▏                        ┫ 172/1.0k [01:08<05:30, 3it/s]
+
4.841132e-03 17.5%┣█████▎                        ┫ 175/1.0k [01:08<05:23, 3it/s]
+
4.583713e-03 17.8%┣█████▍                        ┫ 178/1.0k [01:08<05:17, 3it/s]
+
4.314121e-03 18.1%┣█████▍                        ┫ 181/1.0k [01:08<05:11, 3it/s]
+
4.085616e-03 18.4%┣█████▌                        ┫ 184/1.0k [01:08<05:05, 3it/s]
+
3.902945e-03 18.7%┣█████▋                        ┫ 187/1.0k [01:08<04:59, 3it/s]
+
3.733916e-03 19.0%┣█████▊                        ┫ 190/1.0k [01:08<04:53, 3it/s]
+
3.557297e-03 19.3%┣█████▉                        ┫ 193/1.0k [01:08<04:48, 3it/s]
+
3.384605e-03 19.6%┣█████▉                        ┫ 196/1.0k [01:08<04:43, 3it/s]
+
3.224933e-03 19.9%┣██████                        ┫ 199/1.0k [01:09<04:38, 3it/s]
+
3.058470e-03 20.2%┣██████                        ┫ 202/1.0k [01:09<04:33, 3it/s]
+
2.935248e-03 20.5%┣██████▏                       ┫ 205/1.0k [01:09<04:28, 3it/s]
+
2.826676e-03 20.8%┣██████▎                       ┫ 208/1.0k [01:09<04:23, 3it/s]
+
2.615538e-03 21.1%┣██████▎                       ┫ 211/1.0k [01:09<04:19, 3it/s]
+
2.448796e-03 21.4%┣██████▍                       ┫ 214/1.0k [01:09<04:14, 3it/s]
+
2.364887e-03 21.7%┣██████▌                       ┫ 217/1.0k [01:09<04:10, 3it/s]
+
2.309433e-03 22.0%┣██████▋                       ┫ 220/1.0k [01:09<04:06, 3it/s]
+
2.262747e-03 22.3%┣██████▊                       ┫ 223/1.0k [01:09<04:02, 3it/s]
+
2.218369e-03 22.6%┣██████▊                       ┫ 226/1.0k [01:09<03:58, 3it/s]
+
2.183487e-03 22.9%┣██████▉                       ┫ 229/1.0k [01:09<03:54, 3it/s]
+
2.145058e-03 23.2%┣███████                       ┫ 232/1.0k [01:09<03:50, 3it/s]
+
2.092868e-03 23.5%┣███████                       ┫ 235/1.0k [01:09<03:47, 3it/s]
+
2.041655e-03 23.8%┣███████▏                      ┫ 238/1.0k [01:09<03:43, 3it/s]
+
1.982744e-03 24.1%┣███████▎                      ┫ 241/1.0k [01:09<03:40, 3it/s]
+
1.948201e-03 24.4%┣███████▎                      ┫ 244/1.0k [01:09<03:36, 3it/s]
+
1.923729e-03 24.7%┣███████▍                      ┫ 247/1.0k [01:10<03:33, 4it/s]
+
1.890085e-03 25.0%┣███████▌                      ┫ 250/1.0k [01:10<03:30, 4it/s]
+
1.865692e-03 25.3%┣███████▋                      ┫ 253/1.0k [01:10<03:27, 4it/s]
+
1.841220e-03 25.6%┣███████▊                      ┫ 256/1.0k [01:10<03:24, 4it/s]
+
1.820052e-03 25.9%┣███████▊                      ┫ 259/1.0k [01:10<03:21, 4it/s]
+
1.798225e-03 26.2%┣███████▉                      ┫ 262/1.0k [01:10<03:18, 4it/s]
+
1.776374e-03 26.5%┣████████                      ┫ 265/1.0k [01:10<03:15, 4it/s]
+
1.747196e-03 26.8%┣████████                      ┫ 268/1.0k [01:10<03:12, 4it/s]
+
1.727834e-03 27.1%┣████████▏                     ┫ 271/1.0k [01:10<03:09, 4it/s]
+
1.710449e-03 27.3%┣████████▏                     ┫ 273/1.0k [01:10<03:07, 4it/s]
+
1.684340e-03 27.6%┣████████▎                     ┫ 276/1.0k [01:10<03:05, 4it/s]
+
1.665194e-03 27.9%┣████████▍                     ┫ 279/1.0k [01:10<03:02, 4it/s]
+
1.647521e-03 28.2%┣████████▌                     ┫ 282/1.0k [01:10<03:00, 4it/s]
+
1.632059e-03 28.5%┣████████▌                     ┫ 285/1.0k [01:10<02:57, 4it/s]
+
1.609423e-03 28.8%┣████████▋                     ┫ 288/1.0k [01:10<02:55, 4it/s]
+
1.591914e-03 29.1%┣████████▊                     ┫ 291/1.0k [01:10<02:52, 4it/s]
+
1.567001e-03 29.4%┣████████▉                     ┫ 294/1.0k [01:10<02:50, 4it/s]
+
1.549719e-03 29.7%┣█████████                     ┫ 297/1.0k [01:11<02:48, 4it/s]
+
1.535366e-03 30.0%┣█████████                     ┫ 300/1.0k [01:11<02:46, 4it/s]
+
1.526787e-03 30.3%┣█████████                     ┫ 303/1.0k [01:11<02:43, 4it/s]
+
1.517236e-03 30.6%┣█████████▏                    ┫ 306/1.0k [01:11<02:41, 4it/s]
+
1.507883e-03 30.9%┣█████████▎                    ┫ 309/1.0k [01:11<02:39, 4it/s]
+
1.493978e-03 31.2%┣█████████▍                    ┫ 312/1.0k [01:11<02:37, 4it/s]
+
1.476256e-03 31.5%┣█████████▍                    ┫ 315/1.0k [01:11<02:35, 4it/s]
+
1.463490e-03 31.8%┣█████████▌                    ┫ 318/1.0k [01:11<02:33, 4it/s]
+
1.449678e-03 32.1%┣█████████▋                    ┫ 321/1.0k [01:11<02:31, 5it/s]
+
1.442567e-03 32.4%┣█████████▊                    ┫ 324/1.0k [01:11<02:29, 5it/s]
+
1.433347e-03 32.7%┣█████████▉                    ┫ 327/1.0k [01:11<02:27, 5it/s]
+
1.419636e-03 33.0%┣██████████                    ┫ 330/1.0k [01:11<02:25, 5it/s]
+
1.406990e-03 33.3%┣██████████                    ┫ 333/1.0k [01:11<02:23, 5it/s]
+
1.396558e-03 33.6%┣██████████                    ┫ 336/1.0k [01:11<02:22, 5it/s]
+
1.383758e-03 33.9%┣██████████▏                   ┫ 339/1.0k [01:11<02:20, 5it/s]
+
1.366264e-03 34.2%┣██████████▎                   ┫ 342/1.0k [01:11<02:18, 5it/s]
+
1.354687e-03 34.5%┣██████████▍                   ┫ 345/1.0k [01:12<02:16, 5it/s]
+
1.346302e-03 34.8%┣██████████▍                   ┫ 348/1.0k [01:12<02:15, 5it/s]
+
1.339696e-03 35.1%┣██████████▌                   ┫ 351/1.0k [01:12<02:13, 5it/s]
+
1.331609e-03 35.4%┣██████████▋                   ┫ 354/1.0k [01:12<02:11, 5it/s]
+
1.320173e-03 35.7%┣██████████▊                   ┫ 357/1.0k [01:12<02:10, 5it/s]
+
1.312841e-03 36.0%┣██████████▉                   ┫ 360/1.0k [01:12<02:08, 5it/s]
+
1.302683e-03 36.3%┣██████████▉                   ┫ 363/1.0k [01:12<02:07, 5it/s]
+
1.289342e-03 36.6%┣███████████                   ┫ 366/1.0k [01:12<02:05, 5it/s]
+
1.273511e-03 36.9%┣███████████                   ┫ 369/1.0k [01:12<02:04, 5it/s]
+
1.260279e-03 37.2%┣███████████▏                  ┫ 372/1.0k [01:12<02:02, 5it/s]
+
1.252536e-03 37.5%┣███████████▎                  ┫ 375/1.0k [01:12<02:01, 5it/s]
+
1.244727e-03 37.8%┣███████████▎                  ┫ 378/1.0k [01:12<01:59, 5it/s]
+
1.237342e-03 38.1%┣███████████▍                  ┫ 381/1.0k [01:12<01:58, 5it/s]
+
1.230560e-03 38.4%┣███████████▌                  ┫ 384/1.0k [01:12<01:57, 5it/s]
+
1.217961e-03 38.7%┣███████████▋                  ┫ 387/1.0k [01:12<01:55, 5it/s]
+
1.203909e-03 39.0%┣███████████▊                  ┫ 390/1.0k [01:12<01:54, 5it/s]
+
1.192448e-03 39.3%┣███████████▉                  ┫ 393/1.0k [01:13<01:53, 5it/s]
+
1.185300e-03 39.6%┣███████████▉                  ┫ 396/1.0k [01:13<01:51, 5it/s]
+
1.176910e-03 39.9%┣████████████                  ┫ 399/1.0k [01:13<01:50, 5it/s]
+
1.167255e-03 40.2%┣████████████                  ┫ 402/1.0k [01:13<01:49, 6it/s]
+
1.160411e-03 40.5%┣████████████▏                 ┫ 405/1.0k [01:13<01:47, 6it/s]
+
1.154572e-03 40.8%┣████████████▎                 ┫ 408/1.0k [01:13<01:46, 6it/s]
+
1.145214e-03 41.1%┣████████████▎                 ┫ 411/1.0k [01:13<01:45, 6it/s]
+
1.131628e-03 41.4%┣████████████▍                 ┫ 414/1.0k [01:13<01:44, 6it/s]
+
1.122520e-03 41.7%┣████████████▌                 ┫ 417/1.0k [01:13<01:43, 6it/s]
+
1.113844e-03 42.0%┣████████████▋                 ┫ 420/1.0k [01:13<01:41, 6it/s]
+
1.107769e-03 42.2%┣████████████▋                 ┫ 422/1.0k [01:13<01:41, 6it/s]
+
1.099121e-03 42.4%┣████████████▊                 ┫ 424/1.0k [01:13<01:40, 6it/s]
+
1.091447e-03 42.7%┣████████████▉                 ┫ 427/1.0k [01:13<01:39, 6it/s]
+
1.086385e-03 42.9%┣████████████▉                 ┫ 429/1.0k [01:13<01:38, 6it/s]
+
1.079009e-03 43.2%┣█████████████                 ┫ 432/1.0k [01:13<01:37, 6it/s]
+
1.073067e-03 43.5%┣█████████████                 ┫ 435/1.0k [01:13<01:36, 6it/s]
+
1.065942e-03 43.8%┣█████████████▏                ┫ 438/1.0k [01:14<01:35, 6it/s]
+
1.058397e-03 44.1%┣█████████████▏                ┫ 441/1.0k [01:14<01:34, 6it/s]
+
1.053063e-03 44.4%┣█████████████▎                ┫ 444/1.0k [01:14<01:33, 6it/s]
+
1.049794e-03 44.6%┣█████████████▍                ┫ 446/1.0k [01:14<01:32, 6it/s]
+
1.039955e-03 44.9%┣█████████████▌                ┫ 449/1.0k [01:14<01:31, 6it/s]
+
1.034829e-03 45.2%┣█████████████▌                ┫ 452/1.0k [01:14<01:30, 6it/s]
+
1.028986e-03 45.5%┣█████████████▋                ┫ 455/1.0k [01:14<01:29, 6it/s]
+
1.023605e-03 45.8%┣█████████████▊                ┫ 458/1.0k [01:14<01:28, 6it/s]
+
1.017620e-03 46.1%┣█████████████▉                ┫ 461/1.0k [01:14<01:27, 6it/s]
+
1.011451e-03 46.4%┣██████████████                ┫ 464/1.0k [01:14<01:26, 6it/s]
+
1.007214e-03 46.7%┣██████████████                ┫ 467/1.0k [01:14<01:25, 6it/s]
+
1.001822e-03 47.0%┣██████████████                ┫ 470/1.0k [01:14<01:24, 6it/s]
+
9.965027e-04 47.3%┣██████████████▏               ┫ 473/1.0k [01:14<01:23, 6it/s]
+
9.902903e-04 47.6%┣██████████████▎               ┫ 476/1.0k [01:14<01:22, 6it/s]
+
9.848194e-04 47.8%┣██████████████▎               ┫ 478/1.0k [01:14<01:22, 6it/s]
+
9.786154e-04 48.1%┣██████████████▍               ┫ 481/1.0k [01:14<01:21, 6it/s]
+
9.737109e-04 48.4%┣██████████████▌               ┫ 484/1.0k [01:14<01:20, 6it/s]
+
9.671344e-04 48.7%┣██████████████▋               ┫ 487/1.0k [01:15<01:19, 7it/s]
+
9.589523e-04 49.0%┣██████████████▊               ┫ 490/1.0k [01:15<01:18, 7it/s]
+
9.544257e-04 49.3%┣██████████████▊               ┫ 493/1.0k [01:15<01:17, 7it/s]
+
9.500232e-04 49.6%┣██████████████▉               ┫ 496/1.0k [01:15<01:16, 7it/s]
+
9.449498e-04 49.9%┣███████████████               ┫ 499/1.0k [01:15<01:15, 7it/s]
+
9.404042e-04 50.1%┣███████████████               ┫ 502/1.0k [01:15<01:15, 7it/s]
+
9.376128e-04 50.4%┣███████████████▏              ┫ 505/1.0k [01:15<01:14, 7it/s]
+
9.340614e-04 50.7%┣███████████████▎              ┫ 508/1.0k [01:15<01:13, 7it/s]
+
9.298133e-04 51.0%┣███████████████▎              ┫ 511/1.0k [01:15<01:12, 7it/s]
+
9.272632e-04 51.2%┣███████████████▍              ┫ 513/1.0k [01:15<01:12, 7it/s]
+
9.203513e-04 51.5%┣███████████████▌              ┫ 516/1.0k [01:15<01:11, 7it/s]
+
9.122885e-04 51.8%┣███████████████▌              ┫ 519/1.0k [01:15<01:10, 7it/s]
+
9.051892e-04 52.1%┣███████████████▋              ┫ 522/1.0k [01:15<01:09, 7it/s]
+
9.009420e-04 52.4%┣███████████████▊              ┫ 525/1.0k [01:15<01:08, 7it/s]
+
8.971950e-04 52.7%┣███████████████▉              ┫ 528/1.0k [01:15<01:08, 7it/s]
+
8.940695e-04 52.9%┣███████████████▉              ┫ 530/1.0k [01:15<01:07, 7it/s]
+
8.872502e-04 53.2%┣████████████████              ┫ 533/1.0k [01:16<01:06, 7it/s]
+
8.820093e-04 53.5%┣████████████████              ┫ 536/1.0k [01:16<01:06, 7it/s]
+
8.740395e-04 53.8%┣████████████████▏             ┫ 539/1.0k [01:16<01:05, 7it/s]
+
8.646484e-04 54.1%┣████████████████▎             ┫ 542/1.0k [01:16<01:04, 7it/s]
+
8.584920e-04 54.4%┣████████████████▍             ┫ 545/1.0k [01:16<01:04, 7it/s]
+
8.536807e-04 54.7%┣████████████████▍             ┫ 548/1.0k [01:16<01:03, 7it/s]
+
8.508792e-04 55.0%┣████████████████▌             ┫ 551/1.0k [01:16<01:02, 7it/s]
+
8.441122e-04 55.3%┣████████████████▋             ┫ 554/1.0k [01:16<01:01, 7it/s]
+
8.376469e-04 55.6%┣████████████████▊             ┫ 557/1.0k [01:16<01:01, 7it/s]
+
8.314253e-04 55.9%┣████████████████▉             ┫ 560/1.0k [01:16<01:00, 7it/s]
+
8.223716e-04 56.2%┣████████████████▉             ┫ 563/1.0k [01:16<00:59, 7it/s]
+
8.153172e-04 56.5%┣█████████████████             ┫ 566/1.0k [01:16<00:59, 7it/s]
+
8.085778e-04 56.8%┣█████████████████             ┫ 569/1.0k [01:16<00:58, 7it/s]
+
8.043744e-04 57.1%┣█████████████████▏            ┫ 572/1.0k [01:16<00:57, 7it/s]
+
8.002600e-04 57.4%┣█████████████████▎            ┫ 575/1.0k [01:16<00:57, 8it/s]
+
7.951163e-04 57.7%┣█████████████████▎            ┫ 578/1.0k [01:17<00:56, 8it/s]
+
7.892636e-04 58.0%┣█████████████████▍            ┫ 581/1.0k [01:17<00:55, 8it/s]
+
7.845497e-04 58.3%┣█████████████████▌            ┫ 584/1.0k [01:17<00:55, 8it/s]
+
7.787946e-04 58.6%┣█████████████████▋            ┫ 587/1.0k [01:17<00:54, 8it/s]
+
7.739397e-04 58.9%┣█████████████████▊            ┫ 590/1.0k [01:17<00:54, 8it/s]
+
7.684312e-04 59.2%┣█████████████████▊            ┫ 593/1.0k [01:17<00:53, 8it/s]
+
7.651524e-04 59.5%┣█████████████████▉            ┫ 596/1.0k [01:17<00:52, 8it/s]
+
7.616343e-04 59.8%┣██████████████████            ┫ 599/1.0k [01:17<00:52, 8it/s]
+
7.586593e-04 60.1%┣██████████████████            ┫ 602/1.0k [01:17<00:51, 8it/s]
+
7.540907e-04 60.4%┣██████████████████▏           ┫ 605/1.0k [01:17<00:51, 8it/s]
+
7.499634e-04 60.7%┣██████████████████▏           ┫ 608/1.0k [01:17<00:50, 8it/s]
+
7.459653e-04 61.0%┣██████████████████▎           ┫ 611/1.0k [01:17<00:49, 8it/s]
+
7.418074e-04 61.3%┣██████████████████▍           ┫ 614/1.0k [01:17<00:49, 8it/s]
+
7.385251e-04 61.5%┣██████████████████▌           ┫ 616/1.0k [01:17<00:48, 8it/s]
+
7.319144e-04 61.8%┣██████████████████▌           ┫ 619/1.0k [01:17<00:48, 8it/s]
+
7.276659e-04 62.0%┣██████████████████▋           ┫ 621/1.0k [01:17<00:47, 8it/s]
+
7.196389e-04 62.3%┣██████████████████▊           ┫ 624/1.0k [01:17<00:47, 8it/s]
+
7.146386e-04 62.6%┣██████████████████▉           ┫ 627/1.0k [01:18<00:46, 8it/s]
+
7.101293e-04 62.9%┣██████████████████▉           ┫ 630/1.0k [01:18<00:46, 8it/s]
+
7.045738e-04 63.2%┣███████████████████           ┫ 633/1.0k [01:18<00:45, 8it/s]
+
6.992432e-04 63.5%┣███████████████████           ┫ 636/1.0k [01:18<00:45, 8it/s]
+
6.951509e-04 63.8%┣███████████████████▏          ┫ 639/1.0k [01:18<00:44, 8it/s]
+
6.870928e-04 64.1%┣███████████████████▎          ┫ 642/1.0k [01:18<00:44, 8it/s]
+
6.804472e-04 64.4%┣███████████████████▎          ┫ 645/1.0k [01:18<00:43, 8it/s]
+
6.777305e-04 64.7%┣███████████████████▍          ┫ 648/1.0k [01:18<00:43, 8it/s]
+
6.729425e-04 65.0%┣███████████████████▌          ┫ 651/1.0k [01:18<00:42, 8it/s]
+
6.678701e-04 65.3%┣███████████████████▋          ┫ 654/1.0k [01:18<00:42, 8it/s]
+
6.624558e-04 65.6%┣███████████████████▊          ┫ 657/1.0k [01:18<00:41, 8it/s]
+
6.583430e-04 65.9%┣███████████████████▉          ┫ 660/1.0k [01:18<00:40, 8it/s]
+
6.541348e-04 66.2%┣███████████████████▉          ┫ 663/1.0k [01:18<00:40, 8it/s]
+
6.504063e-04 66.5%┣████████████████████          ┫ 666/1.0k [01:18<00:39, 8it/s]
+
6.449909e-04 66.8%┣████████████████████          ┫ 669/1.0k [01:18<00:39, 9it/s]
+
6.379931e-04 67.1%┣████████████████████▏         ┫ 672/1.0k [01:19<00:38, 9it/s]
+
6.331261e-04 67.4%┣████████████████████▎         ┫ 675/1.0k [01:19<00:38, 9it/s]
+
6.296909e-04 67.7%┣████████████████████▎         ┫ 678/1.0k [01:19<00:38, 9it/s]
+
6.237315e-04 68.0%┣████████████████████▍         ┫ 681/1.0k [01:19<00:37, 9it/s]
+
6.205524e-04 68.2%┣████████████████████▌         ┫ 683/1.0k [01:19<00:37, 9it/s]
+
6.153457e-04 68.5%┣████████████████████▋         ┫ 686/1.0k [01:19<00:36, 9it/s]
+
6.112391e-04 68.8%┣████████████████████▋         ┫ 689/1.0k [01:19<00:36, 9it/s]
+
6.068195e-04 69.1%┣████████████████████▊         ┫ 692/1.0k [01:19<00:35, 9it/s]
+
6.037010e-04 69.4%┣████████████████████▉         ┫ 695/1.0k [01:19<00:35, 9it/s]
+
5.999574e-04 69.7%┣█████████████████████         ┫ 698/1.0k [01:19<00:34, 9it/s]
+
5.957017e-04 70.0%┣█████████████████████         ┫ 701/1.0k [01:19<00:34, 9it/s]
+
5.922861e-04 70.3%┣█████████████████████         ┫ 704/1.0k [01:19<00:33, 9it/s]
+
5.886770e-04 70.6%┣█████████████████████▏        ┫ 707/1.0k [01:19<00:33, 9it/s]
+
5.823499e-04 70.9%┣█████████████████████▎        ┫ 710/1.0k [01:19<00:33, 9it/s]
+
5.745789e-04 71.2%┣█████████████████████▍        ┫ 713/1.0k [01:19<00:32, 9it/s]
+
5.703081e-04 71.5%┣█████████████████████▌        ┫ 716/1.0k [01:19<00:32, 9it/s]
+
5.672129e-04 71.8%┣█████████████████████▌        ┫ 719/1.0k [01:20<00:31, 9it/s]
+
5.637485e-04 72.1%┣█████████████████████▋        ┫ 722/1.0k [01:20<00:31, 9it/s]
+
5.611528e-04 72.4%┣█████████████████████▊        ┫ 725/1.0k [01:20<00:30, 9it/s]
+
5.583630e-04 72.6%┣█████████████████████▉        ┫ 727/1.0k [01:20<00:30, 9it/s]
+
5.545750e-04 72.9%┣█████████████████████▉        ┫ 730/1.0k [01:20<00:30, 9it/s]
+
5.495886e-04 73.2%┣██████████████████████        ┫ 733/1.0k [01:20<00:29, 9it/s]
+
5.446079e-04 73.5%┣██████████████████████        ┫ 736/1.0k [01:20<00:29, 9it/s]
+
5.413033e-04 73.8%┣██████████████████████▏       ┫ 739/1.0k [01:20<00:28, 9it/s]
+
5.368537e-04 74.1%┣██████████████████████▎       ┫ 742/1.0k [01:20<00:28, 9it/s]
+
5.326757e-04 74.4%┣██████████████████████▎       ┫ 745/1.0k [01:20<00:28, 9it/s]
+
5.290243e-04 74.7%┣██████████████████████▍       ┫ 748/1.0k [01:20<00:27, 9it/s]
+
5.256739e-04 75.0%┣██████████████████████▌       ┫ 751/1.0k [01:20<00:27, 9it/s]
+
5.238793e-04 75.2%┣██████████████████████▋       ┫ 753/1.0k [01:20<00:26, 9it/s]
+
5.203084e-04 75.5%┣██████████████████████▋       ┫ 756/1.0k [01:20<00:26, 9it/s]
+
5.180699e-04 75.7%┣██████████████████████▊       ┫ 758/1.0k [01:20<00:26, 9it/s]
+
5.121805e-04 76.0%┣██████████████████████▉       ┫ 761/1.0k [01:20<00:25, 9it/s]
+
5.044945e-04 76.3%┣███████████████████████       ┫ 764/1.0k [01:20<00:25, 9it/s]
+
4.981500e-04 76.6%┣██████████████████████▏      ┫ 767/1.0k [01:21<00:25, 10it/s]
+
4.932784e-04 76.9%┣██████████████████████▎      ┫ 770/1.0k [01:21<00:24, 10it/s]
+
4.886349e-04 77.2%┣██████████████████████▍      ┫ 773/1.0k [01:21<00:24, 10it/s]
+
4.846743e-04 77.5%┣██████████████████████▌      ┫ 776/1.0k [01:21<00:23, 10it/s]
+
4.799662e-04 77.8%┣██████████████████████▋      ┫ 779/1.0k [01:21<00:23, 10it/s]
+
4.768647e-04 78.1%┣██████████████████████▋      ┫ 782/1.0k [01:21<00:23, 10it/s]
+
4.732984e-04 78.4%┣██████████████████████▊      ┫ 785/1.0k [01:21<00:22, 10it/s]
+
4.702272e-04 78.6%┣██████████████████████▉      ┫ 787/1.0k [01:21<00:22, 10it/s]
+
4.656693e-04 78.9%┣██████████████████████▉      ┫ 790/1.0k [01:21<00:22, 10it/s]
+
4.628635e-04 79.2%┣███████████████████████      ┫ 793/1.0k [01:21<00:21, 10it/s]
+
4.609238e-04 79.5%┣███████████████████████      ┫ 796/1.0k [01:21<00:21, 10it/s]
+
4.594420e-04 79.7%┣███████████████████████▏     ┫ 798/1.0k [01:21<00:21, 10it/s]
+
4.563894e-04 80.0%┣███████████████████████▏     ┫ 801/1.0k [01:21<00:20, 10it/s]
+
4.526299e-04 80.2%┣███████████████████████▎     ┫ 803/1.0k [01:21<00:20, 10it/s]
+
4.479421e-04 80.5%┣███████████████████████▍     ┫ 806/1.0k [01:21<00:20, 10it/s]
+
4.451937e-04 80.7%┣███████████████████████▍     ┫ 808/1.0k [01:21<00:19, 10it/s]
+
4.426221e-04 81.0%┣███████████████████████▌     ┫ 811/1.0k [01:22<00:19, 10it/s]
+
4.390554e-04 81.2%┣███████████████████████▌     ┫ 813/1.0k [01:22<00:19, 10it/s]
+
4.341595e-04 81.5%┣███████████████████████▋     ┫ 816/1.0k [01:22<00:19, 10it/s]
+
4.294785e-04 81.8%┣███████████████████████▊     ┫ 819/1.0k [01:22<00:18, 10it/s]
+
4.238078e-04 82.1%┣███████████████████████▉     ┫ 822/1.0k [01:22<00:18, 10it/s]
+
4.190331e-04 82.4%┣████████████████████████     ┫ 825/1.0k [01:22<00:17, 10it/s]
+
4.142807e-04 82.7%┣████████████████████████     ┫ 828/1.0k [01:22<00:17, 10it/s]
+
4.075273e-04 83.0%┣████████████████████████     ┫ 831/1.0k [01:22<00:17, 10it/s]
+
4.020772e-04 83.3%┣████████████████████████▏    ┫ 834/1.0k [01:22<00:16, 10it/s]
+
3.977089e-04 83.6%┣████████████████████████▎    ┫ 837/1.0k [01:22<00:16, 10it/s]
+
3.939446e-04 83.9%┣████████████████████████▍    ┫ 840/1.0k [01:22<00:16, 10it/s]
+
3.917904e-04 84.2%┣████████████████████████▍    ┫ 843/1.0k [01:22<00:15, 10it/s]
+
3.888236e-04 84.5%┣████████████████████████▌    ┫ 846/1.0k [01:22<00:15, 10it/s]
+
3.868438e-04 84.8%┣████████████████████████▋    ┫ 849/1.0k [01:22<00:15, 10it/s]
+
3.847824e-04 85.1%┣████████████████████████▊    ┫ 852/1.0k [01:22<00:14, 10it/s]
+
3.822188e-04 85.4%┣████████████████████████▊    ┫ 855/1.0k [01:22<00:14, 10it/s]
+
3.812668e-04 85.6%┣████████████████████████▉    ┫ 857/1.0k [01:23<00:14, 10it/s]
+
3.793834e-04 85.9%┣█████████████████████████    ┫ 860/1.0k [01:23<00:14, 10it/s]
+
3.781289e-04 86.1%┣█████████████████████████    ┫ 862/1.0k [01:23<00:13, 10it/s]
+
3.758885e-04 86.4%┣█████████████████████████    ┫ 865/1.0k [01:23<00:13, 10it/s]
+
3.745212e-04 86.7%┣█████████████████████████▏   ┫ 868/1.0k [01:23<00:13, 10it/s]
+
3.736464e-04 86.9%┣█████████████████████████▏   ┫ 870/1.0k [01:23<00:12, 10it/s]
+
3.718598e-04 87.1%┣█████████████████████████▎   ┫ 872/1.0k [01:23<00:12, 11it/s]
+
3.688587e-04 87.4%┣█████████████████████████▍   ┫ 875/1.0k [01:23<00:12, 11it/s]
+
3.670958e-04 87.6%┣█████████████████████████▍   ┫ 877/1.0k [01:23<00:12, 11it/s]
+
3.639113e-04 87.9%┣█████████████████████████▌   ┫ 880/1.0k [01:23<00:11, 11it/s]
+
3.621203e-04 88.1%┣█████████████████████████▌   ┫ 882/1.0k [01:23<00:11, 11it/s]
+
3.599480e-04 88.4%┣█████████████████████████▋   ┫ 885/1.0k [01:23<00:11, 11it/s]
+
3.585292e-04 88.7%┣█████████████████████████▊   ┫ 888/1.0k [01:23<00:11, 11it/s]
+
3.567810e-04 88.9%┣█████████████████████████▉   ┫ 890/1.0k [01:23<00:10, 11it/s]
+
3.543316e-04 89.2%┣█████████████████████████▉   ┫ 893/1.0k [01:23<00:10, 11it/s]
+
3.521078e-04 89.5%┣██████████████████████████   ┫ 896/1.0k [01:23<00:10, 11it/s]
+
3.497107e-04 89.8%┣██████████████████████████   ┫ 899/1.0k [01:24<00:09, 11it/s]
+
3.479538e-04 90.1%┣██████████████████████████▏  ┫ 902/1.0k [01:24<00:09, 11it/s]
+
3.466361e-04 90.4%┣██████████████████████████▏  ┫ 905/1.0k [01:24<00:09, 11it/s]
+
3.455751e-04 90.6%┣██████████████████████████▎  ┫ 907/1.0k [01:24<00:09, 11it/s]
+
3.434474e-04 90.9%┣██████████████████████████▍  ┫ 910/1.0k [01:24<00:08, 11it/s]
+
3.418423e-04 91.2%┣██████████████████████████▌  ┫ 913/1.0k [01:24<00:08, 11it/s]
+
3.398776e-04 91.5%┣██████████████████████████▌  ┫ 916/1.0k [01:24<00:08, 11it/s]
+
3.386125e-04 91.7%┣██████████████████████████▋  ┫ 918/1.0k [01:24<00:08, 11it/s]
+
3.370532e-04 92.0%┣██████████████████████████▊  ┫ 921/1.0k [01:24<00:07, 11it/s]
+
3.364013e-04 92.2%┣██████████████████████████▊  ┫ 923/1.0k [01:24<00:07, 11it/s]
+
3.358894e-04 92.4%┣██████████████████████████▉  ┫ 925/1.0k [01:24<00:07, 11it/s]
+
3.347820e-04 92.6%┣██████████████████████████▉  ┫ 927/1.0k [01:24<00:07, 11it/s]
+
3.328402e-04 92.9%┣███████████████████████████  ┫ 930/1.0k [01:24<00:06, 11it/s]
+
3.312832e-04 93.1%┣███████████████████████████  ┫ 932/1.0k [01:24<00:06, 11it/s]
+
3.295682e-04 93.4%┣███████████████████████████  ┫ 935/1.0k [01:24<00:06, 11it/s]
+
3.282629e-04 93.7%┣███████████████████████████▏ ┫ 938/1.0k [01:24<00:06, 11it/s]
+
3.272183e-04 93.9%┣███████████████████████████▎ ┫ 940/1.0k [01:24<00:05, 11it/s]
+
3.265733e-04 94.1%┣███████████████████████████▎ ┫ 942/1.0k [01:25<00:05, 11it/s]
+
3.258205e-04 94.3%┣███████████████████████████▍ ┫ 944/1.0k [01:25<00:05, 11it/s]
+
3.251256e-04 94.5%┣███████████████████████████▍ ┫ 946/1.0k [01:25<00:05, 11it/s]
+
3.242898e-04 94.7%┣███████████████████████████▌ ┫ 948/1.0k [01:25<00:05, 11it/s]
+
3.230671e-04 94.9%┣███████████████████████████▌ ┫ 950/1.0k [01:25<00:05, 11it/s]
+
3.212167e-04 95.1%┣███████████████████████████▋ ┫ 952/1.0k [01:25<00:04, 11it/s]
+
3.190204e-04 95.4%┣███████████████████████████▊ ┫ 955/1.0k [01:25<00:04, 11it/s]
+
3.171997e-04 95.7%┣███████████████████████████▊ ┫ 958/1.0k [01:25<00:04, 11it/s]
+
3.149587e-04 96.0%┣███████████████████████████▉ ┫ 961/1.0k [01:25<00:04, 11it/s]
+
3.127340e-04 96.3%┣████████████████████████████ ┫ 964/1.0k [01:25<00:03, 11it/s]
+
3.103789e-04 96.6%┣████████████████████████████ ┫ 967/1.0k [01:25<00:03, 11it/s]
+
3.085005e-04 96.9%┣████████████████████████████ ┫ 970/1.0k [01:25<00:03, 11it/s]
+
3.071459e-04 97.2%┣████████████████████████████▏┫ 973/1.0k [01:25<00:02, 11it/s]
+
3.058040e-04 97.5%┣████████████████████████████▎┫ 976/1.0k [01:25<00:02, 11it/s]
+
3.043762e-04 97.8%┣████████████████████████████▍┫ 979/1.0k [01:25<00:02, 11it/s]
+
3.033865e-04 98.1%┣████████████████████████████▌┫ 982/1.0k [01:25<00:02, 11it/s]
+
3.022283e-04 98.3%┣████████████████████████████▌┫ 984/1.0k [01:26<00:01, 11it/s]
+
3.004165e-04 98.6%┣████████████████████████████▋┫ 987/1.0k [01:26<00:01, 12it/s]
+
2.994324e-04 98.9%┣████████████████████████████▊┫ 990/1.0k [01:26<00:01, 12it/s]
+
2.988700e-04 99.1%┣████████████████████████████▊┫ 992/1.0k [01:26<00:01, 12it/s]
+
2.980818e-04 99.3%┣████████████████████████████▉┫ 994/1.0k [01:26<00:01, 12it/s]
+
2.960191e-04 99.6%┣████████████████████████████▉┫ 997/1.0k [01:26<00:00, 12it/s]
+
2.949204e-04 99.9%┣████████████████████████████┫ 1.0k/1.0k [01:26<00:00, 12it/s]
+
2.946265e-04 100.0%┣███████████████████████████┫ 1.0k/1.0k [01:26<00:00, 12it/s]

diff --git a/dev/tutorials/Lshape.png b/dev/tutorials/Lshape.png index 5cbbb383..9a64c8d7 100644 Binary files a/dev/tutorials/Lshape.png and b/dev/tutorials/Lshape.png differ diff --git "a/dev/tutorials/Schr\303\266dingerEquation/index.html" "b/dev/tutorials/Schr\303\266dingerEquation/index.html" index b4f178d5..c49ab381 100644 --- "a/dev/tutorials/Schr\303\266dingerEquation/index.html" +++ "b/dev/tutorials/Schr\303\266dingerEquation/index.html" @@ -29,7 +29,7 @@ strategy = NonAdaptiveTraining(1,(10,10,1,1)) prob = Sophon.discretize(pde_system, pinn, sampler, strategy)
OptimizationProblem. In-place: true
-u0: ComponentVector{Float64}(u = (layer_1 = (weight = [0.22964340448379517 -0.20562291145324707; -0.12745380401611328 0.18547427654266357; … ; -0.27596765756607056 -0.05317729711532593; 0.3796898126602173 0.01035386323928833], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = [-0.33720505237579346 0.11578547209501266 … 0.3368074893951416 -0.06941491365432739; 0.5056222081184387 -0.29894885420799255 … 0.2529597282409668 -0.39428040385246277; … ; 0.11343777924776077 -0.033319901674985886 … 0.034632012248039246 -0.4265323877334595; 0.4876216650009155 -0.40572279691696167 … 0.3601861000061035 0.3207268714904785], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_3 = (weight = [0.6000635623931885 -0.3275875449180603 … -0.3346778452396393 0.23089481890201569; 0.20031039416790009 -0.5470900535583496 … -0.252166211605072 -0.6034035682678223; … ; 0.17678971588611603 -0.2279919534921646 … 0.06829844415187836 -0.00546437781304121; 0.19115079939365387 -0.216787189245224 … 0.13993680477142334 -0.42471635341644287], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_4 = (weight = [-0.12319108098745346 0.17530640959739685 … 0.036950431764125824 -0.40246835350990295; 0.19862094521522522 -0.4034794867038727 … 0.16110731661319733 0.04489726573228836; … ; -0.38126340508461 0.4118790626525879 … -0.43695729970932007 -0.1822448968887329; -0.3679440915584564 -0.13968291878700256 … -0.29416316747665405 0.5916492938995361], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_5 = (weight = [-0.2846938371658325 0.541811466217041 … -0.5992764830589294 0.11028298735618591], bias = [0.0;;])), v = (layer_1 = (weight = [0.07568603754043579 0.19658386707305908; -0.3957476019859314 0.41824400424957275; … ; 0.1532365083694458 0.2630653977394104; -0.05837351083755493 0.08897656202316284], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = [0.03662871941924095 -0.13439542055130005 … -0.5520708560943604 -0.5318562984466553; -0.6060840487480164 0.5736543536186218 … 0.4112898111343384 0.15252472460269928; … ; -0.48717358708381653 0.5672550201416016 … 0.4308852553367615 -0.4498139023780823; -0.5503025650978088 -0.5749012231826782 … 0.034845028072595596 0.22680577635765076], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_3 = (weight = [0.14709646999835968 0.6111188530921936 … 0.3952961266040802 -0.21189016103744507; 0.28199800848960876 -0.46874988079071045 … -0.43265411257743835 0.04265972599387169; … ; 0.4247659146785736 -0.25425586104393005 … 0.33384332060813904 -0.5921738743782043; -0.046716656535863876 -0.3907991647720337 … 0.3741101622581482 -0.5967997312545776], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_4 = (weight = [-0.12457816302776337 -0.15933988988399506 … 0.5298596024513245 -0.22143001854419708; -0.5418350100517273 0.20352628827095032 … 0.4676305651664734 -0.04667942598462105; … ; -0.2735021412372589 0.36760032176971436 … 0.4514291286468506 -0.19009360671043396; -0.429028183221817 -0.5858306884765625 … 0.5694872736930847 0.057449404150247574], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_5 = (weight = [-0.05330348759889603 -0.31719154119491577 … 0.16047300398349762 -0.5099985599517822], bias = [0.0;;])))

Now we train the neural nets and resample data while training.

function train(pde_system, prob, sampler, strategy, resample_period = 500, n=10)
+u0: ComponentVector{Float64}(u = (layer_1 = (weight = [-0.27530884742736816 -0.3955625295639038; -0.2521119713783264 -0.3372645378112793; … ; 0.10971057415008545 -0.0411953330039978; -0.08085286617279053 -0.18571776151657104], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = [-0.12364397943019867 0.28989797830581665 … 0.39499974250793457 0.3568327724933624; -0.3192588984966278 -0.05345233902335167 … -0.12438894808292389 -0.32950088381767273; … ; -0.010141591541469097 0.013717738911509514 … 0.04926561564207077 0.2980627119541168; 0.5258228182792664 -0.0329856313765049 … 0.538238525390625 -0.5532993674278259], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_3 = (weight = [0.40529969334602356 0.4032951593399048 … 0.5665891766548157 -0.050196077674627304; 0.5925670862197876 0.07352308928966522 … -0.13622677326202393 0.08223357796669006; … ; -0.38983699679374695 -0.0024838412646204233 … 0.38777095079421997 -0.22445522248744965; 0.14507925510406494 -0.2948514223098755 … -0.14777545630931854 -0.03827356547117233], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_4 = (weight = [-0.4327128827571869 -0.4336124062538147 … 0.19460627436637878 0.6042764186859131; 0.01685340143740177 0.03562912344932556 … 0.02273789793252945 -0.18730835616588593; … ; -0.05188625678420067 -0.28453221917152405 … 0.5467583537101746 -0.12425221502780914; 0.2893866002559662 -0.4119569659233093 … -0.14119620621204376 0.47424745559692383], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_5 = (weight = [0.06310489773750305 0.2953197956085205 … -0.4668687880039215 -0.5034142732620239], bias = [0.0;;])), v = (layer_1 = (weight = [-0.17008519172668457 -0.21689462661743164; 0.21513229608535767 0.07205170392990112; … ; 0.11486649513244629 -0.3071137070655823; -0.008585035800933838 0.1495111584663391], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = [0.538830041885376 -0.16022393107414246 … 0.5460960865020752 0.47112807631492615; -0.15467983484268188 0.28702059388160706 … 0.11163802444934845 0.2310977578163147; … ; 0.07020602375268936 0.4151960611343384 … 0.15957976877689362 0.09812767803668976; -1.3578089237853419e-5 -0.4599781930446625 … -0.5106055736541748 -0.580622136592865], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_3 = (weight = [-0.12586654722690582 0.47114771604537964 … 0.47374260425567627 -0.198220893740654; -0.3175696134567261 0.6013116836547852 … 0.24384503066539764 0.005199240054935217; … ; 0.46794846653938293 -0.5333725810050964 … -0.32454714179039 0.2654152810573578; 0.26902931928634644 0.15978242456912994 … -0.1684001386165619 0.2299613654613495], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_4 = (weight = [0.3831677734851837 0.24969135224819183 … 0.5049445033073425 -0.2507484555244446; 0.24310553073883057 0.2395644336938858 … -0.32465240359306335 0.36577218770980835; … ; 0.05792872607707977 -0.048310987651348114 … 0.0009830244816839695 -0.35817262530326843; 0.21083064377307892 -0.14471228420734406 … -0.3707740306854248 0.21311920881271362], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_5 = (weight = [-0.3970699608325958 0.5198151469230652 … 0.23737923800945282 0.43956324458122253], bias = [0.0;;])))

Now we train the neural nets and resample data while training.

function train(pde_system, prob, sampler, strategy, resample_period = 500, n=10)
      bfgs = BFGS()
      res = Optimization.solve(prob, bfgs; maxiters=2000)
 
@@ -42,7 +42,7 @@
 end
 
 res = train(pde_system, prob, sampler, strategy)
retcode: Failure
-u: ComponentVector{Float64}(u = (layer_1 = (weight = [0.21235830510366635 0.2680283891850633; -0.45441010437339874 0.5272408868657771; … ; -0.012641087637732642 -0.11874687027166592; 1.270684429912298 -0.4699585932243045], bias = [0.4530232334529752; 0.23671951305872013; … ; -0.11308253709979735; 0.08440662091126779;;]), layer_2 = (weight = [-0.42857487812090944 0.2458059194711604 … 0.2701789263122683 -0.28282335990278856; 0.49385938634697363 -0.07082276024708827 … 0.2710352741985176 -0.3328344288374426; … ; -0.013170804484413342 0.11106983000262939 … 0.14634856288132697 -0.4513430688328091; 0.837799435886789 -0.4194466497045731 … 0.09069386898459723 0.7721847747449293], bias = [-0.06498843167447485; -0.3411647353670892; … ; 0.16752147282395385; -0.04802249607649675;;]), layer_3 = (weight = [0.3413608918164652 -0.3998676556441467 … -0.19123499464068425 0.5030296059914127; 0.22191415553801175 -0.5726112565845367 … 0.008994027222791417 -0.6459546298056476; … ; 0.29755799622550655 -0.1766613969659203 … 0.5312692804553979 -0.05330055297057241; 0.2573832433870322 -0.06258355995933354 … 0.25344740077728845 -0.002411604141597575], bias = [0.10027068587908458; -0.10766957234124348; … ; 0.42349703635753366; 0.17434276484229624;;]), layer_4 = (weight = [-0.014711502235522571 -0.13855367498089602 … 0.3958780860342399 -0.1795812590826653; -0.2524841634681116 -0.6257097652656107 … 0.026188336653333432 -0.015253077308645018; … ; -0.116091103256931 -0.05339885811149825 … -0.0016860410623169809 0.07939168165377526; -0.3969502416184398 -0.24549130524807955 … -0.48117528897575185 0.5646282303036994], bias = [0.037176101454920785; -0.44818922493378854; … ; 0.43575038752194967; -0.030357611197554358;;]), layer_5 = (weight = [-0.03416507403224282 0.5627565407390558 … -0.7811649208379156 0.19014754340463821], bias = [-0.3175848838271405;;])), v = (layer_1 = (weight = [0.42393580763835986 -0.08222799963924218; -0.24096409489256806 0.18440935646319753; … ; 0.4692579954367364 0.005630931561363912; -0.03408337876903774 0.1326133584227415], bias = [0.012777870460193937; -0.09784916125493073; … ; -0.24983871306340227; -0.006953952806489529;;]), layer_2 = (weight = [-0.05901279926655112 -0.001998000944212048 … -0.6377568193942622 -0.2578776458815099; -0.2314613986164158 0.3548141053789896 … 0.5351012640970332 0.16679895958596708; … ; -0.4089744555833859 0.7043700461100437 … 0.5665233003566028 -0.32455562987385767; -0.6489501484159728 -0.4147235954216101 … 0.11500322605468674 0.3830209882047479], bias = [-0.346452001758481; -0.20491547264363158; … ; 0.11616287077508201; -0.19509086947496332;;]), layer_3 = (weight = [-0.05842860221782342 0.7192420350996159 … 0.5645331221679623 -0.5702986013719848; 0.10858509225471244 -0.15387966340337342 … -0.5156844672409528 0.26058446700181354; … ; 0.3338646286031855 -0.19310702427999069 … 0.36615132416928947 -0.9626579537353352; 0.06798185131871602 -0.4378071929502009 … 0.39926338989236704 -0.9139614708212102], bias = [0.19184249740268017; -0.014762858654255487; … ; -0.15383477961763742; -0.15620901931092523;;]), layer_4 = (weight = [0.23972350414682536 -0.03009819674644918 … 0.301229955771434 0.32293963593509023; -0.4688192778461245 0.3210657562967202 … 0.6647884342380783 0.2229652560272662; … ; 0.22144750369439617 0.14577511822459366 … 0.8569186601087826 -0.19679269521089335; -0.04783597667729154 -0.8081348459620753 … 0.6106845575082103 0.10374996545736871], bias = [0.1841322364412494; -0.0334663946741817; … ; 0.09064010164619349; 0.40764636584931807;;]), layer_5 = (weight = [-0.8516231577138725 0.18241852334944356 … 0.3008294250406561 -0.25278271136825614], bias = [0.1479285404417682;;])))
phi = pinn.phi
+u: ComponentVector{Float64}(u = (layer_1 = (weight = [-0.48699429085187385 -0.44289284420707303; -0.8721354176728243 -0.5419596061072773; … ; 0.43259830235355584 0.15430169983281578; -0.27742367200918455 0.5952190886533241], bias = [0.4365535630717237; 0.45925392870971504; … ; 0.12939995396552653; -0.1912638881050873;;]), layer_2 = (weight = [-0.34481373220543915 0.032979018385474665 … 0.30518198393801893 0.22767044198680286; -0.40826558132309826 0.11734510542649201 … 0.09948341418863074 -0.46597982401245136; … ; -0.3326936566890714 -0.02234420512396556 … -0.0960643247530121 0.3811195721552705; 0.45724871009971985 -0.1284634652623294 … 0.5945940111328981 -0.8351446676294998], bias = [-0.27121826281938594; -0.2047758456741545; … ; -0.22344959375288295; 0.11327647895536734;;]), layer_3 = (weight = [0.4342280267238861 0.6292497817109693 … 0.6827980068317406 -0.24705246881496365; 1.027887500954219 -0.0037808067139295724 … -0.17184177402286338 0.637982049942793; … ; -0.23960129793706733 -0.29903054478952484 … 0.20272343146838928 -0.1440482295608264; 0.13954049224745868 -0.14414149135657026 … -0.21485601231361035 -0.021724885839211342], bias = [-0.009560201991292747; 0.0018001823480277524; … ; 0.0004722222978457131; -0.31039981531739563;;]), layer_4 = (weight = [-0.6236000866840291 -0.6785677834633723 … 0.45533141427384555 0.586632795898397; -0.1500977457653176 0.10229503251131447 … -0.18445692630268232 -0.2596064789881811; … ; -0.21237414914971683 -0.6594066815112672 … 0.7231136666146463 -0.19326472290636731; 0.373953127126457 -0.18501810228192156 … -0.5332868272548089 0.9046712488200367], bias = [0.3616155659657889; -0.06319024064424644; … ; 0.12767846701998006; -0.06446301090820564;;]), layer_5 = (weight = [0.35352133445056255 0.2486460006624632 … -1.4572313403603547 -0.03408740180494113], bias = [0.8487799374390652;;])), v = (layer_1 = (weight = [0.0843912105263371 0.12413992824969379; 0.9267179622371593 0.2163033956853533; … ; 0.2564010768541753 0.2506858937930206; 0.11204165385205478 0.7250505202960196], bias = [0.24353520257703198; -0.5127039337057188; … ; -0.6107347408250747; -0.36787337341422566;;]), layer_2 = (weight = [0.407847546481347 0.288539088251196 … 0.7025206797449124 0.940181131748632; 0.03363152653870015 0.33406661512102426 … 0.3853086131102247 0.37713502255632564; … ; 0.15355881955968503 0.20903019364437608 … 0.1262872673779253 0.17571717503006298; -0.029860158763338456 -0.587389818037452 … -0.6849669974213755 -0.18168395451510388], bias = [-0.06331590366084755; 0.057604798456145086; … ; -0.0006791078174871073; 0.3712216751859612;;]), layer_3 = (weight = [-0.09160458309821874 0.6446630744204302 … 0.32545765065325755 -0.0046398137803364894; -0.2610133032249579 0.5795231094138125 … 0.2248429006659782 -0.21215928664708852; … ; 0.18095120421260288 -0.15379627675473334 … -0.36051749017626644 0.3736384278538636; 0.06671465027948062 -0.21799996553311382 … -0.12456750394300366 0.12375262901382003], bias = [-0.18729115968061846; 0.06250196079207758; … ; 0.18304894191951065; 0.06530898324255346;;]), layer_4 = (weight = [-0.22987934764463488 0.3849537752598192 … 0.33446049639451164 -0.2993052685904012; 0.04280429693133818 0.18319379000149708 … -0.46707016597787393 0.2875012163906353; … ; -0.06465563404503999 -0.11810830248785659 … 0.16091373508954077 -0.04184321448877294; 0.41664169587220073 -0.2865192244368099 … -0.2688388129669361 0.16257832326077934], bias = [0.31022675934909333; -0.2628171334146378; … ; 0.14859150312309558; 0.43246286894226377;;]), layer_5 = (weight = [-0.588962599612402 1.2297697200606348 … 0.3600628605110921 0.39859579919788707], bias = [0.03250433479533244;;])))
phi = pinn.phi
 ps = res.u
 
 xs, ts= [infimum(d.domain):0.01:supremum(d.domain) for d in pde_system.domain]
@@ -67,4 +67,4 @@
 prob.p[2] = new_data
 prob = remake(prob; u0 = res.u)
 # res = Optimization.solve(prob, bfgs; maxiters=1000)
OptimizationProblem. In-place: true
-u0: ComponentVector{Float64}(u = (layer_1 = (weight = [0.21235830510366635 0.2680283891850633; -0.45441010437339874 0.5272408868657771; … ; -0.012641087637732642 -0.11874687027166592; 1.270684429912298 -0.4699585932243045], bias = [0.4530232334529752; 0.23671951305872013; … ; -0.11308253709979735; 0.08440662091126779;;]), layer_2 = (weight = [-0.42857487812090944 0.2458059194711604 … 0.2701789263122683 -0.28282335990278856; 0.49385938634697363 -0.07082276024708827 … 0.2710352741985176 -0.3328344288374426; … ; -0.013170804484413342 0.11106983000262939 … 0.14634856288132697 -0.4513430688328091; 0.837799435886789 -0.4194466497045731 … 0.09069386898459723 0.7721847747449293], bias = [-0.06498843167447485; -0.3411647353670892; … ; 0.16752147282395385; -0.04802249607649675;;]), layer_3 = (weight = [0.3413608918164652 -0.3998676556441467 … -0.19123499464068425 0.5030296059914127; 0.22191415553801175 -0.5726112565845367 … 0.008994027222791417 -0.6459546298056476; … ; 0.29755799622550655 -0.1766613969659203 … 0.5312692804553979 -0.05330055297057241; 0.2573832433870322 -0.06258355995933354 … 0.25344740077728845 -0.002411604141597575], bias = [0.10027068587908458; -0.10766957234124348; … ; 0.42349703635753366; 0.17434276484229624;;]), layer_4 = (weight = [-0.014711502235522571 -0.13855367498089602 … 0.3958780860342399 -0.1795812590826653; -0.2524841634681116 -0.6257097652656107 … 0.026188336653333432 -0.015253077308645018; … ; -0.116091103256931 -0.05339885811149825 … -0.0016860410623169809 0.07939168165377526; -0.3969502416184398 -0.24549130524807955 … -0.48117528897575185 0.5646282303036994], bias = [0.037176101454920785; -0.44818922493378854; … ; 0.43575038752194967; -0.030357611197554358;;]), layer_5 = (weight = [-0.03416507403224282 0.5627565407390558 … -0.7811649208379156 0.19014754340463821], bias = [-0.3175848838271405;;])), v = (layer_1 = (weight = [0.42393580763835986 -0.08222799963924218; -0.24096409489256806 0.18440935646319753; … ; 0.4692579954367364 0.005630931561363912; -0.03408337876903774 0.1326133584227415], bias = [0.012777870460193937; -0.09784916125493073; … ; -0.24983871306340227; -0.006953952806489529;;]), layer_2 = (weight = [-0.05901279926655112 -0.001998000944212048 … -0.6377568193942622 -0.2578776458815099; -0.2314613986164158 0.3548141053789896 … 0.5351012640970332 0.16679895958596708; … ; -0.4089744555833859 0.7043700461100437 … 0.5665233003566028 -0.32455562987385767; -0.6489501484159728 -0.4147235954216101 … 0.11500322605468674 0.3830209882047479], bias = [-0.346452001758481; -0.20491547264363158; … ; 0.11616287077508201; -0.19509086947496332;;]), layer_3 = (weight = [-0.05842860221782342 0.7192420350996159 … 0.5645331221679623 -0.5702986013719848; 0.10858509225471244 -0.15387966340337342 … -0.5156844672409528 0.26058446700181354; … ; 0.3338646286031855 -0.19310702427999069 … 0.36615132416928947 -0.9626579537353352; 0.06798185131871602 -0.4378071929502009 … 0.39926338989236704 -0.9139614708212102], bias = [0.19184249740268017; -0.014762858654255487; … ; -0.15383477961763742; -0.15620901931092523;;]), layer_4 = (weight = [0.23972350414682536 -0.03009819674644918 … 0.301229955771434 0.32293963593509023; -0.4688192778461245 0.3210657562967202 … 0.6647884342380783 0.2229652560272662; … ; 0.22144750369439617 0.14577511822459366 … 0.8569186601087826 -0.19679269521089335; -0.04783597667729154 -0.8081348459620753 … 0.6106845575082103 0.10374996545736871], bias = [0.1841322364412494; -0.0334663946741817; … ; 0.09064010164619349; 0.40764636584931807;;]), layer_5 = (weight = [-0.8516231577138725 0.18241852334944356 … 0.3008294250406561 -0.25278271136825614], bias = [0.1479285404417682;;])))
+u0: ComponentVector{Float64}(u = (layer_1 = (weight = [-0.48699429085187385 -0.44289284420707303; -0.8721354176728243 -0.5419596061072773; … ; 0.43259830235355584 0.15430169983281578; -0.27742367200918455 0.5952190886533241], bias = [0.4365535630717237; 0.45925392870971504; … ; 0.12939995396552653; -0.1912638881050873;;]), layer_2 = (weight = [-0.34481373220543915 0.032979018385474665 … 0.30518198393801893 0.22767044198680286; -0.40826558132309826 0.11734510542649201 … 0.09948341418863074 -0.46597982401245136; … ; -0.3326936566890714 -0.02234420512396556 … -0.0960643247530121 0.3811195721552705; 0.45724871009971985 -0.1284634652623294 … 0.5945940111328981 -0.8351446676294998], bias = [-0.27121826281938594; -0.2047758456741545; … ; -0.22344959375288295; 0.11327647895536734;;]), layer_3 = (weight = [0.4342280267238861 0.6292497817109693 … 0.6827980068317406 -0.24705246881496365; 1.027887500954219 -0.0037808067139295724 … -0.17184177402286338 0.637982049942793; … ; -0.23960129793706733 -0.29903054478952484 … 0.20272343146838928 -0.1440482295608264; 0.13954049224745868 -0.14414149135657026 … -0.21485601231361035 -0.021724885839211342], bias = [-0.009560201991292747; 0.0018001823480277524; … ; 0.0004722222978457131; -0.31039981531739563;;]), layer_4 = (weight = [-0.6236000866840291 -0.6785677834633723 … 0.45533141427384555 0.586632795898397; -0.1500977457653176 0.10229503251131447 … -0.18445692630268232 -0.2596064789881811; … ; -0.21237414914971683 -0.6594066815112672 … 0.7231136666146463 -0.19326472290636731; 0.373953127126457 -0.18501810228192156 … -0.5332868272548089 0.9046712488200367], bias = [0.3616155659657889; -0.06319024064424644; … ; 0.12767846701998006; -0.06446301090820564;;]), layer_5 = (weight = [0.35352133445056255 0.2486460006624632 … -1.4572313403603547 -0.03408740180494113], bias = [0.8487799374390652;;])), v = (layer_1 = (weight = [0.0843912105263371 0.12413992824969379; 0.9267179622371593 0.2163033956853533; … ; 0.2564010768541753 0.2506858937930206; 0.11204165385205478 0.7250505202960196], bias = [0.24353520257703198; -0.5127039337057188; … ; -0.6107347408250747; -0.36787337341422566;;]), layer_2 = (weight = [0.407847546481347 0.288539088251196 … 0.7025206797449124 0.940181131748632; 0.03363152653870015 0.33406661512102426 … 0.3853086131102247 0.37713502255632564; … ; 0.15355881955968503 0.20903019364437608 … 0.1262872673779253 0.17571717503006298; -0.029860158763338456 -0.587389818037452 … -0.6849669974213755 -0.18168395451510388], bias = [-0.06331590366084755; 0.057604798456145086; … ; -0.0006791078174871073; 0.3712216751859612;;]), layer_3 = (weight = [-0.09160458309821874 0.6446630744204302 … 0.32545765065325755 -0.0046398137803364894; -0.2610133032249579 0.5795231094138125 … 0.2248429006659782 -0.21215928664708852; … ; 0.18095120421260288 -0.15379627675473334 … -0.36051749017626644 0.3736384278538636; 0.06671465027948062 -0.21799996553311382 … -0.12456750394300366 0.12375262901382003], bias = [-0.18729115968061846; 0.06250196079207758; … ; 0.18304894191951065; 0.06530898324255346;;]), layer_4 = (weight = [-0.22987934764463488 0.3849537752598192 … 0.33446049639451164 -0.2993052685904012; 0.04280429693133818 0.18319379000149708 … -0.46707016597787393 0.2875012163906353; … ; -0.06465563404503999 -0.11810830248785659 … 0.16091373508954077 -0.04184321448877294; 0.41664169587220073 -0.2865192244368099 … -0.2688388129669361 0.16257832326077934], bias = [0.31022675934909333; -0.2628171334146378; … ; 0.14859150312309558; 0.43246286894226377;;]), layer_5 = (weight = [-0.588962599612402 1.2297697200606348 … 0.3600628605110921 0.39859579919788707], bias = [0.03250433479533244;;]))) diff --git a/dev/tutorials/allen.png b/dev/tutorials/allen.png index 8d714335..aa774bf7 100644 Binary files a/dev/tutorials/allen.png and b/dev/tutorials/allen.png differ diff --git a/dev/tutorials/allen_cahn/index.html b/dev/tutorials/allen_cahn/index.html index 52fc5978..37d4f389 100644 --- a/dev/tutorials/allen_cahn/index.html +++ b/dev/tutorials/allen_cahn/index.html @@ -24,7 +24,7 @@ sampler = QuasiRandomSampler(500, (300, 100)) strategy = NonAdaptiveTraining(1, (50, 1)) prob = Sophon.discretize(allen, pinn, sampler, strategy)
OptimizationProblem. In-place: true
-u0: ComponentVector{Float64}(layer_1 = (weight = [-1.90497624874115 -0.5052847266197205; 1.7149256467819214 0.782629132270813; … ; 0.9136808514595032 1.9797195196151733; -0.827705979347229 0.3881980776786804], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = [-0.597122311592102 -0.6003364324569702 … -0.1445828676223755 0.5894452333450317; -0.02416171133518219 0.13886630535125732 … 0.5808997750282288 0.05358504876494408; … ; -0.28547850251197815 -0.2232820838689804 … -0.12016254663467407 0.23830394446849823; -0.05340937152504921 -0.6151185631752014 … -0.029530102387070656 -0.47029513120651245], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_3 = (weight = [0.17813539505004883 0.4108351469039917 … 0.651142954826355 0.10730241239070892; -0.6052757501602173 0.09043336659669876 … 0.17894220352172852 -0.608291745185852; … ; -0.7174080610275269 0.1529277116060257 … 0.36324289441108704 -0.3257288634777069; -0.1559743583202362 -0.5963383913040161 … -0.25012439489364624 0.7197853922843933], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_4 = (weight = [-0.6600335836410522 -0.5522302389144897 … -0.2946714460849762 -0.25092291831970215; 0.3752587139606476 0.4414036273956299 … 0.5433818697929382 -0.4244040846824646; … ; -0.5307813882827759 -0.5451334118843079 … 0.05296699330210686 -0.4112752676010132; 0.6428700089454651 0.41608771681785583 … 0.5899282097816467 -0.5910473465919495], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_5 = (weight = [0.3891390860080719 0.25204160809516907 … -0.29719898104667664 -0.4872767925262451], bias = [0.0;;]))

We solve the equation sequentially in time.

function train(allen, prob, sampler, strategy)
+u0: ComponentVector{Float64}(layer_1 = (weight = [-1.9405659437179565 -1.5879656076431274; 0.3153097629547119 -1.407899260520935; … ; 1.3696730136871338 1.9997707605361938; -0.841397225856781 -0.2626766562461853], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = [-0.4295133352279663 -0.5162713527679443 … -0.11731850355863571 -0.5434175133705139; -0.34614792466163635 0.07568603754043579 … 0.3592473268508911 0.005838295910507441; … ; 0.223590686917305 0.30537596344947815 … 0.34158584475517273 -0.35143253207206726; -0.29853805899620056 -0.13304564356803894 … -0.6490473747253418 0.7096888422966003], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_3 = (weight = [0.3973458409309387 0.35017043352127075 … -0.5264913439750671 -0.6108897924423218; 0.16252851486206055 0.05907474458217621 … 0.1412469893693924 -0.4892854690551758; … ; -0.33154091238975525 -0.013726215809583664 … -0.2604060471057892 -0.5785760879516602; 0.18257515132427216 0.535335898399353 … -0.4754335582256317 -0.18667250871658325], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_4 = (weight = [-0.20740197598934174 -0.13300375640392303 … -0.44765418767929077 -0.432991623878479; 0.22740629315376282 0.070879265666008 … 0.48599758744239807 0.2214891016483307; … ; -0.6215541362762451 0.48001760244369507 … 0.4411102831363678 0.44129860401153564; -0.2898039221763611 0.6109877228736877 … 0.23752930760383606 -0.5122167468070984], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_5 = (weight = [0.632839024066925 -0.13037823140621185 … 0.08803772181272507 0.15518182516098022], bias = [0.0;;]))

We solve the equation sequentially in time.

function train(allen, prob, sampler, strategy)
     bfgs = BFGS()
     res = Optimization.solve(prob, bfgs; maxiters=2000)
 
@@ -38,10 +38,10 @@
 end
 
 res = train(allen, prob, sampler, strategy)
retcode: Failure
-u: ComponentVector{Float64}(layer_1 = (weight = [-1.7124722584443561 -0.6626288357429024; 2.5431462852976203 1.1582848883367194; … ; 2.218189152818594 1.537098332527324; -0.403239799671029 0.3461117262137156], bias = [0.2612446173907737; 0.016879769505023026; … ; -2.0818736736540546; 0.3150483824818491;;]), layer_2 = (weight = [-0.5132657694835222 -0.8170322995222681 … -0.10458962723643223 0.6334459436449862; -0.35584247085279447 0.2730446188916798 … 0.6048244222202275 0.05348830981783482; … ; -0.27110890801409215 -0.33858753059198904 … -0.11782228298231515 0.4142270560087858; 0.18241444155261957 -0.7644687356288308 … -0.1997939992264041 -0.05090736171375768], bias = [-0.05058027143234058; 0.25536625471748703; … ; 0.2449267113499491; 0.2897480542287442;;]), layer_3 = (weight = [0.33024073929637476 0.020625055576855608 … 0.7402060343917204 0.0614647138472577; -0.7432309685599191 0.6068203302029738 … -0.501263328959821 -1.144490396632953; … ; -0.5137310684636779 0.3069519972600186 … 0.27522279912536357 0.00470179061642502; 0.30458632661213497 -0.9690926927379679 … -0.24450884637993645 1.7725986756425411], bias = [0.3017055970879573; -0.3170475789828939; … ; 0.3868456755360319; 0.22748667214141266;;]), layer_4 = (weight = [-0.4639093260086085 -0.9894413625181959 … -0.4838768305279882 0.09028369026259245; 0.3032994670410524 0.7168168386486594 … 0.32441660225962343 -0.04380208345195881; … ; -0.6239585315189241 -0.5648313513674894 … 0.16225366680558811 -0.299101620149835; 1.0629105550674804 0.1383639700338704 … 1.1948135262109587 -2.8112902356228022], bias = [-0.2669774377276677; -0.006556603391913627; … ; -0.08937771758218023; 0.8984422946079207;;]), layer_5 = (weight = [0.11760106782533883 0.4748638100016508 … -0.016632762444827292 -0.6464780060140096], bias = [-0.5194116655382327;;]))

Let's plot the result.

using CairoMakie
+u: ComponentVector{Float64}(layer_1 = (weight = [-3.22904725292894 -0.9194010894871775; 0.18318590097472412 -1.2879168437047546; … ; 2.189022223934265 1.59625736730474; -1.4708519727096891 -0.5436088547814939], bias = [-0.6890708192736198; -0.5980018122548397; … ; -0.9214798119030119; 1.3211070913906633;;]), layer_2 = (weight = [-0.19990881013856823 -0.2209307397971279 … -0.12480483035788345 -0.9842242244901418; -0.49529554431720446 0.10542489977533254 … 0.8077797375785135 0.17272392173493203; … ; 0.0712953136860848 0.65362482942665 … 0.7716812396136031 -0.24976481836346565; -0.9802264391581639 -0.20348971448232722 … -0.26204398534489853 0.7031792854961609], bias = [-0.303511763919823; -0.1094856821855322; … ; -0.39499809286935594; 0.3503880802398379;;]), layer_3 = (weight = [0.6829105450816213 1.3594612939461157 … 0.001170396239907667 -1.9375919623196447; 0.12529439006323634 0.3875929834467002 … 0.4035793723735131 -0.20173555163266996; … ; -0.708297017188542 -0.16445197703844383 … -0.5656520975888647 -0.34005459302601954; -0.34066429957756705 0.6054188956295289 … -0.30770112889856716 -0.08632780532793395], bias = [0.06141746393321522; -0.5743306166908436; … ; 0.2899898088089976; 0.4912764242603182;;]), layer_4 = (weight = [-0.09527440148560803 -0.6291061947429013 … -1.0400805741751566 0.2846757593330914; 0.5242416772134829 0.21708924861225645 … 0.3681379793926206 -0.13620155467008258; … ; -1.1949460504504488 0.09480146781756098 … 0.7923908202161452 1.8984397814540976; -0.12822503486708617 0.5649260748797924 … -0.2652542799118422 -0.3297163993072842], bias = [-0.10886310869368375; -0.06729488606405641; … ; 0.36386222850314726; -0.24703319498094345;;]), layer_5 = (weight = [0.5720404262834169 0.06099568315011946 … 0.8408457607236473 0.7721270274371559], bias = [-1.2586760090434286;;]))

Let's plot the result.

using CairoMakie
 
 phi = pinn.phi
 xs, ts = [infimum(d.domain):0.01:supremum(d.domain) for d in allen.domain]
 axis = (xlabel="t", ylabel="x", title="Prediction")
 u_pred = [sum(pinn.phi([x, t], res.u)) for x in xs, t in ts]
-fig, ax, hm = heatmap(ts, xs, u_pred', axis=axis)

+fig, ax, hm = heatmap(ts, xs, u_pred', axis=axis)

diff --git a/dev/tutorials/convection.png b/dev/tutorials/convection.png index b1965a22..fa9dba22 100644 Binary files a/dev/tutorials/convection.png and b/dev/tutorials/convection.png differ diff --git a/dev/tutorials/convection/index.html b/dev/tutorials/convection/index.html index 0fdeb3cc..16a6aab5 100644 --- a/dev/tutorials/convection/index.html +++ b/dev/tutorials/convection/index.html @@ -43,7 +43,7 @@ prob = Sophon.discretize(convection, pinn, sampler, strategy) @showprogress res = Optimization.solve(prob, BFGS(); maxiters = 1000)
retcode: Success
-u: ComponentVector{Float64}(filters = (filter_1 = (bias = [0.0801127350552688; 0.9878304985537804; … ; -0.42547965386480846; 0.7087228347171748;;]), filter_2 = (bias = [-0.9899670950127777; 0.19506044158455318; … ; 0.4795276037949839; 0.3662388680782763;;]), filter_3 = (bias = [-0.11744226335160356; 0.12291122138400137; … ; 0.6794857826452417; -0.18792362494744289;;]), filter_4 = (bias = [0.2816299502014777; 0.05812022800600594; … ; 0.739285052766362; 0.3516385018477974;;])), linear_layers = (layer_1 = (weight = [0.3211150042364514 -0.24361861661259787 … 0.03155220126589449 -0.04336113290779269; 0.019264760355609065 0.3213186901240722 … -0.4445017009376741 -0.275239636510808; … ; 0.07804445177265511 0.1907187184470353 … -0.18351843138252905 0.16042902169457007; -0.420872010926429 0.03585877845866477 … 0.30679885654627725 0.09194097141819864], bias = [-0.06846251165331603; 0.05643469527135615; … ; 0.16649833563059901; -0.04793738447726613;;]), layer_2 = (weight = [-0.1569104207997399 0.11492097184374174 … 0.17977086652010665 0.22610162198698514; -0.18488281402777346 -0.3206469493924477 … -0.09328562600820964 -0.3092812581564792; … ; 0.10362591450764067 0.2757689800985728 … -0.07339736288535825 0.4035695131141563; 0.15365454233768525 -0.04016402193597685 … 0.3483514024023386 -0.3137619748394624], bias = [-0.022730701068193502; -0.11366771418706852; … ; 0.032126472723722155; -0.010849519061315016;;]), layer_3 = (weight = [-0.025978767091962673 0.3174275281111931 … -0.04234667970930941 -0.12939162994884346; -0.3141759886144951 0.2721081090841828 … -0.06553830706903276 -0.28089743573714543; … ; 0.10055132198918212 0.4588639191025014 … -0.006105154115224082 -0.2334118108853631; 0.38994106533414086 0.49811340647709096 … 0.01188750473227132 0.01144855946659487], bias = [0.007357181111507272; -0.024658665303578273; … ; -0.058185349521786424; 0.04820247327206237;;])), output_layer = (weight = [-0.02187257485991355 -0.0019512606657578488 … -0.29367464216842476 0.055838179086272394], bias = [0.13471491686784043;;]))

Let's visualize the result.

phi = pinn.phi
+u: ComponentVector{Float64}(filters = (filter_1 = (bias = [-0.223632198697637; 0.20272356651094506; … ; -0.6041644525908507; -0.4844691522622191;;]), filter_2 = (bias = [0.6935748614485714; -0.9865401708651692; … ; -0.43202833529269563; -0.5401951699703942;;]), filter_3 = (bias = [0.16314665822792648; -0.9595818144084337; … ; 0.5946956273529056; 0.7001304160499392;;]), filter_4 = (bias = [-0.136953805914684; -0.4414979625068518; … ; -0.5020302508369624; 0.2891228560355374;;])), linear_layers = (layer_1 = (weight = [-0.16976110582232826 0.02119284138602038 … -0.3593949940937835 0.12267051751976801; -0.2723453145779497 0.2557073837274297 … 0.13963541386743109 0.3010523949305811; … ; -0.01812985093830456 0.5229193725540827 … -0.1300189137590181 0.14213140136797067; 0.015104961396967391 -0.03184977266974181 … -0.3741632882105871 -0.13739806554800257], bias = [0.13965823016465087; -0.0011537705117211103; … ; 0.16206060943867184; -0.02752309047055468;;]), layer_2 = (weight = [0.21155821253452967 0.3820943067829835 … 0.10974235524544174 0.26052679419381725; 0.2638052062023701 0.27044155761053 … -0.3415986526853229 0.11707178722032237; … ; -0.2520549872467363 0.08160960964966443 … 0.2876433270633382 0.5620696706952033; 0.37284970488799013 -0.45537476868938054 … -0.42906552444747886 -0.09984532016718561], bias = [-0.0418680527564855; 0.1636185491375752; … ; -0.09880142458729324; -0.0014247839803322568;;]), layer_3 = (weight = [-0.19584198189392757 -0.07740531736831244 … 0.23253805304969627 0.28219621711045506; 0.5531425805915967 -0.06012529096474475 … 0.12706256856743944 0.24309727579626492; … ; 0.3337345381237087 -0.26185423125623414 … -0.31234231929716944 0.35447785596439324; -0.3361126537008412 -0.04388342208256112 … 0.09595210930988893 0.027305576714161364], bias = [-0.12309999562955885; -0.04486121853633411; … ; 0.12710302980802946; -0.01412555620510203;;])), output_layer = (weight = [0.00026474578969388644 0.15111581465444696 … 0.279353568848775 -0.010925502141569747], bias = [0.20869566534947281;;]))

Let's visualize the result.

phi = pinn.phi
 
 xs, ts= [infimum(d.domain):0.01:supremum(d.domain) for d in domains]
 u_pred = [sum(phi([x,t],res.u)) for x in xs, t in ts]
@@ -54,4 +54,4 @@
 display(fig)

We can verify that our model is indeed, periodic.

xs, ts= [infimum(d.domain):0.01:supremum(d.domain)*2 for d in domains]
 u_pred = [sum(phi([x,t],res.u)) for x in xs, t in ts]
 fig, ax, hm = heatmap(ts, xs, u_pred', axis=(xlabel="t", ylabel="x", title="c = $c"))
-display(fig)

+display(fig)

diff --git a/dev/tutorials/data.png b/dev/tutorials/data.png index af87efe1..735b21ec 100644 Binary files a/dev/tutorials/data.png and b/dev/tutorials/data.png differ diff --git a/dev/tutorials/discontinuous/index.html b/dev/tutorials/discontinuous/index.html index 48c023d5..2db072f6 100644 --- a/dev/tutorials/discontinuous/index.html +++ b/dev/tutorials/discontinuous/index.html @@ -42,28 +42,28 @@ end
train (generic function with 1 method)

Plot the result

@time ps, st = train(model, x_train, y_train)
 y_pred = model(x_test,ps,st)[1]
 Plots.plot(vec(x_test), vec(y_pred),label="Prediction",line = (:dot, 4))
-Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.45990733437577325
-Epoch 101 ||  0.016613809489428647
-Epoch 201 ||  0.014863441983620244
-Epoch 301 ||  0.013429513264967994
-Epoch 401 ||  0.012966258053056316
-Epoch 501 ||  0.01276935752567519
-Epoch 601 ||  0.012494646172296193
-Epoch 701 ||  0.012108585905123505
-Epoch 801 ||  0.011453405171473162
-Epoch 901 ||  0.010423841053027601
-Epoch 1001 ||  0.008964601725663893
-Epoch 1101 ||  0.007920167102386582
-Epoch 1201 ||  0.007606479033105528
-Epoch 1301 ||  0.007549593095683187
-Epoch 1401 ||  0.0076046829769499755
-Epoch 1501 ||  0.007558388000577735
-Epoch 1601 ||  0.007675849447096801
-Epoch 1701 ||  0.007554336496579123
-Epoch 1801 ||  0.007499389840002896
-Epoch 1901 ||  0.007490394147148696
-Epoch 2000 ||  0.0074848047520083075
-  7.916466 seconds (9.14 M allocations: 1.210 GiB, 2.56% gc time, 93.89% compilation time)

Siren

We use four hidden layers with 50 neurons in each.

model = Siren(1,50,50,50,50,1; omega = 30f0)
Chain(
+Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  1.0590699266642791
+Epoch 101 ||  0.020231271908541827
+Epoch 201 ||  0.01578716013468039
+Epoch 301 ||  0.015510059245085634
+Epoch 401 ||  0.015148235408212968
+Epoch 501 ||  0.014706740423341049
+Epoch 601 ||  0.014120411518537148
+Epoch 701 ||  0.013585754334795029
+Epoch 801 ||  0.012997857403312106
+Epoch 901 ||  0.012239499662658377
+Epoch 1001 ||  0.011119786859999212
+Epoch 1101 ||  0.009404929764343324
+Epoch 1201 ||  0.008027226255993275
+Epoch 1301 ||  0.007516315741717599
+Epoch 1401 ||  0.0073703771310022525
+Epoch 1501 ||  0.007314786548702886
+Epoch 1601 ||  0.007288798632380535
+Epoch 1701 ||  0.007263203840318896
+Epoch 1801 ||  0.007251144122996908
+Epoch 1901 ||  0.00722760400097479
+Epoch 2000 ||  0.007214029536895217
+  7.534014 seconds (9.18 M allocations: 1.213 GiB, 1.88% gc time, 94.37% compilation time)

Siren

We use four hidden layers with 50 neurons in each.

model = Siren(1,50,50,50,50,1; omega = 30f0)
Chain(
     layer_1 = Dense(1 => 50, sin),      # 100 parameters
     layer_2 = Dense(50 => 50, sin),     # 2_550 parameters
     layer_3 = Dense(50 => 50, sin),     # 2_550 parameters
@@ -73,28 +73,28 @@
           #        plus 0 states.
@time ps, st = train(model, x_train, y_train)
 y_pred = model(x_test,ps,st)[1]
 Plots.plot(vec(x_test), vec(y_pred),label="Prediction",line = (:dot, 4))
-Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.6130885283588181
-Epoch 101 ||  0.0004031800758426102
-Epoch 201 ||  1.5161865704404341e-5
-Epoch 301 ||  4.2456456689293724e-7
-Epoch 401 ||  4.83454865562784e-9
-Epoch 501 ||  2.1341341491253954e-11
-Epoch 601 ||  4.4810905566664547e-13
-Epoch 701 ||  7.129119223984394e-14
-Epoch 801 ||  5.4109617866255905e-14
-Epoch 901 ||  3.1173931934102643e-14
-Epoch 1001 ||  7.23706717794571e-14
-Epoch 1101 ||  4.123988368868827e-14
-Epoch 1201 ||  5.3380706605301184e-14
-Epoch 1301 ||  3.9133614911788304e-14
-Epoch 1401 ||  3.187446272673868e-14
-Epoch 1501 ||  7.124675098741101e-14
-Epoch 1601 ||  7.121764590540485e-14
-Epoch 1701 ||  4.228649770885185e-14
-Epoch 1801 ||  4.848354778987959e-14
-Epoch 1901 ||  3.253724429851565e-14
-Epoch 2000 ||  1.2332888043376128e-13
-  5.453377 seconds (5.20 M allocations: 1.085 GiB, 1.83% gc time, 90.03% compilation time)

As we can see the model overfits the data, and the high frequencies cannot be optimized away. We need to tunning the hyperparameter omega

model = Siren(1,50,50,50,50,1; omega = 10f0)
Chain(
+Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  1.3091156999033473
+Epoch 101 ||  0.0009150918135832426
+Epoch 201 ||  4.436020435837387e-5
+Epoch 301 ||  4.839416688452758e-6
+Epoch 401 ||  4.789257463747301e-7
+Epoch 501 ||  3.337163790694092e-8
+Epoch 601 ||  1.5826176927545448e-9
+Epoch 701 ||  5.353938845142209e-11
+Epoch 801 ||  2.609884475499275e-12
+Epoch 901 ||  3.284602634667257e-13
+Epoch 1001 ||  1.0040992602041842e-13
+Epoch 1101 ||  4.739677424358578e-14
+Epoch 1201 ||  3.3700773349166116e-14
+Epoch 1301 ||  3.693241774066285e-14
+Epoch 1401 ||  5.5104991834592697e-14
+Epoch 1501 ||  3.645220201060331e-14
+Epoch 1601 ||  3.280297456692818e-14
+Epoch 1701 ||  5.80493224798081e-14
+Epoch 1801 ||  8.97736495844449e-14
+Epoch 1901 ||  2.803676906813886e-14
+Epoch 2000 ||  8.199231570241994e-14
+  5.259710 seconds (5.20 M allocations: 1.085 GiB, 1.54% gc time, 89.75% compilation time)

As we can see the model overfits the data, and the high frequencies cannot be optimized away. We need to tunning the hyperparameter omega

model = Siren(1,50,50,50,50,1; omega = 10f0)
Chain(
     layer_1 = Dense(1 => 50, sin),      # 100 parameters
     layer_2 = Dense(50 => 50, sin),     # 2_550 parameters
     layer_3 = Dense(50 => 50, sin),     # 2_550 parameters
@@ -104,28 +104,28 @@
           #        plus 0 states.
@time ps, st = train(model, x_train, y_train)
 y_pred = model(x_test,ps,st)[1]
 Plots.plot(vec(x_test), vec(y_pred),label="Prediction",line = (:dot, 4))
-Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.5573159010019687
-Epoch 101 ||  0.007573990260572845
-Epoch 201 ||  0.005277319984967942
-Epoch 301 ||  0.003933312923977755
-Epoch 401 ||  0.0030014281420378624
-Epoch 501 ||  0.00236744686072354
-Epoch 601 ||  0.0019036545263728544
-Epoch 701 ||  0.0014504805943072905
-Epoch 801 ||  0.0009867253501203562
-Epoch 901 ||  0.0005849682094765978
-Epoch 1001 ||  0.0003166966116396673
-Epoch 1101 ||  0.00016827082273708943
-Epoch 1201 ||  9.39843780968591e-5
-Epoch 1301 ||  5.904837945662935e-5
-Epoch 1401 ||  4.343129684291858e-5
-Epoch 1501 ||  3.645792335788726e-5
-Epoch 1601 ||  3.292123664860043e-5
-Epoch 1701 ||  3.065596494554646e-5
-Epoch 1801 ||  2.9408976336515656e-5
-Epoch 1901 ||  2.7450847922396047e-5
-Epoch 2000 ||  0.0002529427576773681
-  0.502393 seconds (1.26 M allocations: 901.506 MiB, 3.48% gc time)

Gaussian activation function

We can also try using a fully connected net with the gaussian activation function.

model = FullyConnected((1,50,50,50,50,1), gaussian)
Chain(
+Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.4522222762724683
+Epoch 101 ||  0.007544571385746059
+Epoch 201 ||  0.004984005812522651
+Epoch 301 ||  0.0034260633015820497
+Epoch 401 ||  0.002119768996513167
+Epoch 501 ||  0.0011391593461045294
+Epoch 601 ||  0.0005136306966784585
+Epoch 701 ||  0.0002736740366971622
+Epoch 801 ||  0.00019232905524836346
+Epoch 901 ||  0.00015024829641068963
+Epoch 1001 ||  0.00012172413672631833
+Epoch 1101 ||  0.00010004970050064713
+Epoch 1201 ||  0.00011150011105013645
+Epoch 1301 ||  6.756450541224577e-5
+Epoch 1401 ||  5.475064622701493e-5
+Epoch 1501 ||  4.434424184441749e-5
+Epoch 1601 ||  3.569371560719929e-5
+Epoch 1701 ||  2.913377728879314e-5
+Epoch 1801 ||  2.4051938953134517e-5
+Epoch 1901 ||  2.0033772196373887e-5
+Epoch 2000 ||  0.0009244428011692872
+  0.488891 seconds (1.26 M allocations: 901.550 MiB, 3.24% gc time)

Gaussian activation function

We can also try using a fully connected net with the gaussian activation function.

model = FullyConnected((1,50,50,50,50,1), gaussian)
Chain(
     layer_1 = Dense(1 => 50, gaussian),  # 100 parameters
     layer_2 = Dense(50 => 50, gaussian),  # 2_550 parameters
     layer_3 = Dense(50 => 50, gaussian),  # 2_550 parameters
@@ -135,28 +135,28 @@
           #        plus 0 states.
@time ps, st = train(model, x_train, y_train)
 y_pred = model(x_test,ps,st)[1]
 Plots.plot(vec(x_test), vec(y_pred),label="Prediction",line = (:dot, 4))
-Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.4141395407397555
-Epoch 101 ||  0.005514539776648461
-Epoch 201 ||  0.004896088932032563
-Epoch 301 ||  0.0029758474376051737
-Epoch 401 ||  0.0018911001506062425
-Epoch 501 ||  0.00038507696493150377
-Epoch 601 ||  1.0924326850615652e-5
-Epoch 701 ||  8.16716115508975e-7
-Epoch 801 ||  3.821551735547835e-7
-Epoch 901 ||  2.754016636104058e-6
-Epoch 1001 ||  4.381502861553284e-5
-Epoch 1101 ||  4.397567759335814e-9
-Epoch 1201 ||  7.556670391618637e-8
-Epoch 1301 ||  1.1686306229180665e-8
-Epoch 1401 ||  5.166991956909003e-10
-Epoch 1501 ||  1.8200894454841075e-7
-Epoch 1601 ||  4.0085786203630275e-10
-Epoch 1701 ||  5.258931855319772e-6
-Epoch 1801 ||  3.685079561617746e-9
-Epoch 1901 ||  1.0513837475556872e-7
-Epoch 2000 ||  2.1440031552013167e-10
-  4.275857 seconds (4.72 M allocations: 1.058 GiB, 2.13% gc time, 84.67% compilation time)

Quadratic activation function

quadratic is much cheaper to compute compared to the Gaussian activation function.

model = FullyConnected((1,50,50,50,50,1), quadratic)
Chain(
+Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.7651910872722903
+Epoch 101 ||  0.00717718325999821
+Epoch 201 ||  0.006130867896277117
+Epoch 301 ||  0.005727006153693893
+Epoch 401 ||  0.005409087178588007
+Epoch 501 ||  0.005089141724973403
+Epoch 601 ||  0.004718700022103287
+Epoch 701 ||  0.0034923326529008474
+Epoch 801 ||  0.00048310260881961436
+Epoch 901 ||  1.496432005528383e-5
+Epoch 1001 ||  1.0892783311592464e-6
+Epoch 1101 ||  3.5285216364037157e-7
+Epoch 1201 ||  1.823997820478425e-7
+Epoch 1301 ||  3.8450603049147106e-7
+Epoch 1401 ||  2.255256859999873e-6
+Epoch 1501 ||  4.533366074648209e-8
+Epoch 1601 ||  6.912124243662774e-7
+Epoch 1701 ||  2.704871796811748e-8
+Epoch 1801 ||  3.6226912240018825e-7
+Epoch 1901 ||  2.1247640093538603e-8
+Epoch 2000 ||  1.3559410549641136e-7
+  4.163979 seconds (4.72 M allocations: 1.058 GiB, 1.92% gc time, 84.40% compilation time)

Quadratic activation function

quadratic is much cheaper to compute compared to the Gaussian activation function.

model = FullyConnected((1,50,50,50,50,1), quadratic)
Chain(
     layer_1 = Dense(1 => 50, quadratic),  # 100 parameters
     layer_2 = Dense(50 => 50, quadratic),  # 2_550 parameters
     layer_3 = Dense(50 => 50, quadratic),  # 2_550 parameters
@@ -166,25 +166,25 @@
           #        plus 0 states.
@time ps, st = train(model, x_train, y_train)
 y_pred = model(x_test,ps,st)[1]
 Plots.plot(vec(x_test), vec(y_pred),label="Prediction",line = (:dot, 4))
-Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.5956744723915243
-Epoch 101 ||  0.0060953908559624615
-Epoch 201 ||  0.005581325959782406
-Epoch 301 ||  0.004898174475475823
-Epoch 401 ||  0.004014006586370171
-Epoch 501 ||  0.0025465243798491618
-Epoch 601 ||  0.0014137643619251475
-Epoch 701 ||  0.0006668807096132256
-Epoch 801 ||  0.00024054953281778236
-Epoch 901 ||  7.981238428015441e-5
-Epoch 1001 ||  1.8094970565695965e-5
-Epoch 1101 ||  3.439651146132967e-6
-Epoch 1201 ||  5.609564892140758e-7
-Epoch 1301 ||  7.729927583262037e-8
-Epoch 1401 ||  6.355703747200907e-9
-Epoch 1501 ||  9.881231812205006e-8
-Epoch 1601 ||  4.672103735821966e-10
-Epoch 1701 ||  1.1664905308939484e-11
-Epoch 1801 ||  7.331620992822458e-13
-Epoch 1901 ||  1.4966680709385856e-6
-Epoch 2000 ||  6.764300089527838e-11
-  4.081881 seconds (4.49 M allocations: 1.043 GiB, 2.03% gc time, 89.41% compilation time)

Conclusion

"Neural networks suppress high-frequency components" is a misinterpretation of the spectral bias. The accurate way of putting it is that the lower frequencies in the error are optimized first in the optimization process. This can be seen in Siren's example of overfitting data, where you do not have implicit regularization. The high frequency in the network will never go away because it has fitted the data perfectly.

Mainstream attributes the phenomenon that neural networks "suppress" high frequencies to gradient descent. This is not the whole picture. Initialization also plays an important role. Siren mitigates this problem by initializing larger weights in the first layer. In contrast, activation functions such as Gaussian have sufficiently large gradients and sufficiently large support of the second derivative with proper hyperparameters. Please refer to [1], [2] and [3] if you want to dive deeper into this.

+Plots.plot!(vec(x_test), vec(y_test),label="Exact",legend=:topleft)
Epoch 1 ||  0.2783266698114169
+Epoch 101 ||  0.005455941120455336
+Epoch 201 ||  0.004018162278432149
+Epoch 301 ||  0.0031642279303104293
+Epoch 401 ||  0.0017122366951467251
+Epoch 501 ||  0.00023197464162650068
+Epoch 601 ||  5.7620186456173e-6
+Epoch 701 ||  0.0007508500926152867
+Epoch 801 ||  1.0489280113533782e-8
+Epoch 901 ||  1.1171900356206456e-11
+Epoch 1001 ||  3.227332211611269e-13
+Epoch 1101 ||  2.6884792703371227e-6
+Epoch 1201 ||  1.531048588857288e-10
+Epoch 1301 ||  5.60263382341083e-6
+Epoch 1401 ||  2.440613731542617e-10
+Epoch 1501 ||  7.7142012237938e-7
+Epoch 1601 ||  5.01484447369686e-11
+Epoch 1701 ||  1.4849930331971447e-5
+Epoch 1801 ||  5.2631800434871996e-8
+Epoch 1901 ||  1.8839674951827722e-6
+Epoch 2000 ||  1.5594418974466195e-5
+  3.957018 seconds (4.48 M allocations: 1.042 GiB, 1.95% gc time, 89.26% compilation time)

Conclusion

"Neural networks suppress high-frequency components" is a misinterpretation of the spectral bias. The accurate way of putting it is that the lower frequencies in the error are optimized first in the optimization process. This can be seen in Siren's example of overfitting data, where you do not have implicit regularization. The high frequency in the network will never go away because it has fitted the data perfectly.

Mainstream attributes the phenomenon that neural networks "suppress" high frequencies to gradient descent. This is not the whole picture. Initialization also plays an important role. Siren mitigates this problem by initializing larger weights in the first layer. In contrast, activation functions such as Gaussian have sufficiently large gradients and sufficiently large support of the second derivative with proper hyperparameters. Please refer to [1], [2] and [3] if you want to dive deeper into this.

diff --git a/dev/tutorials/helmholtz.png b/dev/tutorials/helmholtz.png index 4273d882..a54dc454 100644 Binary files a/dev/tutorials/helmholtz.png and b/dev/tutorials/helmholtz.png differ diff --git a/dev/tutorials/helmholtz/index.html b/dev/tutorials/helmholtz/index.html index 9eef6093..8f358e5d 100644 --- a/dev/tutorials/helmholtz/index.html +++ b/dev/tutorials/helmholtz/index.html @@ -30,7 +30,7 @@ prob = Sophon.discretize(helmholtz, pinn, sampler, strategy) @showprogress res = Optimization.solve(prob, BFGS(); maxiters=1000)
retcode: Success
-u: ComponentVector{Float64}(filters = (filter_1 = (bias = [-0.4501021866631812; -0.14426951026432308; … ; -1.0424926617935315; -0.8858506061985617;;]), filter_2 = (bias = [0.22400592819029252; 0.5530384675215112; … ; 0.5824383659233768; 0.8912807109662234;;]), filter_3 = (bias = [-1.329010808217309; 0.08581227815459556; … ; -0.5434156139409319; 0.5139710688934496;;]), filter_4 = (bias = [-0.5508566123494708; 0.622196555447819; … ; -0.5091007697636055; 0.876441905782786;;]), filter_5 = (bias = [-0.9077998265520945; -0.6716020738581584; … ; 0.3289425747253741; -0.5498641368120656;;])), linear_layers = (layer_1 = (weight = [0.04221376985370541 0.12654064030456721 … -0.4124152854887392 -0.04930973671932143; -0.20618199270956725 0.16040654899037401 … -0.02969990380076121 0.2818713443785055; … ; -0.17673505711974316 0.06698997459623889 … -0.2068831148608149 -0.06695944949313529; 0.12210402776095071 0.29430329424234397 … -0.17612093951412167 -0.4859258689011493], bias = [-0.06950448781618893; -0.020530446037206898; … ; -0.07765190711109776; -0.015963155062720273;;]), layer_2 = (weight = [0.1912076075297271 0.4253435218176997 … -0.16159186415304685 -0.39384936591908765; -0.37270188418584166 0.17141777225100768 … -0.02160307610192841 0.17012960987762027; … ; -0.20957942007450125 -0.15516100050542683 … -0.44297707434259453 -0.1648301611958269; 0.2806496831559195 -0.3642642428299952 … 0.41643676444984895 -0.21733911263449257], bias = [-0.04990030094503441; -0.012829414302676114; … ; -0.012754221955793473; 0.028657362581770752;;]), layer_3 = (weight = [0.3561953366690573 -0.26656808956031613 … 0.16359003576657163 -0.21693821080660622; 0.10510745084769589 -0.25666861744228114 … -0.48741074998425743 0.17635563615381616; … ; -0.0630902313582285 -0.17673414642237326 … -0.22283433853369386 0.26924523643244874; 0.1449167354443314 0.3686261312982656 … -0.18728815245276798 0.24653096262384808], bias = [-0.027814544785646694; -0.0009630590521911547; … ; 0.023096914788353986; 0.003844315705627638;;]), layer_4 = (weight = [0.3349499266774192 -0.36924964624652507 … -0.2462086396459318 0.27705700638214115; 0.019720217264244062 -0.11402088222070277 … 0.2912919955476075 -0.1926518961666229; … ; 0.3402423882869826 0.24899755905171195 … 0.035509535657252275 -0.19596537194310615; -0.09154374299312848 0.1404374373226019 … -0.23843134176885 -0.05153600186799945], bias = [-0.0005702178499310689; -0.010692222590787406; … ; 0.00011468923837104198; -0.008614579258410492;;])), output_layer = (weight = [-0.03921779300558101 -0.2819819611341026 … -0.04789185536546425 0.05091221180323392], bias = [-0.04598117537911919;;]))

Let's plot the result.

phi = pinn.phi
+u: ComponentVector{Float64}(filters = (filter_1 = (bias = [-0.5747030054711377; -0.008425288486733531; … ; -0.8307000789158714; 0.25221859229264354;;]), filter_2 = (bias = [0.06413749364547709; 0.032710679833003437; … ; -0.35253205824165174; 0.5024799212045745;;]), filter_3 = (bias = [-0.659739075048217; 0.45385508188067125; … ; -0.4527587192806839; -0.6458731596309841;;]), filter_4 = (bias = [-0.7136022734030198; 0.23317855735495044; … ; 0.8364321187088444; -0.7534473062607875;;]), filter_5 = (bias = [-0.8546397216548066; -0.4058940101878304; … ; 1.2405518718185; -0.046444782070866296;;])), linear_layers = (layer_1 = (weight = [0.1092619103732732 0.09825284284734824 … 0.08473481620957261 0.3701953884899453; -0.15441081112052157 0.04503638501506098 … -0.3463409899132103 0.22519989437803306; … ; -0.3727169169874578 -0.08358826212470051 … 0.05610280438877534 0.25529300625336004; -0.23344004645762523 0.031197733076700016 … 0.07841998241426712 -0.34320399109191085], bias = [-0.0036585308546688558; 0.02041317878774791; … ; 0.0855488697928977; -0.01757951444786611;;]), layer_2 = (weight = [-0.3037895326917414 0.24456539700456326 … -0.4061089758004204 0.21074281373518272; 0.27721836869985644 -0.17178497333823375 … 0.23025357198869498 -0.1201620612277868; … ; -0.001885764219286839 0.21166446007564665 … -0.4077909683397925 0.39475597660197287; -0.13572340589331255 0.4028610987766618 … -0.1120784258447938 0.2056045900331838], bias = [-0.0008712240079252959; 0.0030805535033192753; … ; 0.04578277304021506; -0.002721398973533081;;]), layer_3 = (weight = [-0.07077713599973903 0.30727478933112606 … -0.2415316845615649 -0.0959414631580575; -0.3103266092605765 0.10289924260538837 … -0.43447238697432716 -0.39274350029737853; … ; -0.19446808840530516 0.25535435478052015 … 0.03701994294126222 -0.3383735948071197; -0.3333875811371799 0.359912384106956 … 0.06655750822612527 0.31621665146622174], bias = [0.012463670840786987; -0.029017279105347585; … ; -0.0008739663610191163; 0.025438042687345834;;]), layer_4 = (weight = [0.3027373890550928 -0.32936618079805097 … 0.3104357294596664 0.1265324543095176; 0.22613828848467543 -0.015735765453983697 … -0.2243550526272218 0.14080226202713456; … ; 0.023228436571823993 -0.07994106746376504 … 0.13302123864515686 -0.037133763421293; 0.29319397393873636 0.2371473445474859 … 0.2984580487592344 0.25089823250666604], bias = [0.002602910263920431; -0.021344768239838996; … ; -0.0033448484712859223; -0.02882546126258474;;])), output_layer = (weight = [0.2466636652817428 0.4179191595274341 … 0.7035143007927483 -0.34836360478223677], bias = [-0.021884447584900706;;]))

Let's plot the result.

phi = pinn.phi
 ps = res.u
 xs, ys= [infimum(d.domain):0.01:supremum(d.domain) for d in domains]
 u_analytic(x,y) = sinpi(a1*x)*sinpi(a2*y)
@@ -46,4 +46,4 @@
 Colorbar(fig[:, end+1], hm2)
 ax3, hm3 = heatmap(fig[1, end+1], xs, ys, abs.(u_pred-u_real), axis= merge(axis, (;title = "Absolute Error")))
 Colorbar(fig[:, end+1], hm3)
-fig

+fig

diff --git a/dev/tutorials/inverse/index.html b/dev/tutorials/inverse/index.html index 1c98c81b..ccee712e 100644 --- a/dev/tutorials/inverse/index.html +++ b/dev/tutorials/inverse/index.html @@ -52,4 +52,4 @@ x_pred = phi.x(ts, θ.x) y_pred = phi.x(ts, θ.y) z_pred = phi.x(ts, θ.z)
using Plots
-Plots.plot(vec(ts), [vec(x_pred),vec(y_pred),vec(z_pred)],  label=["x(t)" "y(t)" "z(t)"])   
+Plots.plot(vec(ts), [vec(x_pred),vec(y_pred),vec(z_pred)], label=["x(t)" "y(t)" "z(t)"]) diff --git a/dev/tutorials/ode/5038e852.svg b/dev/tutorials/ode/5038e852.svg deleted file mode 100644 index efa9b00e..00000000 --- a/dev/tutorials/ode/5038e852.svg +++ /dev/null @@ -1,54 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/dev/tutorials/ode/780a35d0.svg b/dev/tutorials/ode/780a35d0.svg new file mode 100644 index 00000000..02e285ca --- /dev/null +++ b/dev/tutorials/ode/780a35d0.svg @@ -0,0 +1,54 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/ode/index.html b/dev/tutorials/ode/index.html index 85e78069..f977d129 100644 --- a/dev/tutorials/ode/index.html +++ b/dev/tutorials/ode/index.html @@ -37,7 +37,7 @@ # Solving the problem using BFGS optimization @showprogress res = Optimization.solve(prob, BFGS(); maxiters=1000)
retcode: Failure
-u: ComponentVector{Float64}(x = (layer_1 = (weight = [-0.10307240235945218; 0.20524335785563627; … ; 0.5355470499641308; -1.372231864769174;;], bias = [-0.4023952689296157; 0.5170012855914634; … ; 0.5431277963064509; -0.44970827953211023;;]), layer_2 = (weight = [-0.48615462961323164 -0.22301580363358164 … 0.053135453712861345 0.3675847703174753; -0.14984715547734276 -0.0430466069953873 … 0.6261031249393715 0.36893068048976757; … ; -0.2267705425254419 -0.18977108063681816 … -0.4616675530798528 0.6203626416941829; -0.6285321727145035 -0.29932326973939616 … 0.8345211156587612 -0.2718009959978952], bias = [0.9806591886541035; 0.02200838114105574; … ; 0.0723841543868401; -0.050004968277878495;;]), layer_3 = (weight = [-0.5778425754072337 0.4562027964412001 … -1.0682195209765786 -0.4494081685320386; -0.27378381355830594 -0.05904911833252848 … 0.1722972309475631 -0.31330713640451385; … ; -0.14816537373574556 -0.6974447683086618 … -0.7472278411876068 0.6401713351311352; -0.8193130245253449 0.2710508155495368 … 0.20040783802229914 -0.032984809057605695], bias = [0.5019486093538965; -0.046078670219664046; … ; -0.09528381209093761; -0.812198111913398;;]), layer_4 = (weight = [-1.3891727992165617 0.7486414634380202 … -0.538177826122283 -0.4705516329022403], bias = [2.1649500510522914;;])), y = (layer_1 = (weight = [2.129918105698266; 1.5720661035521606; … ; -1.8527707677977703; -0.5118695775719995;;], bias = [0.5630420171380514; -0.06789345728023005; … ; 0.41528198096851987; -0.5014894717108193;;]), layer_2 = (weight = [-0.4030448848039122 -0.1763231810060179 … 0.450046849234256 0.3398376648893833; 0.5202684919028292 -0.7229884908764388 … 1.41647874693753 -0.26000977786703827; … ; 0.8951184084818947 -0.6974538866912285 … -0.6462587739920874 -0.28671971967004084; -0.011006380644749071 0.8153958004779922 … -0.41228140811867325 0.2896045425974877], bias = [0.0031715665710318115; 0.38289189836644755; … ; 0.193700777742735; 0.20351811042015946;;]), layer_3 = (weight = [-0.02080888025353576 -0.21905600693572272 … -0.14384045194954154 0.1729866883515601; 0.11029792309173195 -0.6573256097761087 … -0.4283885862006359 0.4390340594835812; … ; -0.6957146439280073 -0.0021347537892875672 … -0.7619608228441267 -0.5536917579128944; 0.14702082038730888 -0.6827585731485772 … 0.8506825908043109 -0.28108297214702815], bias = [0.4539156645371344; -0.01221266115534409; … ; -0.000910711758665037; 0.5771560803735261;;]), layer_4 = (weight = [0.7713402670882464 0.03737414429680034 … 0.3774388893141537 1.1466607764015544], bias = [1.2086947173977622;;])))

Next, we'll compare our results with a reference solution to verify our computations.

using OrdinaryDiffEq
+u: ComponentVector{Float64}(x = (layer_1 = (weight = [-1.6208004286621858; -1.022802061501601; … ; -1.2233343405827664; -0.11092959448713563;;], bias = [0.015057136428600798; 0.08769892976208132; … ; -0.03824307531941346; -0.25130606548563944;;]), layer_2 = (weight = [-0.1511505928692868 0.09849382913192219 … 0.8967319105821223 -1.1646901386192892; -0.2028496377485932 0.2727355696065825 … 0.5244582881773451 -0.1475365515974392; … ; 0.18199748798081772 0.02781509764703356 … 0.6254780574764074 0.280424751677824; 0.2732220920816425 -0.4770988799977176 … 1.1070626359166045 -0.5905478698394485], bias = [0.12555776376094527; 0.13776454416622985; … ; 0.07844852922048183; -0.30038172679555597;;]), layer_3 = (weight = [1.8960741767328433 0.897593930026357 … -0.7475096143180775 0.7392693849676583; 0.7483388351693743 0.3043077248944923 … -1.1730192584382415 0.5883579656700318; … ; 0.8564556841505296 0.5333925582909655 … -0.292708225212297 0.35938470324267624; -0.007238412031216462 0.2514326076577677 … -0.5727754038253156 0.23915710657925762], bias = [1.0145035035426202; 0.06374735579337364; … ; 0.6231947636486973; -0.010398762143412258;;]), layer_4 = (weight = [-0.5261598414767211 2.3611111169548487 … 1.594284451982884 0.16928240026894317], bias = [1.631938442628271;;])), y = (layer_1 = (weight = [0.37774348982921235; 0.403490634194952; … ; 0.6122792318312761; -1.1025702494125322;;], bias = [0.1270583024663026; -0.012346469871814982; … ; 0.7226785030103654; -0.2446454847650411;;]), layer_2 = (weight = [0.20822480313592054 0.7424926008087538 … 0.5235122928120196 -0.5400163415560155; 0.4646753735404627 0.20689657393053892 … 0.2796368539607262 -0.6286055591027553; … ; 0.8719830378362975 -0.4113789464014356 … 0.6690494620899925 -0.6627250738903566; 0.22100359639001538 0.04565439077119202 … 0.8562236757268986 -0.8038410315018917], bias = [0.6068392998538864; -0.10807479195594707; … ; 0.6884628452622397; -0.029920772668541942;;]), layer_3 = (weight = [0.37998815148175014 0.33209764947217324 … 0.5910273326113317 -0.3511879028514545; -0.7104981143235486 -0.058230144188401924 … -0.5524071364423069 -0.433306555022958; … ; 0.6470390829457576 0.8495308122960852 … -0.6399327599769287 -0.6168705058749492; 0.42473170365307455 0.5832370971800808 … 0.5515927949329564 -0.5235810740364669], bias = [-0.13406091355729974; -0.4014956784803984; … ; 0.6907371589572177; 0.1116804611727228;;]), layer_4 = (weight = [-0.6454299924043357 -1.15885191327384 … 1.2731623382405974 0.16505116983878176], bias = [0.7074460010827709;;])))

Next, we'll compare our results with a reference solution to verify our computations.

using OrdinaryDiffEq
 
 function f(u, p, t)
     return [p[1] * u[1] - p[2] * u[1] * u[2], -p[3] * u[2] + p[4] * u[1] * u[2]]
@@ -55,7 +55,7 @@
 
 plot(vec(ts), vec(x_pred), label="x_pred")
 plot!(vec(ts), vec(y_pred), label="y_pred")
-plot!(true_sol)
Example block output

While the initial results are encouraging, we can further refine our model. By remaking the sampler, we can gradually transition to the uniform distribution for improved results.

# Adjusting the sampler to uniform distribution and re-solving the problem
+plot!(true_sol)
Example block output

While the initial results are encouraging, we can further refine our model. By remaking the sampler, we can gradually transition to the uniform distribution for improved results.

# Adjusting the sampler to uniform distribution and re-solving the problem
 for α in [0.6, 0.8, 1.0] # when α = 1.0, it is equivalent to uniform sampling
     sampler = remake(sampler; α=α)
     data = Sophon.sample(lotka_volterra, sampler)
@@ -66,4 +66,4 @@
 # Generating new predictions and calculating the absolute error
 x_pred = phi.x(ts, res.u.x)
 y_pred = phi.y(ts, res.u.y)
-maximum(sum(abs2, vcat(x_pred, y_pred) .- stack(true_sol.u); dims=1)) # print the absolute error
+maximum(sum(abs2, vcat(x_pred, y_pred) .- stack(true_sol.u); dims=1)) # print the absolute error diff --git a/dev/tutorials/phi.png b/dev/tutorials/phi.png index 892209d7..7885c293 100644 Binary files a/dev/tutorials/phi.png and b/dev/tutorials/phi.png differ diff --git a/dev/tutorials/poisson/index.html b/dev/tutorials/poisson/index.html index e100c8df..c3b2ff44 100644 --- a/dev/tutorials/poisson/index.html +++ b/dev/tutorials/poisson/index.html @@ -33,395 +33,413 @@ lines!(xs, vec(us); label="Prediction") axislegend(axis) fig

0.0%┣                                         ┫ 1/2.0k [00:21<Inf:Inf, InfGs/it]
-
3.406639e+06 0.1%┣                             ┫ 2/2.0k [00:21<11:45:55, 21s/it]
-
3.036452e+06 0.1%┣                             ┫ 3/2.0k [00:21<05:54:25, 11s/it]
-
2.854198e+06 0.2%┣                              ┫ 4/2.0k [00:21<03:56:53, 7s/it]
-
2.740487e+06 0.2%┣                              ┫ 5/2.0k [00:21<02:58:33, 5s/it]
-
2.562201e+06 0.3%┣                              ┫ 6/2.0k [00:22<02:23:14, 4s/it]
-
2.484190e+06 0.3%┣                              ┫ 7/2.0k [00:22<01:59:48, 4s/it]
-
2.318024e+06 0.4%┣▏                             ┫ 8/2.0k [00:22<01:42:57, 3s/it]
-
2.185029e+06 0.4%┣▏                             ┫ 9/2.0k [00:22<01:30:21, 3s/it]
-
2.039535e+06 0.5%┣▏                            ┫ 10/2.0k [00:22<01:20:33, 2s/it]
-
1.924296e+06 0.5%┣▏                            ┫ 11/2.0k [00:22<01:12:44, 2s/it]
-
1.804181e+06 0.6%┣▏                            ┫ 12/2.0k [00:22<01:06:18, 2s/it]
-
1.666347e+06 0.6%┣▏                            ┫ 13/2.0k [00:22<01:00:55, 2s/it]
-
1.627486e+06 0.7%┣▎                               ┫ 14/2.0k [00:22<56:24, 2s/it]
-
1.483547e+06 0.7%┣▎                               ┫ 15/2.0k [00:22<52:32, 2s/it]
-
1.330257e+06 0.8%┣▎                               ┫ 16/2.0k [00:22<49:09, 1s/it]
-
1.199582e+06 0.8%┣▎                               ┫ 17/2.0k [00:22<46:12, 1s/it]
-
1.143482e+06 0.9%┣▎                               ┫ 18/2.0k [00:22<43:36, 1s/it]
-
1.068897e+06 0.9%┣▎                               ┫ 19/2.0k [00:22<41:17, 1s/it]
-
9.874733e+05 1.0%┣▎                               ┫ 20/2.0k [00:23<39:13, 1s/it]
-
9.092495e+05 1.0%┣▍                               ┫ 21/2.0k [00:23<37:21, 1s/it]
-
8.732264e+05 1.1%┣▍                               ┫ 22/2.0k [00:23<35:41, 1s/it]
-
7.494943e+05 1.1%┣▍                               ┫ 23/2.0k [00:23<34:09, 1s/it]
-
6.795709e+05 1.2%┣▍                               ┫ 24/2.0k [00:23<32:47, 1it/s]
-
5.931949e+05 1.2%┣▍                               ┫ 25/2.0k [00:23<31:30, 1it/s]
-
5.274718e+05 1.3%┣▍                               ┫ 26/2.0k [00:23<30:19, 1it/s]
-
4.728307e+05 1.3%┣▍                               ┫ 27/2.0k [00:23<29:13, 1it/s]
-
4.295213e+05 1.4%┣▌                               ┫ 28/2.0k [00:23<28:13, 1it/s]
-
3.907298e+05 1.4%┣▌                               ┫ 29/2.0k [00:23<27:16, 1it/s]
-
3.479329e+05 1.5%┣▌                               ┫ 30/2.0k [00:23<26:23, 1it/s]
-
3.044329e+05 1.5%┣▌                               ┫ 31/2.0k [00:23<25:35, 1it/s]
-
2.638748e+05 1.6%┣▌                               ┫ 32/2.0k [00:23<24:49, 1it/s]
-
2.270700e+05 1.6%┣▌                               ┫ 33/2.0k [00:24<24:06, 1it/s]
-
1.928778e+05 1.7%┣▌                               ┫ 34/2.0k [00:24<23:26, 1it/s]
-
1.550343e+05 1.7%┣▋                               ┫ 35/2.0k [00:24<22:47, 1it/s]
-
1.313305e+05 1.8%┣▋                               ┫ 36/2.0k [00:24<22:11, 1it/s]
-
1.078370e+05 1.8%┣▋                               ┫ 37/2.0k [00:24<21:37, 2it/s]
-
9.418585e+04 1.9%┣▋                               ┫ 38/2.0k [00:24<21:05, 2it/s]
-
8.899134e+04 1.9%┣▋                               ┫ 39/2.0k [00:24<20:34, 2it/s]
-
8.080013e+04 2.0%┣▋                               ┫ 40/2.0k [00:24<20:05, 2it/s]
-
7.790201e+04 2.0%┣▋                               ┫ 41/2.0k [00:24<19:37, 2it/s]
-
6.958695e+04 2.1%┣▊                               ┫ 43/2.0k [00:24<18:45, 2it/s]
-
6.578819e+04 2.2%┣▊                               ┫ 44/2.0k [00:24<18:22, 2it/s]
-
5.295837e+04 2.3%┣▊                               ┫ 46/2.0k [00:24<17:36, 2it/s]
-
4.799783e+04 2.3%┣▊                               ┫ 47/2.0k [00:24<17:15, 2it/s]
-
4.112851e+04 2.4%┣▊                               ┫ 48/2.0k [00:24<16:55, 2it/s]
-
3.522977e+04 2.4%┣▉                               ┫ 49/2.0k [00:24<16:36, 2it/s]
-
3.062234e+04 2.5%┣▉                               ┫ 50/2.0k [00:25<16:17, 2it/s]
-
2.653695e+04 2.6%┣▉                               ┫ 52/2.0k [00:25<15:42, 2it/s]
-
2.420867e+04 2.6%┣▉                               ┫ 53/2.0k [00:25<15:26, 2it/s]
-
1.995815e+04 2.7%┣▉                               ┫ 54/2.0k [00:25<15:11, 2it/s]
-
1.731285e+04 2.7%┣▉                               ┫ 55/2.0k [00:25<14:55, 2it/s]
-
1.442249e+04 2.8%┣█                               ┫ 56/2.0k [00:25<14:40, 2it/s]
-
1.377452e+04 2.8%┣█                               ┫ 57/2.0k [00:25<14:26, 2it/s]
-
1.256766e+04 2.9%┣█                               ┫ 58/2.0k [00:25<14:12, 2it/s]
-
1.141598e+04 2.9%┣█                               ┫ 59/2.0k [00:25<13:59, 2it/s]
-
9.861938e+03 3.0%┣█                               ┫ 60/2.0k [00:25<13:47, 2it/s]
-
7.369776e+03 3.1%┣█                               ┫ 62/2.0k [00:25<13:22, 2it/s]
-
7.157710e+03 3.1%┣█                               ┫ 63/2.0k [00:25<13:11, 2it/s]
-
6.727203e+03 3.2%┣█                               ┫ 64/2.0k [00:25<13:00, 2it/s]
-
6.048558e+03 3.3%┣█                               ┫ 66/2.0k [00:25<12:38, 3it/s]
-
5.083159e+03 3.4%┣█                               ┫ 68/2.0k [00:26<12:17, 3it/s]
-
4.347375e+03 3.5%┣█▏                              ┫ 70/2.0k [00:26<11:58, 3it/s]
-
3.867418e+03 3.6%┣█▏                              ┫ 72/2.0k [00:26<11:39, 3it/s]
-
3.384427e+03 3.7%┣█▏                              ┫ 74/2.0k [00:26<11:22, 3it/s]
-
2.991784e+03 3.8%┣█▏                              ┫ 76/2.0k [00:26<11:05, 3it/s]
-
2.734084e+03 3.9%┣█▎                              ┫ 78/2.0k [00:26<10:50, 3it/s]
-
2.350991e+03 4.0%┣█▎                              ┫ 80/2.0k [00:26<10:35, 3it/s]
-
2.207798e+03 4.1%┣█▎                              ┫ 82/2.0k [00:26<10:21, 3it/s]
-
1.985990e+03 4.2%┣█▍                              ┫ 84/2.0k [00:26<10:08, 3it/s]
-
1.817199e+03 4.3%┣█▍                              ┫ 86/2.0k [00:26<09:55, 3it/s]
-
1.719490e+03 4.4%┣█▍                              ┫ 88/2.0k [00:26<09:42, 3it/s]
-
1.498070e+03 4.5%┣█▍                              ┫ 90/2.0k [00:27<09:31, 3it/s]
-
1.317895e+03 4.6%┣█▌                              ┫ 92/2.0k [00:27<09:20, 3it/s]
-
1.173369e+03 4.7%┣█▌                              ┫ 94/2.0k [00:27<09:09, 3it/s]
-
1.089946e+03 4.7%┣█▌                              ┫ 95/2.0k [00:27<09:04, 4it/s]
-
9.397325e+02 4.8%┣█▌                              ┫ 97/2.0k [00:27<08:54, 4it/s]
-
8.359782e+02 4.9%┣█▋                              ┫ 99/2.0k [00:27<08:44, 4it/s]
-
7.545342e+02 5.0%┣█▋                             ┫ 101/2.0k [00:27<08:34, 4it/s]
-
6.866138e+02 5.1%┣█▋                             ┫ 103/2.0k [00:27<08:26, 4it/s]
-
6.303943e+02 5.2%┣█▋                             ┫ 105/2.0k [00:27<08:17, 4it/s]
-
5.899407e+02 5.3%┣█▋                             ┫ 107/2.0k [00:27<08:08, 4it/s]
-
5.370404e+02 5.4%┣█▊                             ┫ 109/2.0k [00:27<08:00, 4it/s]
-
5.007168e+02 5.5%┣█▊                             ┫ 110/2.0k [00:27<07:56, 4it/s]
-
4.358647e+02 5.6%┣█▊                             ┫ 112/2.0k [00:28<07:48, 4it/s]
-
4.057244e+02 5.6%┣█▊                             ┫ 113/2.0k [00:28<07:45, 4it/s]
-
3.622590e+02 5.7%┣█▉                             ┫ 115/2.0k [00:28<07:38, 4it/s]
-
3.261360e+02 5.8%┣█▉                             ┫ 117/2.0k [00:28<07:31, 4it/s]
-
2.758989e+02 5.9%┣█▉                             ┫ 119/2.0k [00:28<07:24, 4it/s]
-
2.444424e+02 6.0%┣█▉                             ┫ 121/2.0k [00:28<07:17, 4it/s]
-
2.223665e+02 6.1%┣██                             ┫ 123/2.0k [00:28<07:11, 4it/s]
-
1.945461e+02 6.2%┣██                             ┫ 125/2.0k [00:28<07:04, 4it/s]
-
1.783145e+02 6.3%┣██                             ┫ 127/2.0k [00:28<06:58, 4it/s]
-
1.602898e+02 6.4%┣██                             ┫ 129/2.0k [00:28<06:52, 5it/s]
-
1.446717e+02 6.5%┣██                             ┫ 131/2.0k [00:28<06:47, 5it/s]
-
1.337863e+02 6.6%┣██                             ┫ 133/2.0k [00:28<06:41, 5it/s]
-
1.230380e+02 6.7%┣██                             ┫ 135/2.0k [00:28<06:36, 5it/s]
-
1.190367e+02 6.8%┣██▏                            ┫ 137/2.0k [00:28<06:30, 5it/s]
-
1.096370e+02 6.9%┣██▏                            ┫ 139/2.0k [00:29<06:25, 5it/s]
-
9.957033e+01 7.0%┣██▏                            ┫ 141/2.0k [00:29<06:20, 5it/s]
-
9.511509e+01 7.1%┣██▏                            ┫ 143/2.0k [00:29<06:16, 5it/s]
-
9.103453e+01 7.2%┣██▎                            ┫ 145/2.0k [00:29<06:11, 5it/s]
-
8.587396e+01 7.3%┣██▎                            ┫ 147/2.0k [00:29<06:06, 5it/s]
-
8.184455e+01 7.4%┣██▎                            ┫ 149/2.0k [00:29<06:02, 5it/s]
-
7.757973e+01 7.5%┣██▍                            ┫ 151/2.0k [00:29<05:57, 5it/s]
-
7.293454e+01 7.6%┣██▍                            ┫ 153/2.0k [00:29<05:53, 5it/s]
-
6.921791e+01 7.7%┣██▍                            ┫ 155/2.0k [00:29<05:49, 5it/s]
-
6.442561e+01 7.8%┣██▍                            ┫ 157/2.0k [00:29<05:45, 5it/s]
-
6.080378e+01 7.9%┣██▌                            ┫ 159/2.0k [00:29<05:41, 5it/s]
-
5.780417e+01 8.0%┣██▌                            ┫ 161/2.0k [00:29<05:37, 5it/s]
-
5.461006e+01 8.1%┣██▌                            ┫ 163/2.0k [00:29<05:34, 6it/s]
-
5.274705e+01 8.2%┣██▋                            ┫ 165/2.0k [00:29<05:30, 6it/s]
-
5.056355e+01 8.3%┣██▋                            ┫ 167/2.0k [00:30<05:26, 6it/s]
-
4.843604e+01 8.4%┣██▋                            ┫ 169/2.0k [00:30<05:23, 6it/s]
-
4.591658e+01 8.5%┣██▋                            ┫ 171/2.0k [00:30<05:19, 6it/s]
-
4.376440e+01 8.6%┣██▊                            ┫ 173/2.0k [00:30<05:16, 6it/s]
-
4.218368e+01 8.7%┣██▊                            ┫ 175/2.0k [00:30<05:13, 6it/s]
-
4.069852e+01 8.8%┣██▊                            ┫ 177/2.0k [00:30<05:09, 6it/s]
-
3.928530e+01 8.9%┣██▊                            ┫ 179/2.0k [00:30<05:06, 6it/s]
-
3.733779e+01 9.0%┣██▉                            ┫ 181/2.0k [00:30<05:03, 6it/s]
-
3.534324e+01 9.1%┣██▉                            ┫ 183/2.0k [00:30<05:00, 6it/s]
-
3.318797e+01 9.2%┣██▉                            ┫ 185/2.0k [00:30<04:57, 6it/s]
-
3.119567e+01 9.3%┣███                            ┫ 187/2.0k [00:30<04:54, 6it/s]
-
2.980184e+01 9.4%┣███                            ┫ 189/2.0k [00:30<04:51, 6it/s]
-
2.787065e+01 9.5%┣███                            ┫ 191/2.0k [00:30<04:48, 6it/s]
-
2.603024e+01 9.6%┣███                            ┫ 193/2.0k [00:30<04:46, 6it/s]
-
2.460396e+01 9.7%┣███                            ┫ 195/2.0k [00:30<04:43, 6it/s]
-
2.382596e+01 9.8%┣███                            ┫ 197/2.0k [00:30<04:41, 6it/s]
-
2.246219e+01 9.9%┣███                            ┫ 199/2.0k [00:31<04:38, 6it/s]
-
2.155858e+01 10.0%┣███                           ┫ 201/2.0k [00:31<04:36, 7it/s]
-
2.022441e+01 10.1%┣███                           ┫ 203/2.0k [00:31<04:33, 7it/s]
-
1.868059e+01 10.3%┣███                           ┫ 206/2.0k [00:31<04:29, 7it/s]
-
1.766617e+01 10.4%┣███▏                          ┫ 208/2.0k [00:31<04:27, 7it/s]
-
1.695465e+01 10.5%┣███▏                          ┫ 210/2.0k [00:31<04:25, 7it/s]
-
1.639332e+01 10.6%┣███▏                          ┫ 212/2.0k [00:31<04:22, 7it/s]
-
1.569684e+01 10.7%┣███▎                          ┫ 215/2.0k [00:31<04:19, 7it/s]
-
1.478964e+01 10.8%┣███▎                          ┫ 217/2.0k [00:31<04:17, 7it/s]
-
1.379968e+01 10.9%┣███▎                          ┫ 219/2.0k [00:31<04:15, 7it/s]
-
1.318512e+01 11.0%┣███▎                          ┫ 221/2.0k [00:31<04:12, 7it/s]
-
1.207098e+01 11.1%┣███▍                          ┫ 223/2.0k [00:31<04:10, 7it/s]
-
1.090178e+01 11.2%┣███▍                          ┫ 225/2.0k [00:31<04:08, 7it/s]
-
1.009785e+01 11.3%┣███▍                          ┫ 227/2.0k [00:31<04:06, 7it/s]
-
9.228957e+00 11.4%┣███▍                          ┫ 229/2.0k [00:31<04:04, 7it/s]
-
8.827614e+00 11.5%┣███▌                          ┫ 231/2.0k [00:32<04:02, 7it/s]
-
8.306911e+00 11.6%┣███▌                          ┫ 233/2.0k [00:32<04:01, 7it/s]
-
7.689669e+00 11.7%┣███▌                          ┫ 235/2.0k [00:32<03:59, 7it/s]
-
7.115464e+00 11.8%┣███▌                          ┫ 237/2.0k [00:32<03:57, 7it/s]
-
6.487847e+00 12.0%┣███▋                          ┫ 240/2.0k [00:32<03:54, 8it/s]
-
6.125100e+00 12.1%┣███▋                          ┫ 242/2.0k [00:32<03:52, 8it/s]
-
5.615207e+00 12.2%┣███▊                          ┫ 245/2.0k [00:32<03:49, 8it/s]
-
5.124559e+00 12.4%┣███▊                          ┫ 248/2.0k [00:32<03:47, 8it/s]
-
4.769319e+00 12.5%┣███▊                          ┫ 250/2.0k [00:32<03:45, 8it/s]
-
4.476356e+00 12.6%┣███▉                          ┫ 253/2.0k [00:32<03:42, 8it/s]
-
4.134047e+00 12.8%┣███▉                          ┫ 256/2.0k [00:32<03:40, 8it/s]
-
3.831963e+00 12.9%┣███▉                          ┫ 258/2.0k [00:32<03:38, 8it/s]
-
3.675827e+00 13.0%┣████                          ┫ 260/2.0k [00:32<03:37, 8it/s]
-
3.301021e+00 13.1%┣████                          ┫ 263/2.0k [00:32<03:34, 8it/s]
-
3.098669e+00 13.3%┣████                          ┫ 266/2.0k [00:32<03:32, 8it/s]
-
2.907836e+00 13.4%┣████                          ┫ 269/2.0k [00:32<03:30, 8it/s]
-
2.745016e+00 13.6%┣████                          ┫ 272/2.0k [00:32<03:27, 8it/s]
-
2.627626e+00 13.7%┣████▏                         ┫ 275/2.0k [00:33<03:25, 8it/s]
-
2.460634e+00 13.9%┣████▏                         ┫ 278/2.0k [00:33<03:23, 8it/s]
-
2.302191e+00 14.0%┣████▏                         ┫ 281/2.0k [00:33<03:21, 9it/s]
-
2.189074e+00 14.2%┣████▎                         ┫ 284/2.0k [00:33<03:19, 9it/s]
-
2.034145e+00 14.3%┣████▎                         ┫ 287/2.0k [00:33<03:17, 9it/s]
-
1.882948e+00 14.5%┣████▍                         ┫ 290/2.0k [00:33<03:15, 9it/s]
-
1.729352e+00 14.6%┣████▍                         ┫ 293/2.0k [00:33<03:13, 9it/s]
-
1.620326e+00 14.8%┣████▍                         ┫ 296/2.0k [00:33<03:11, 9it/s]
-
1.518054e+00 15.0%┣████▌                         ┫ 300/2.0k [00:33<03:08, 9it/s]
-
1.429054e+00 15.1%┣████▌                         ┫ 303/2.0k [00:33<03:06, 9it/s]
-
1.351820e+00 15.3%┣████▋                         ┫ 306/2.0k [00:33<03:04, 9it/s]
-
1.254449e+00 15.5%┣████▋                         ┫ 310/2.0k [00:33<03:02, 9it/s]
-
1.136138e+00 15.7%┣████▊                         ┫ 314/2.0k [00:33<02:59, 9it/s]
-
9.964395e-01 15.9%┣████▋                        ┫ 318/2.0k [00:33<02:57, 10it/s]
-
9.103730e-01 16.0%┣████▋                        ┫ 321/2.0k [00:33<02:55, 10it/s]
-
8.401602e-01 16.2%┣████▊                        ┫ 325/2.0k [00:33<02:53, 10it/s]
-
7.749811e-01 16.4%┣████▊                        ┫ 329/2.0k [00:34<02:51, 10it/s]
-
7.109908e-01 16.6%┣████▉                        ┫ 332/2.0k [00:34<02:49, 10it/s]
-
6.795215e-01 16.7%┣████▉                        ┫ 335/2.0k [00:34<02:48, 10it/s]
-
6.366900e-01 16.9%┣█████                        ┫ 339/2.0k [00:34<02:46, 10it/s]
-
5.755769e-01 17.1%┣█████                        ┫ 343/2.0k [00:34<02:43, 10it/s]
-
5.360708e-01 17.3%┣█████                        ┫ 347/2.0k [00:34<02:41, 10it/s]
-
4.977968e-01 17.5%┣█████                        ┫ 351/2.0k [00:34<02:40, 10it/s]
-
4.597401e-01 17.8%┣█████▏                       ┫ 356/2.0k [00:34<02:37, 10it/s]
-
4.312233e-01 18.1%┣█████▎                       ┫ 362/2.0k [00:34<02:34, 11it/s]
-
4.056026e-01 18.4%┣█████▍                       ┫ 368/2.0k [00:34<02:31, 11it/s]
-
3.833184e-01 18.7%┣█████▍                       ┫ 375/2.0k [00:34<02:28, 11it/s]
-
3.624541e-01 19.1%┣█████▌                       ┫ 382/2.0k [00:34<02:25, 11it/s]
-
3.412792e-01 19.4%┣█████▋                       ┫ 389/2.0k [00:34<02:22, 11it/s]
-
3.240653e-01 19.8%┣█████▊                       ┫ 396/2.0k [00:34<02:19, 12it/s]
-
3.029935e-01 20.1%┣█████▉                       ┫ 403/2.0k [00:34<02:16, 12it/s]
-
2.871124e-01 20.5%┣██████                       ┫ 410/2.0k [00:34<02:14, 12it/s]
-
2.717028e-01 20.7%┣██████                       ┫ 415/2.0k [00:34<02:12, 12it/s]
-
2.565346e-01 21.0%┣██████                       ┫ 421/2.0k [00:34<02:10, 12it/s]
-
2.440492e-01 21.3%┣██████▏                      ┫ 426/2.0k [00:35<02:08, 12it/s]
-
2.297783e-01 21.6%┣██████▎                      ┫ 433/2.0k [00:35<02:05, 12it/s]
-
2.206485e-01 21.9%┣██████▍                      ┫ 438/2.0k [00:35<02:04, 13it/s]
-
2.103868e-01 22.2%┣██████▍                      ┫ 444/2.0k [00:35<02:02, 13it/s]
-
2.007489e-01 22.4%┣██████▌                      ┫ 449/2.0k [00:35<02:00, 13it/s]
-
1.892148e-01 22.8%┣██████▋                      ┫ 456/2.0k [00:35<01:58, 13it/s]
-
1.815711e-01 23.1%┣██████▊                      ┫ 462/2.0k [00:35<01:56, 13it/s]
-
1.725767e-01 23.5%┣██████▉                      ┫ 470/2.0k [00:35<01:54, 13it/s]
-
1.675811e-01 23.8%┣███████                      ┫ 477/2.0k [00:35<01:52, 14it/s]
-
1.612233e-01 24.2%┣███████                      ┫ 485/2.0k [00:35<01:50, 14it/s]
-
1.540435e-01 24.6%┣███████▏                     ┫ 492/2.0k [00:35<01:48, 14it/s]
-
1.468701e-01 24.9%┣███████▎                     ┫ 499/2.0k [00:35<01:46, 14it/s]
-
1.413845e-01 25.3%┣███████▍                     ┫ 506/2.0k [00:35<01:44, 14it/s]
-
1.354925e-01 25.6%┣███████▍                     ┫ 513/2.0k [00:35<01:42, 15it/s]
-
1.296055e-01 26.0%┣███████▌                     ┫ 520/2.0k [00:35<01:41, 15it/s]
-
1.243710e-01 26.3%┣███████▋                     ┫ 527/2.0k [00:35<01:39, 15it/s]
-
1.192630e-01 26.7%┣███████▊                     ┫ 534/2.0k [00:35<01:37, 15it/s]
-
1.131353e-01 27.0%┣███████▉                     ┫ 541/2.0k [00:35<01:36, 15it/s]
-
1.071741e-01 27.4%┣████████                     ┫ 548/2.0k [00:36<01:34, 15it/s]
-
1.019045e-01 27.7%┣████████                     ┫ 555/2.0k [00:36<01:33, 16it/s]
-
9.691297e-02 28.1%┣████████▏                    ┫ 562/2.0k [00:36<01:31, 16it/s]
-
9.320312e-02 28.4%┣████████▎                    ┫ 569/2.0k [00:36<01:30, 16it/s]
-
8.933106e-02 28.8%┣████████▍                    ┫ 576/2.0k [00:36<01:29, 16it/s]
-
8.673317e-02 29.1%┣████████▍                    ┫ 582/2.0k [00:36<01:27, 16it/s]
-
8.395447e-02 29.4%┣████████▌                    ┫ 588/2.0k [00:36<01:26, 16it/s]
-
8.042847e-02 29.7%┣████████▋                    ┫ 595/2.0k [00:36<01:25, 17it/s]
-
7.740242e-02 30.0%┣████████▊                    ┫ 601/2.0k [00:36<01:24, 17it/s]
-
7.401165e-02 30.4%┣████████▉                    ┫ 609/2.0k [00:36<01:22, 17it/s]
-
7.131909e-02 30.7%┣█████████                    ┫ 615/2.0k [00:36<01:21, 17it/s]
-
6.720897e-02 31.1%┣█████████                    ┫ 623/2.0k [00:36<01:20, 17it/s]
-
6.410320e-02 31.4%┣█████████▏                   ┫ 629/2.0k [00:36<01:19, 17it/s]
-
6.033065e-02 31.8%┣█████████▎                   ┫ 637/2.0k [00:36<01:18, 18it/s]
-
5.781131e-02 32.1%┣█████████▎                   ┫ 643/2.0k [00:36<01:17, 18it/s]
-
5.398705e-02 32.5%┣█████████▍                   ┫ 651/2.0k [00:36<01:15, 18it/s]
-
5.120860e-02 32.8%┣█████████▌                   ┫ 657/2.0k [00:36<01:15, 18it/s]
-
4.776453e-02 33.2%┣█████████▋                   ┫ 665/2.0k [00:36<01:13, 18it/s]
-
4.590200e-02 33.5%┣█████████▊                   ┫ 671/2.0k [00:36<01:12, 18it/s]
-
4.366912e-02 33.9%┣█████████▉                   ┫ 679/2.0k [00:37<01:11, 19it/s]
-
4.226397e-02 34.2%┣██████████                   ┫ 685/2.0k [00:37<01:10, 19it/s]
-
3.990425e-02 34.6%┣██████████                   ┫ 693/2.0k [00:37<01:09, 19it/s]
-
3.790969e-02 34.9%┣██████████▏                  ┫ 699/2.0k [00:37<01:08, 19it/s]
-
3.557146e-02 35.3%┣██████████▎                  ┫ 707/2.0k [00:37<01:07, 19it/s]
-
3.364612e-02 35.6%┣██████████▍                  ┫ 713/2.0k [00:37<01:07, 19it/s]
-
3.181700e-02 36.0%┣██████████▍                  ┫ 720/2.0k [00:37<01:06, 20it/s]
-
3.035299e-02 36.3%┣██████████▌                  ┫ 727/2.0k [00:37<01:05, 20it/s]
-
2.837953e-02 36.7%┣██████████▋                  ┫ 735/2.0k [00:37<01:04, 20it/s]
-
2.676241e-02 37.1%┣██████████▊                  ┫ 742/2.0k [00:37<01:03, 20it/s]
-
2.453278e-02 37.5%┣██████████▉                  ┫ 750/2.0k [00:37<01:02, 20it/s]
-
2.324475e-02 37.8%┣███████████                  ┫ 757/2.0k [00:37<01:01, 20it/s]
-
2.201730e-02 38.2%┣███████████                  ┫ 764/2.0k [00:37<01:00, 21it/s]
-
2.078726e-02 38.5%┣███████████▏                 ┫ 771/2.0k [00:37<01:00, 21it/s]
-
1.945705e-02 38.9%┣███████████▎                 ┫ 779/2.0k [00:37<00:59, 21it/s]
-
1.859215e-02 39.3%┣███████████▍                 ┫ 786/2.0k [00:37<00:58, 21it/s]
-
1.739912e-02 39.6%┣███████████▌                 ┫ 793/2.0k [00:37<00:57, 21it/s]
-
1.637463e-02 40.0%┣███████████▋                 ┫ 800/2.0k [00:37<00:56, 21it/s]
-
1.568812e-02 40.3%┣███████████▊                 ┫ 807/2.0k [00:38<00:56, 21it/s]
-
1.485332e-02 40.7%┣███████████▉                 ┫ 814/2.0k [00:38<00:55, 22it/s]
-
1.388904e-02 41.0%┣████████████                 ┫ 821/2.0k [00:38<00:54, 22it/s]
-
1.317465e-02 41.4%┣████████████                 ┫ 828/2.0k [00:38<00:53, 22it/s]
-
1.240421e-02 41.7%┣████████████                 ┫ 835/2.0k [00:38<00:53, 22it/s]
-
1.172464e-02 42.1%┣████████████▏                ┫ 842/2.0k [00:38<00:52, 22it/s]
-
1.108648e-02 42.4%┣████████████▎                ┫ 849/2.0k [00:38<00:51, 22it/s]
-
1.058554e-02 42.7%┣████████████▍                ┫ 855/2.0k [00:38<00:51, 23it/s]
-
9.935804e-03 43.1%┣████████████▌                ┫ 862/2.0k [00:38<00:50, 23it/s]
-
9.388285e-03 43.4%┣████████████▋                ┫ 869/2.0k [00:38<00:50, 23it/s]
-
8.922221e-03 43.8%┣████████████▊                ┫ 876/2.0k [00:38<00:49, 23it/s]
-
8.518671e-03 44.1%┣████████████▉                ┫ 883/2.0k [00:38<00:48, 23it/s]
-
8.087601e-03 44.5%┣█████████████                ┫ 891/2.0k [00:38<00:48, 23it/s]
-
7.663643e-03 44.9%┣█████████████                ┫ 898/2.0k [00:38<00:47, 23it/s]
-
7.358870e-03 45.2%┣█████████████▏               ┫ 905/2.0k [00:38<00:46, 24it/s]
-
6.983850e-03 45.6%┣█████████████▏               ┫ 912/2.0k [00:38<00:46, 24it/s]
-
6.720359e-03 46.0%┣█████████████▍               ┫ 920/2.0k [00:38<00:45, 24it/s]
-
6.382729e-03 46.3%┣█████████████▍               ┫ 927/2.0k [00:38<00:45, 24it/s]
-
6.070335e-03 46.7%┣█████████████▌               ┫ 934/2.0k [00:39<00:44, 24it/s]
-
5.728778e-03 47.1%┣█████████████▋               ┫ 942/2.0k [00:39<00:43, 24it/s]
-
5.438535e-03 47.4%┣█████████████▊               ┫ 949/2.0k [00:39<00:43, 25it/s]
-
5.217413e-03 47.9%┣█████████████▉               ┫ 958/2.0k [00:39<00:42, 25it/s]
-
4.948238e-03 48.3%┣██████████████               ┫ 966/2.0k [00:39<00:42, 25it/s]
-
4.640650e-03 48.7%┣██████████████▏              ┫ 975/2.0k [00:39<00:41, 25it/s]
-
4.299279e-03 49.1%┣██████████████▎              ┫ 982/2.0k [00:39<00:40, 25it/s]
-
4.058092e-03 49.5%┣██████████████▍              ┫ 991/2.0k [00:39<00:40, 25it/s]
-
3.925460e-03 49.8%┣██████████████▌              ┫ 997/2.0k [00:39<00:39, 26it/s]
-
3.813010e-03 50.3%┣██████████████              ┫ 1.0k/2.0k [00:39<00:39, 26it/s]
-
3.705965e-03 50.7%┣██████████████▏             ┫ 1.0k/2.0k [00:39<00:38, 26it/s]
-
3.563598e-03 51.2%┣██████████████▍             ┫ 1.0k/2.0k [00:39<00:37, 26it/s]
-
3.424059e-03 51.6%┣██████████████▌             ┫ 1.0k/2.0k [00:39<00:37, 26it/s]
-
3.231981e-03 52.1%┣██████████████▋             ┫ 1.0k/2.0k [00:39<00:36, 27it/s]
-
3.048180e-03 52.5%┣██████████████▊             ┫ 1.1k/2.0k [00:39<00:36, 27it/s]
-
2.890340e-03 53.0%┣██████████████▉             ┫ 1.1k/2.0k [00:39<00:35, 27it/s]
-
2.791467e-03 53.4%┣███████████████             ┫ 1.1k/2.0k [00:39<00:34, 27it/s]
-
2.674536e-03 53.8%┣███████████████             ┫ 1.1k/2.0k [00:39<00:34, 27it/s]
-
2.592556e-03 54.2%┣███████████████▏            ┫ 1.1k/2.0k [00:39<00:33, 27it/s]
-
2.447324e-03 54.7%┣███████████████▎            ┫ 1.1k/2.0k [00:40<00:33, 28it/s]
-
2.339866e-03 55.1%┣███████████████▍            ┫ 1.1k/2.0k [00:40<00:32, 28it/s]
-
2.255981e-03 55.6%┣███████████████▋            ┫ 1.1k/2.0k [00:40<00:32, 28it/s]
-
2.188002e-03 56.0%┣███████████████▊            ┫ 1.1k/2.0k [00:40<00:31, 28it/s]
-
2.084544e-03 56.5%┣███████████████▉            ┫ 1.1k/2.0k [00:40<00:31, 28it/s]
-
2.025780e-03 57.0%┣████████████████            ┫ 1.1k/2.0k [00:40<00:30, 29it/s]
-
1.988949e-03 57.4%┣████████████████            ┫ 1.1k/2.0k [00:40<00:30, 29it/s]
-
1.938739e-03 57.9%┣████████████████▏           ┫ 1.2k/2.0k [00:40<00:29, 29it/s]
-
1.878840e-03 58.3%┣████████████████▎           ┫ 1.2k/2.0k [00:40<00:29, 29it/s]
-
1.817030e-03 58.7%┣████████████████▍           ┫ 1.2k/2.0k [00:40<00:28, 29it/s]
-
1.750998e-03 59.0%┣████████████████▌           ┫ 1.2k/2.0k [00:40<00:28, 29it/s]
-
1.645733e-03 59.5%┣████████████████▋           ┫ 1.2k/2.0k [00:40<00:27, 30it/s]
-
1.586866e-03 59.9%┣████████████████▊           ┫ 1.2k/2.0k [00:40<00:27, 30it/s]
-
1.509385e-03 60.4%┣█████████████████           ┫ 1.2k/2.0k [00:40<00:26, 30it/s]
-
1.426615e-03 60.8%┣█████████████████           ┫ 1.2k/2.0k [00:40<00:26, 30it/s]
-
1.364026e-03 61.3%┣█████████████████▏          ┫ 1.2k/2.0k [00:40<00:25, 30it/s]
-
1.292138e-03 61.7%┣█████████████████▎          ┫ 1.2k/2.0k [00:40<00:25, 31it/s]
-
1.235253e-03 62.2%┣█████████████████▍          ┫ 1.2k/2.0k [00:40<00:25, 31it/s]
-
1.187918e-03 62.7%┣█████████████████▌          ┫ 1.3k/2.0k [00:40<00:24, 31it/s]
-
1.146915e-03 63.2%┣█████████████████▊          ┫ 1.3k/2.0k [00:41<00:24, 31it/s]
-
1.117281e-03 63.6%┣█████████████████▉          ┫ 1.3k/2.0k [00:41<00:23, 31it/s]
-
1.077413e-03 64.1%┣██████████████████          ┫ 1.3k/2.0k [00:41<00:23, 32it/s]
-
1.032842e-03 64.6%┣██████████████████          ┫ 1.3k/2.0k [00:41<00:22, 32it/s]
-
9.832105e-04 65.0%┣██████████████████▏         ┫ 1.3k/2.0k [00:41<00:22, 32it/s]
-
9.399564e-04 65.5%┣██████████████████▍         ┫ 1.3k/2.0k [00:41<00:21, 32it/s]
-
9.009709e-04 65.9%┣██████████████████▌         ┫ 1.3k/2.0k [00:41<00:21, 32it/s]
-
8.511425e-04 66.4%┣██████████████████▋         ┫ 1.3k/2.0k [00:41<00:21, 32it/s]
-
8.091717e-04 66.8%┣██████████████████▊         ┫ 1.3k/2.0k [00:41<00:20, 33it/s]
-
7.686166e-04 67.3%┣██████████████████▉         ┫ 1.3k/2.0k [00:41<00:20, 33it/s]
-
7.523773e-04 67.6%┣███████████████████         ┫ 1.4k/2.0k [00:41<00:20, 33it/s]
-
7.331933e-04 68.1%┣███████████████████         ┫ 1.4k/2.0k [00:41<00:19, 33it/s]
-
7.210707e-04 68.5%┣███████████████████▏        ┫ 1.4k/2.0k [00:41<00:19, 33it/s]
-
6.952058e-04 69.0%┣███████████████████▎        ┫ 1.4k/2.0k [00:41<00:19, 33it/s]
-
6.649946e-04 69.4%┣███████████████████▍        ┫ 1.4k/2.0k [00:41<00:18, 34it/s]
-
6.390293e-04 69.9%┣███████████████████▋        ┫ 1.4k/2.0k [00:41<00:18, 34it/s]
-
6.193768e-04 70.3%┣███████████████████▊        ┫ 1.4k/2.0k [00:41<00:17, 34it/s]
-
5.939128e-04 70.8%┣███████████████████▉        ┫ 1.4k/2.0k [00:41<00:17, 34it/s]
-
5.647707e-04 71.3%┣████████████████████        ┫ 1.4k/2.0k [00:41<00:17, 34it/s]
-
5.457648e-04 71.8%┣████████████████████        ┫ 1.4k/2.0k [00:42<00:16, 35it/s]
-
5.222914e-04 72.2%┣████████████████████▏       ┫ 1.4k/2.0k [00:42<00:16, 35it/s]
-
4.903480e-04 72.7%┣████████████████████▍       ┫ 1.5k/2.0k [00:42<00:16, 35it/s]
-
4.700393e-04 73.2%┣████████████████████▌       ┫ 1.5k/2.0k [00:42<00:15, 35it/s]
-
4.550022e-04 73.6%┣████████████████████▋       ┫ 1.5k/2.0k [00:42<00:15, 35it/s]
-
4.310892e-04 74.1%┣████████████████████▊       ┫ 1.5k/2.0k [00:42<00:15, 35it/s]
-
4.121129e-04 74.6%┣████████████████████▉       ┫ 1.5k/2.0k [00:42<00:14, 36it/s]
-
3.974393e-04 75.1%┣█████████████████████       ┫ 1.5k/2.0k [00:42<00:14, 36it/s]
-
3.848921e-04 75.5%┣█████████████████████▏      ┫ 1.5k/2.0k [00:42<00:14, 36it/s]
-
3.599760e-04 76.0%┣█████████████████████▎      ┫ 1.5k/2.0k [00:42<00:13, 36it/s]
-
3.368368e-04 76.4%┣█████████████████████▍      ┫ 1.5k/2.0k [00:42<00:13, 36it/s]
-
3.194032e-04 76.9%┣█████████████████████▌      ┫ 1.5k/2.0k [00:42<00:13, 36it/s]
-
2.988794e-04 77.3%┣█████████████████████▋      ┫ 1.5k/2.0k [00:42<00:12, 37it/s]
-
2.664378e-04 77.8%┣█████████████████████▊      ┫ 1.6k/2.0k [00:42<00:12, 37it/s]
-
2.578191e-04 78.2%┣█████████████████████▉      ┫ 1.6k/2.0k [00:42<00:12, 37it/s]
-
2.482563e-04 78.7%┣██████████████████████      ┫ 1.6k/2.0k [00:42<00:11, 37it/s]
-
2.375229e-04 79.1%┣██████████████████████▏     ┫ 1.6k/2.0k [00:42<00:11, 37it/s]
-
2.262543e-04 79.6%┣██████████████████████▎     ┫ 1.6k/2.0k [00:42<00:11, 37it/s]
-
2.184065e-04 80.0%┣██████████████████████▍     ┫ 1.6k/2.0k [00:42<00:11, 38it/s]
-
2.115027e-04 80.5%┣██████████████████████▌     ┫ 1.6k/2.0k [00:43<00:10, 38it/s]
-
2.033767e-04 81.0%┣██████████████████████▊     ┫ 1.6k/2.0k [00:43<00:10, 38it/s]
-
1.934361e-04 81.4%┣██████████████████████▉     ┫ 1.6k/2.0k [00:43<00:10, 38it/s]
-
1.854356e-04 81.9%┣███████████████████████     ┫ 1.6k/2.0k [00:43<00:09, 38it/s]
-
1.722868e-04 82.4%┣███████████████████████     ┫ 1.6k/2.0k [00:43<00:09, 39it/s]
-
1.574661e-04 82.9%┣███████████████████████▏    ┫ 1.7k/2.0k [00:43<00:09, 39it/s]
-
1.492840e-04 83.3%┣███████████████████████▎    ┫ 1.7k/2.0k [00:43<00:09, 39it/s]
-
1.383051e-04 83.8%┣███████████████████████▌    ┫ 1.7k/2.0k [00:43<00:08, 39it/s]
-
1.336091e-04 84.2%┣███████████████████████▋    ┫ 1.7k/2.0k [00:43<00:08, 39it/s]
-
1.279232e-04 84.7%┣███████████████████████▊    ┫ 1.7k/2.0k [00:43<00:08, 39it/s]
-
1.246881e-04 85.2%┣███████████████████████▉    ┫ 1.7k/2.0k [00:43<00:08, 40it/s]
-
1.220748e-04 85.7%┣████████████████████████    ┫ 1.7k/2.0k [00:43<00:07, 40it/s]
-
1.212185e-04 86.0%┣████████████████████████    ┫ 1.7k/2.0k [00:43<00:07, 40it/s]
-
1.196990e-04 86.5%┣████████████████████████▏   ┫ 1.7k/2.0k [00:43<00:07, 40it/s]
-
1.189803e-04 86.9%┣████████████████████████▎   ┫ 1.7k/2.0k [00:43<00:07, 40it/s]
-
1.169607e-04 87.4%┣████████████████████████▌   ┫ 1.7k/2.0k [00:43<00:06, 40it/s]
-
1.151376e-04 87.8%┣████████████████████████▋   ┫ 1.8k/2.0k [00:43<00:06, 40it/s]
-
1.123685e-04 88.3%┣████████████████████████▊   ┫ 1.8k/2.0k [00:43<00:06, 41it/s]
-
1.092620e-04 88.8%┣████████████████████████▉   ┫ 1.8k/2.0k [00:43<00:06, 41it/s]
-
1.068198e-04 89.2%┣█████████████████████████   ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
-
1.043200e-04 89.7%┣█████████████████████████▏  ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
-
1.002397e-04 90.2%┣█████████████████████████▎  ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
-
9.583209e-05 90.7%┣█████████████████████████▍  ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
-
9.502690e-05 91.1%┣█████████████████████████▌  ┫ 1.8k/2.0k [00:44<00:04, 42it/s]
-
9.292847e-05 91.6%┣█████████████████████████▋  ┫ 1.8k/2.0k [00:44<00:04, 42it/s]
-
9.112773e-05 92.0%┣█████████████████████████▊  ┫ 1.8k/2.0k [00:44<00:04, 42it/s]
-
8.822537e-05 92.5%┣█████████████████████████▉  ┫ 1.9k/2.0k [00:44<00:04, 42it/s]
-
8.743475e-05 92.9%┣██████████████████████████  ┫ 1.9k/2.0k [00:44<00:03, 42it/s]
-
8.665028e-05 93.4%┣██████████████████████████▏ ┫ 1.9k/2.0k [00:44<00:03, 42it/s]
-
8.532758e-05 93.8%┣██████████████████████████▎ ┫ 1.9k/2.0k [00:44<00:03, 43it/s]
-
8.137279e-05 94.3%┣██████████████████████████▍ ┫ 1.9k/2.0k [00:44<00:03, 43it/s]
-
7.992484e-05 94.7%┣██████████████████████████▌ ┫ 1.9k/2.0k [00:44<00:02, 43it/s]
-
7.875641e-05 95.1%┣██████████████████████████▋ ┫ 1.9k/2.0k [00:44<00:02, 43it/s]
-
7.625438e-05 95.6%┣██████████████████████████▊ ┫ 1.9k/2.0k [00:44<00:02, 43it/s]
-
7.423706e-05 96.1%┣███████████████████████████ ┫ 1.9k/2.0k [00:44<00:02, 43it/s]
-
7.234973e-05 96.5%┣███████████████████████████ ┫ 1.9k/2.0k [00:44<00:02, 43it/s]
-
7.096104e-05 97.0%┣███████████████████████████▏┫ 1.9k/2.0k [00:44<00:01, 44it/s]
-
7.000037e-05 97.4%┣███████████████████████████▎┫ 1.9k/2.0k [00:45<00:01, 44it/s]
-
6.915890e-05 97.9%┣███████████████████████████▍┫ 2.0k/2.0k [00:45<00:01, 44it/s]
-
6.770198e-05 98.4%┣███████████████████████████▌┫ 2.0k/2.0k [00:45<00:01, 44it/s]
-
6.670864e-05 98.8%┣███████████████████████████▋┫ 2.0k/2.0k [00:45<00:01, 44it/s]
-
6.609586e-05 99.3%┣███████████████████████████▉┫ 2.0k/2.0k [00:45<00:00, 44it/s]
-
6.477584e-05 99.7%┣████████████████████████████┫ 2.0k/2.0k [00:45<00:00, 45it/s]
-
6.423756e-05 100.0%┣███████████████████████████┫ 2.0k/2.0k [00:45<00:00, 45it/s]

Compute the relative L2 error

using Integrals
+
3.449626e+06 0.1%┣                             ┫ 2/2.0k [00:21<11:34:19, 21s/it]
+
3.210697e+06 0.1%┣                             ┫ 3/2.0k [00:21<05:48:34, 10s/it]
+
3.022042e+06 0.2%┣                              ┫ 4/2.0k [00:21<03:53:12, 7s/it]
+
2.847703e+06 0.2%┣                              ┫ 5/2.0k [00:21<02:55:30, 5s/it]
+
2.737138e+06 0.3%┣                              ┫ 6/2.0k [00:21<02:20:50, 4s/it]
+
2.617447e+06 0.3%┣                              ┫ 7/2.0k [00:21<01:57:42, 4s/it]
+
2.528132e+06 0.4%┣▏                             ┫ 8/2.0k [00:21<01:41:07, 3s/it]
+
2.462512e+06 0.4%┣▏                             ┫ 9/2.0k [00:21<01:28:51, 3s/it]
+
2.169471e+06 0.5%┣▏                            ┫ 10/2.0k [00:21<01:19:15, 2s/it]
+
1.936574e+06 0.5%┣▏                            ┫ 11/2.0k [00:22<01:11:36, 2s/it]
+
1.800832e+06 0.6%┣▏                            ┫ 12/2.0k [00:22<01:05:19, 2s/it]
+
1.687308e+06 0.6%┣▏                            ┫ 13/2.0k [00:22<01:00:04, 2s/it]
+
1.621351e+06 0.7%┣▎                               ┫ 14/2.0k [00:22<55:36, 2s/it]
+
1.496447e+06 0.7%┣▎                               ┫ 15/2.0k [00:22<51:46, 2s/it]
+
1.396776e+06 0.8%┣▎                               ┫ 16/2.0k [00:22<48:30, 1s/it]
+
1.299268e+06 0.8%┣▎                               ┫ 17/2.0k [00:22<45:36, 1s/it]
+
1.201218e+06 0.9%┣▎                               ┫ 18/2.0k [00:22<43:02, 1s/it]
+
1.132898e+06 0.9%┣▎                               ┫ 19/2.0k [00:22<40:45, 1s/it]
+
1.076207e+06 1.0%┣▎                               ┫ 20/2.0k [00:22<38:43, 1s/it]
+
9.933535e+05 1.0%┣▍                               ┫ 21/2.0k [00:22<36:53, 1s/it]
+
9.346051e+05 1.1%┣▍                               ┫ 22/2.0k [00:22<35:13, 1s/it]
+
8.838396e+05 1.1%┣▍                               ┫ 23/2.0k [00:22<33:42, 1s/it]
+
8.176727e+05 1.2%┣▍                               ┫ 24/2.0k [00:23<32:19, 1it/s]
+
7.778934e+05 1.2%┣▍                               ┫ 25/2.0k [00:23<31:03, 1it/s]
+
7.486485e+05 1.3%┣▍                               ┫ 26/2.0k [00:23<29:53, 1it/s]
+
7.029999e+05 1.3%┣▍                               ┫ 27/2.0k [00:23<28:47, 1it/s]
+
6.900467e+05 1.4%┣▌                               ┫ 28/2.0k [00:23<27:47, 1it/s]
+
6.650323e+05 1.4%┣▌                               ┫ 29/2.0k [00:23<26:52, 1it/s]
+
5.682066e+05 1.5%┣▌                               ┫ 30/2.0k [00:23<26:00, 1it/s]
+
4.837313e+05 1.5%┣▌                               ┫ 31/2.0k [00:23<25:13, 1it/s]
+
3.701621e+05 1.6%┣▌                               ┫ 32/2.0k [00:23<24:28, 1it/s]
+
2.713662e+05 1.6%┣▌                               ┫ 33/2.0k [00:23<23:45, 1it/s]
+
2.589274e+05 1.7%┣▌                               ┫ 34/2.0k [00:23<23:05, 1it/s]
+
2.223272e+05 1.7%┣▋                               ┫ 35/2.0k [00:23<22:27, 1it/s]
+
1.878034e+05 1.8%┣▋                               ┫ 36/2.0k [00:23<21:51, 1it/s]
+
1.658949e+05 1.8%┣▋                               ┫ 37/2.0k [00:23<21:17, 2it/s]
+
1.447550e+05 1.9%┣▋                               ┫ 38/2.0k [00:23<20:45, 2it/s]
+
1.196316e+05 1.9%┣▋                               ┫ 39/2.0k [00:24<20:16, 2it/s]
+
9.833521e+04 2.0%┣▋                               ┫ 40/2.0k [00:24<19:46, 2it/s]
+
8.461754e+04 2.0%┣▋                               ┫ 41/2.0k [00:24<19:19, 2it/s]
+
7.346661e+04 2.1%┣▊                               ┫ 42/2.0k [00:24<18:54, 2it/s]
+
6.348979e+04 2.1%┣▊                               ┫ 43/2.0k [00:24<18:29, 2it/s]
+
5.312643e+04 2.2%┣▊                               ┫ 44/2.0k [00:24<18:05, 2it/s]
+
4.454665e+04 2.2%┣▊                               ┫ 45/2.0k [00:24<17:43, 2it/s]
+
3.951608e+04 2.3%┣▊                               ┫ 46/2.0k [00:24<17:21, 2it/s]
+
3.339099e+04 2.3%┣▊                               ┫ 47/2.0k [00:24<17:00, 2it/s]
+
3.034889e+04 2.4%┣▊                               ┫ 48/2.0k [00:24<16:40, 2it/s]
+
2.886274e+04 2.4%┣▉                               ┫ 49/2.0k [00:24<16:21, 2it/s]
+
2.641039e+04 2.5%┣▉                               ┫ 50/2.0k [00:24<16:03, 2it/s]
+
2.213990e+04 2.6%┣▉                               ┫ 52/2.0k [00:24<15:28, 2it/s]
+
2.006926e+04 2.6%┣▉                               ┫ 53/2.0k [00:24<15:12, 2it/s]
+
1.788785e+04 2.7%┣▉                               ┫ 54/2.0k [00:24<14:56, 2it/s]
+
1.690932e+04 2.7%┣▉                               ┫ 55/2.0k [00:24<14:42, 2it/s]
+
1.550387e+04 2.8%┣█                               ┫ 56/2.0k [00:25<14:28, 2it/s]
+
1.427511e+04 2.8%┣█                               ┫ 57/2.0k [00:25<14:13, 2it/s]
+
1.305311e+04 2.9%┣█                               ┫ 58/2.0k [00:25<14:00, 2it/s]
+
1.201805e+04 2.9%┣█                               ┫ 59/2.0k [00:25<13:47, 2it/s]
+
1.052268e+04 3.0%┣█                               ┫ 60/2.0k [00:25<13:35, 2it/s]
+
9.080636e+03 3.1%┣█                               ┫ 62/2.0k [00:25<13:10, 2it/s]
+
8.468490e+03 3.1%┣█                               ┫ 63/2.0k [00:25<12:59, 2it/s]
+
7.616312e+03 3.2%┣█                               ┫ 65/2.0k [00:25<12:37, 3it/s]
+
7.086552e+03 3.3%┣█                               ┫ 66/2.0k [00:25<12:26, 3it/s]
+
6.600463e+03 3.3%┣█                               ┫ 67/2.0k [00:25<12:16, 3it/s]
+
6.112391e+03 3.4%┣█                               ┫ 68/2.0k [00:25<12:06, 3it/s]
+
5.817186e+03 3.4%┣█                               ┫ 69/2.0k [00:25<11:57, 3it/s]
+
5.448828e+03 3.5%┣█▏                              ┫ 70/2.0k [00:25<11:48, 3it/s]
+
5.093709e+03 3.5%┣█▏                              ┫ 71/2.0k [00:25<11:39, 3it/s]
+
4.576264e+03 3.6%┣█▏                              ┫ 73/2.0k [00:25<11:21, 3it/s]
+
4.228771e+03 3.7%┣█▏                              ┫ 75/2.0k [00:26<11:04, 3it/s]
+
3.934790e+03 3.8%┣█▏                              ┫ 76/2.0k [00:26<10:57, 3it/s]
+
3.332067e+03 3.9%┣█▎                              ┫ 78/2.0k [00:26<10:42, 3it/s]
+
3.064749e+03 3.9%┣█▎                              ┫ 79/2.0k [00:26<10:34, 3it/s]
+
2.677493e+03 4.0%┣█▎                              ┫ 81/2.0k [00:26<10:20, 3it/s]
+
2.284125e+03 4.1%┣█▎                              ┫ 83/2.0k [00:26<10:07, 3it/s]
+
2.044185e+03 4.2%┣█▍                              ┫ 85/2.0k [00:26<09:54, 3it/s]
+
1.779316e+03 4.3%┣█▍                              ┫ 87/2.0k [00:26<09:42, 3it/s]
+
1.677757e+03 4.4%┣█▍                              ┫ 88/2.0k [00:26<09:36, 3it/s]
+
1.579135e+03 4.4%┣█▍                              ┫ 89/2.0k [00:26<09:30, 3it/s]
+
1.385838e+03 4.5%┣█▌                              ┫ 91/2.0k [00:26<09:19, 3it/s]
+
1.311041e+03 4.6%┣█▌                              ┫ 92/2.0k [00:26<09:14, 3it/s]
+
1.161101e+03 4.7%┣█▌                              ┫ 94/2.0k [00:26<09:03, 4it/s]
+
1.029239e+03 4.8%┣█▌                              ┫ 96/2.0k [00:27<08:53, 4it/s]
+
9.813399e+02 4.9%┣█▋                              ┫ 98/2.0k [00:27<08:43, 4it/s]
+
8.945981e+02 5.0%┣█▌                             ┫ 100/2.0k [00:27<08:33, 4it/s]
+
8.346768e+02 5.1%┣█▋                             ┫ 102/2.0k [00:27<08:24, 4it/s]
+
7.695729e+02 5.2%┣█▋                             ┫ 104/2.0k [00:27<08:16, 4it/s]
+
7.008413e+02 5.3%┣█▋                             ┫ 106/2.0k [00:27<08:07, 4it/s]
+
6.275846e+02 5.4%┣█▊                             ┫ 108/2.0k [00:27<07:59, 4it/s]
+
5.658086e+02 5.5%┣█▊                             ┫ 110/2.0k [00:27<07:52, 4it/s]
+
5.067478e+02 5.6%┣█▊                             ┫ 112/2.0k [00:27<07:44, 4it/s]
+
4.624592e+02 5.7%┣█▊                             ┫ 114/2.0k [00:27<07:37, 4it/s]
+
4.268522e+02 5.8%┣█▉                             ┫ 116/2.0k [00:27<07:30, 4it/s]
+
3.947273e+02 5.9%┣█▉                             ┫ 118/2.0k [00:28<07:23, 4it/s]
+
3.631964e+02 6.0%┣█▉                             ┫ 120/2.0k [00:28<07:17, 4it/s]
+
3.146796e+02 6.1%┣██                             ┫ 122/2.0k [00:28<07:10, 4it/s]
+
2.862678e+02 6.2%┣██                             ┫ 124/2.0k [00:28<07:04, 4it/s]
+
2.499888e+02 6.3%┣██                             ┫ 126/2.0k [00:28<06:58, 4it/s]
+
2.245613e+02 6.4%┣██                             ┫ 128/2.0k [00:28<06:52, 5it/s]
+
2.065086e+02 6.5%┣██                             ┫ 130/2.0k [00:28<06:46, 5it/s]
+
1.935059e+02 6.6%┣██                             ┫ 132/2.0k [00:28<06:41, 5it/s]
+
1.845654e+02 6.7%┣██                             ┫ 134/2.0k [00:28<06:36, 5it/s]
+
1.732868e+02 6.8%┣██                             ┫ 136/2.0k [00:28<06:31, 5it/s]
+
1.624620e+02 6.9%┣██▏                            ┫ 138/2.0k [00:28<06:25, 5it/s]
+
1.531365e+02 7.0%┣██▏                            ┫ 140/2.0k [00:28<06:21, 5it/s]
+
1.448172e+02 7.1%┣██▏                            ┫ 142/2.0k [00:28<06:16, 5it/s]
+
1.355647e+02 7.2%┣██▎                            ┫ 144/2.0k [00:29<06:11, 5it/s]
+
1.311614e+02 7.3%┣██▎                            ┫ 146/2.0k [00:29<06:06, 5it/s]
+
1.218334e+02 7.4%┣██▎                            ┫ 148/2.0k [00:29<06:02, 5it/s]
+
1.116240e+02 7.5%┣██▎                            ┫ 150/2.0k [00:29<05:57, 5it/s]
+
1.051729e+02 7.6%┣██▍                            ┫ 152/2.0k [00:29<05:53, 5it/s]
+
9.456426e+01 7.7%┣██▍                            ┫ 154/2.0k [00:29<05:49, 5it/s]
+
8.702544e+01 7.8%┣██▍                            ┫ 156/2.0k [00:29<05:45, 5it/s]
+
8.083021e+01 7.9%┣██▌                            ┫ 158/2.0k [00:29<05:41, 5it/s]
+
7.616084e+01 8.0%┣██▌                            ┫ 160/2.0k [00:29<05:37, 5it/s]
+
7.388081e+01 8.1%┣██▌                            ┫ 162/2.0k [00:29<05:33, 6it/s]
+
6.940686e+01 8.2%┣██▌                            ┫ 164/2.0k [00:29<05:30, 6it/s]
+
6.633972e+01 8.2%┣██▋                            ┫ 165/2.0k [00:29<05:28, 6it/s]
+
6.155064e+01 8.3%┣██▋                            ┫ 167/2.0k [00:29<05:25, 6it/s]
+
5.593658e+01 8.4%┣██▋                            ┫ 169/2.0k [00:29<05:21, 6it/s]
+
5.306214e+01 8.5%┣██▋                            ┫ 171/2.0k [00:30<05:18, 6it/s]
+
5.024253e+01 8.6%┣██▊                            ┫ 173/2.0k [00:30<05:15, 6it/s]
+
4.670664e+01 8.7%┣██▊                            ┫ 175/2.0k [00:30<05:11, 6it/s]
+
4.349452e+01 8.8%┣██▊                            ┫ 177/2.0k [00:30<05:08, 6it/s]
+
4.012725e+01 8.9%┣██▊                            ┫ 179/2.0k [00:30<05:05, 6it/s]
+
3.784273e+01 9.0%┣██▉                            ┫ 181/2.0k [00:30<05:02, 6it/s]
+
3.466098e+01 9.1%┣██▉                            ┫ 183/2.0k [00:30<04:59, 6it/s]
+
3.219557e+01 9.2%┣██▉                            ┫ 185/2.0k [00:30<04:56, 6it/s]
+
3.108793e+01 9.3%┣███                            ┫ 187/2.0k [00:30<04:53, 6it/s]
+
2.862140e+01 9.4%┣███                            ┫ 189/2.0k [00:30<04:50, 6it/s]
+
2.617275e+01 9.5%┣███                            ┫ 191/2.0k [00:30<04:48, 6it/s]
+
2.415737e+01 9.6%┣███                            ┫ 193/2.0k [00:30<04:45, 6it/s]
+
2.240315e+01 9.7%┣███                            ┫ 195/2.0k [00:30<04:42, 6it/s]
+
2.086250e+01 9.8%┣███                            ┫ 197/2.0k [00:30<04:40, 6it/s]
+
1.939552e+01 9.9%┣███                            ┫ 199/2.0k [00:30<04:37, 7it/s]
+
1.831251e+01 10.0%┣███                           ┫ 201/2.0k [00:30<04:34, 7it/s]
+
1.733375e+01 10.1%┣███                           ┫ 203/2.0k [00:31<04:32, 7it/s]
+
1.652585e+01 10.2%┣███                           ┫ 205/2.0k [00:31<04:30, 7it/s]
+
1.558993e+01 10.3%┣███                           ┫ 207/2.0k [00:31<04:27, 7it/s]
+
1.456385e+01 10.4%┣███▏                          ┫ 209/2.0k [00:31<04:25, 7it/s]
+
1.363940e+01 10.5%┣███▏                          ┫ 211/2.0k [00:31<04:23, 7it/s]
+
1.229869e+01 10.7%┣███▏                          ┫ 214/2.0k [00:31<04:19, 7it/s]
+
1.157458e+01 10.8%┣███▎                          ┫ 216/2.0k [00:31<04:17, 7it/s]
+
1.088388e+01 10.9%┣███▎                          ┫ 218/2.0k [00:31<04:15, 7it/s]
+
1.024653e+01 11.0%┣███▎                          ┫ 220/2.0k [00:31<04:13, 7it/s]
+
9.442511e+00 11.1%┣███▎                          ┫ 222/2.0k [00:31<04:10, 7it/s]
+
8.513503e+00 11.2%┣███▍                          ┫ 225/2.0k [00:31<04:07, 7it/s]
+
8.040768e+00 11.3%┣███▍                          ┫ 227/2.0k [00:31<04:05, 7it/s]
+
7.493484e+00 11.5%┣███▌                          ┫ 230/2.0k [00:31<04:02, 7it/s]
+
7.100380e+00 11.6%┣███▌                          ┫ 232/2.0k [00:31<04:00, 7it/s]
+
6.669313e+00 11.7%┣███▌                          ┫ 234/2.0k [00:31<03:59, 7it/s]
+
6.214928e+00 11.8%┣███▌                          ┫ 236/2.0k [00:32<03:57, 7it/s]
+
5.808669e+00 11.9%┣███▋                          ┫ 239/2.0k [00:32<03:54, 8it/s]
+
5.467886e+00 12.1%┣███▋                          ┫ 242/2.0k [00:32<03:51, 8it/s]
+
5.260139e+00 12.2%┣███▋                          ┫ 244/2.0k [00:32<03:49, 8it/s]
+
4.944886e+00 12.3%┣███▊                          ┫ 247/2.0k [00:32<03:47, 8it/s]
+
4.614976e+00 12.5%┣███▊                          ┫ 250/2.0k [00:32<03:44, 8it/s]
+
4.427758e+00 12.6%┣███▉                          ┫ 252/2.0k [00:32<03:42, 8it/s]
+
4.111345e+00 12.7%┣███▉                          ┫ 255/2.0k [00:32<03:40, 8it/s]
+
3.866444e+00 12.8%┣███▉                          ┫ 257/2.0k [00:32<03:38, 8it/s]
+
3.678647e+00 13.0%┣████                          ┫ 260/2.0k [00:32<03:35, 8it/s]
+
3.558763e+00 13.1%┣████                          ┫ 263/2.0k [00:32<03:33, 8it/s]
+
3.350214e+00 13.3%┣████                          ┫ 266/2.0k [00:32<03:31, 8it/s]
+
3.115146e+00 13.4%┣████                          ┫ 269/2.0k [00:32<03:28, 8it/s]
+
2.919205e+00 13.6%┣████                          ┫ 272/2.0k [00:32<03:26, 8it/s]
+
2.769096e+00 13.7%┣████▏                         ┫ 275/2.0k [00:32<03:24, 8it/s]
+
2.581184e+00 13.9%┣████▏                         ┫ 278/2.0k [00:32<03:22, 9it/s]
+
2.394951e+00 14.0%┣████▏                         ┫ 281/2.0k [00:33<03:20, 9it/s]
+
2.221354e+00 14.2%┣████▎                         ┫ 284/2.0k [00:33<03:18, 9it/s]
+
2.098463e+00 14.3%┣████▎                         ┫ 287/2.0k [00:33<03:16, 9it/s]
+
1.980453e+00 14.5%┣████▍                         ┫ 290/2.0k [00:33<03:14, 9it/s]
+
1.874042e+00 14.6%┣████▍                         ┫ 293/2.0k [00:33<03:12, 9it/s]
+
1.800158e+00 14.8%┣████▍                         ┫ 296/2.0k [00:33<03:10, 9it/s]
+
1.689793e+00 14.9%┣████▌                         ┫ 299/2.0k [00:33<03:08, 9it/s]
+
1.552073e+00 15.1%┣████▌                         ┫ 302/2.0k [00:33<03:06, 9it/s]
+
1.448550e+00 15.3%┣████▋                         ┫ 306/2.0k [00:33<03:03, 9it/s]
+
1.353985e+00 15.4%┣████▋                         ┫ 309/2.0k [00:33<03:02, 9it/s]
+
1.289689e+00 15.6%┣████▊                         ┫ 312/2.0k [00:33<03:00, 9it/s]
+
1.222974e+00 15.7%┣████▊                         ┫ 315/2.0k [00:33<02:58, 9it/s]
+
1.137828e+00 15.9%┣████▋                        ┫ 319/2.0k [00:33<02:56, 10it/s]
+
1.071346e+00 16.1%┣████▊                        ┫ 322/2.0k [00:33<02:54, 10it/s]
+
1.027791e+00 16.3%┣████▊                        ┫ 326/2.0k [00:33<02:52, 10it/s]
+
9.687889e-01 16.5%┣████▉                        ┫ 330/2.0k [00:33<02:50, 10it/s]
+
9.400396e-01 16.6%┣████▉                        ┫ 333/2.0k [00:33<02:48, 10it/s]
+
8.856533e-01 16.8%┣████▉                        ┫ 336/2.0k [00:34<02:47, 10it/s]
+
8.227219e-01 17.0%┣█████                        ┫ 340/2.0k [00:34<02:45, 10it/s]
+
7.757657e-01 17.2%┣█████                        ┫ 344/2.0k [00:34<02:43, 10it/s]
+
7.217234e-01 17.4%┣█████                        ┫ 348/2.0k [00:34<02:41, 10it/s]
+
6.812064e-01 17.6%┣█████                        ┫ 352/2.0k [00:34<02:39, 10it/s]
+
6.372631e-01 17.8%┣█████▏                       ┫ 356/2.0k [00:34<02:37, 10it/s]
+
5.970611e-01 18.0%┣█████▏                       ┫ 360/2.0k [00:34<02:35, 11it/s]
+
5.502127e-01 18.2%┣█████▎                       ┫ 365/2.0k [00:34<02:33, 11it/s]
+
4.926176e-01 18.4%┣█████▍                       ┫ 369/2.0k [00:34<02:31, 11it/s]
+
4.561056e-01 18.6%┣█████▍                       ┫ 373/2.0k [00:34<02:29, 11it/s]
+
4.256850e-01 18.8%┣█████▌                       ┫ 377/2.0k [00:34<02:27, 11it/s]
+
3.946217e-01 19.1%┣█████▌                       ┫ 382/2.0k [00:34<02:25, 11it/s]
+
3.512799e-01 19.3%┣█████▋                       ┫ 387/2.0k [00:34<02:23, 11it/s]
+
3.323678e-01 19.6%┣█████▊                       ┫ 393/2.0k [00:34<02:21, 11it/s]
+
3.139548e-01 19.9%┣█████▉                       ┫ 399/2.0k [00:34<02:18, 12it/s]
+
2.943868e-01 20.2%┣█████▉                       ┫ 405/2.0k [00:34<02:16, 12it/s]
+
2.760423e-01 20.5%┣██████                       ┫ 411/2.0k [00:34<02:14, 12it/s]
+
2.640383e-01 20.7%┣██████                       ┫ 415/2.0k [00:35<02:12, 12it/s]
+
2.448832e-01 21.0%┣██████                       ┫ 420/2.0k [00:35<02:10, 12it/s]
+
2.220141e-01 21.3%┣██████▏                      ┫ 426/2.0k [00:35<02:08, 12it/s]
+
2.042357e-01 21.5%┣██████▎                      ┫ 431/2.0k [00:35<02:07, 12it/s]
+
1.873298e-01 21.8%┣██████▎                      ┫ 436/2.0k [00:35<02:05, 13it/s]
+
1.704014e-01 22.1%┣██████▍                      ┫ 443/2.0k [00:35<02:03, 13it/s]
+
1.600044e-01 22.5%┣██████▌                      ┫ 450/2.0k [00:35<02:00, 13it/s]
+
1.500028e-01 22.7%┣██████▋                      ┫ 455/2.0k [00:35<01:59, 13it/s]
+
1.390815e-01 23.1%┣██████▊                      ┫ 462/2.0k [00:35<01:57, 13it/s]
+
1.258707e-01 23.3%┣██████▊                      ┫ 467/2.0k [00:35<01:55, 13it/s]
+
1.165963e-01 23.6%┣██████▉                      ┫ 473/2.0k [00:35<01:53, 13it/s]
+
1.084351e-01 23.9%┣███████                      ┫ 479/2.0k [00:35<01:52, 14it/s]
+
1.019123e-01 24.2%┣███████                      ┫ 485/2.0k [00:35<01:50, 14it/s]
+
9.283235e-02 24.6%┣███████▏                     ┫ 492/2.0k [00:35<01:48, 14it/s]
+
8.672678e-02 24.9%┣███████▏                     ┫ 498/2.0k [00:35<01:47, 14it/s]
+
7.939853e-02 25.2%┣███████▎                     ┫ 505/2.0k [00:35<01:45, 14it/s]
+
7.349814e-02 25.5%┣███████▍                     ┫ 511/2.0k [00:35<01:43, 14it/s]
+
6.902960e-02 25.9%┣███████▌                     ┫ 518/2.0k [00:35<01:42, 15it/s]
+
6.510985e-02 26.2%┣███████▋                     ┫ 525/2.0k [00:35<01:40, 15it/s]
+
6.169216e-02 26.5%┣███████▊                     ┫ 531/2.0k [00:36<01:39, 15it/s]
+
5.730092e-02 26.8%┣███████▉                     ┫ 537/2.0k [00:36<01:37, 15it/s]
+
5.247910e-02 27.2%┣███████▉                     ┫ 544/2.0k [00:36<01:36, 15it/s]
+
4.866153e-02 27.5%┣████████                     ┫ 551/2.0k [00:36<01:34, 15it/s]
+
4.550076e-02 27.9%┣████████                     ┫ 558/2.0k [00:36<01:33, 16it/s]
+
4.213091e-02 28.2%┣████████▏                    ┫ 565/2.0k [00:36<01:31, 16it/s]
+
3.900969e-02 28.6%┣████████▎                    ┫ 572/2.0k [00:36<01:30, 16it/s]
+
3.677024e-02 28.9%┣████████▍                    ┫ 579/2.0k [00:36<01:28, 16it/s]
+
3.472606e-02 29.3%┣████████▌                    ┫ 586/2.0k [00:36<01:27, 16it/s]
+
3.319640e-02 29.6%┣████████▋                    ┫ 593/2.0k [00:36<01:26, 16it/s]
+
3.137980e-02 30.0%┣████████▊                    ┫ 600/2.0k [00:36<01:24, 17it/s]
+
2.917913e-02 30.3%┣████████▉                    ┫ 607/2.0k [00:36<01:23, 17it/s]
+
2.694958e-02 30.7%┣█████████                    ┫ 614/2.0k [00:36<01:22, 17it/s]
+
2.478032e-02 31.0%┣█████████                    ┫ 621/2.0k [00:36<01:21, 17it/s]
+
2.278852e-02 31.4%┣█████████                    ┫ 628/2.0k [00:36<01:19, 17it/s]
+
2.106085e-02 31.7%┣█████████▏                   ┫ 635/2.0k [00:36<01:18, 17it/s]
+
1.989781e-02 32.1%┣█████████▎                   ┫ 642/2.0k [00:36<01:17, 18it/s]
+
1.893901e-02 32.4%┣█████████▍                   ┫ 649/2.0k [00:36<01:16, 18it/s]
+
1.777541e-02 32.8%┣█████████▌                   ┫ 656/2.0k [00:37<01:15, 18it/s]
+
1.666015e-02 33.1%┣█████████▋                   ┫ 663/2.0k [00:37<01:14, 18it/s]
+
1.557781e-02 33.4%┣█████████▊                   ┫ 669/2.0k [00:37<01:13, 18it/s]
+
1.463196e-02 33.8%┣█████████▉                   ┫ 676/2.0k [00:37<01:12, 18it/s]
+
1.377993e-02 34.1%┣██████████                   ┫ 683/2.0k [00:37<01:11, 19it/s]
+
1.300681e-02 34.5%┣██████████                   ┫ 690/2.0k [00:37<01:10, 19it/s]
+
1.225560e-02 34.8%┣██████████                   ┫ 697/2.0k [00:37<01:09, 19it/s]
+
1.175614e-02 35.2%┣██████████▏                  ┫ 704/2.0k [00:37<01:08, 19it/s]
+
1.131073e-02 35.5%┣██████████▎                  ┫ 711/2.0k [00:37<01:07, 19it/s]
+
1.084708e-02 35.9%┣██████████▍                  ┫ 718/2.0k [00:37<01:06, 19it/s]
+
1.039581e-02 36.2%┣██████████▌                  ┫ 725/2.0k [00:37<01:05, 20it/s]
+
9.987068e-03 36.6%┣██████████▋                  ┫ 732/2.0k [00:37<01:04, 20it/s]
+
9.615231e-03 36.9%┣██████████▊                  ┫ 739/2.0k [00:37<01:04, 20it/s]
+
9.219408e-03 37.3%┣██████████▉                  ┫ 746/2.0k [00:37<01:03, 20it/s]
+
8.915399e-03 37.6%┣███████████                  ┫ 753/2.0k [00:37<01:02, 20it/s]
+
8.594979e-03 38.0%┣███████████                  ┫ 760/2.0k [00:37<01:01, 20it/s]
+
8.233401e-03 38.3%┣███████████▏                 ┫ 767/2.0k [00:37<01:00, 20it/s]
+
7.859428e-03 38.7%┣███████████▏                 ┫ 774/2.0k [00:37<00:59, 21it/s]
+
7.490760e-03 39.0%┣███████████▎                 ┫ 781/2.0k [00:37<00:59, 21it/s]
+
7.137330e-03 39.4%┣███████████▍                 ┫ 788/2.0k [00:38<00:58, 21it/s]
+
6.824673e-03 39.7%┣███████████▌                 ┫ 795/2.0k [00:38<00:57, 21it/s]
+
6.565204e-03 40.0%┣███████████▋                 ┫ 801/2.0k [00:38<00:56, 21it/s]
+
6.233886e-03 40.4%┣███████████▊                 ┫ 808/2.0k [00:38<00:56, 21it/s]
+
5.897948e-03 40.7%┣███████████▉                 ┫ 815/2.0k [00:38<00:55, 22it/s]
+
5.604426e-03 41.1%┣████████████                 ┫ 822/2.0k [00:38<00:54, 22it/s]
+
5.246367e-03 41.4%┣████████████                 ┫ 829/2.0k [00:38<00:54, 22it/s]
+
4.886677e-03 41.8%┣████████████▏                ┫ 836/2.0k [00:38<00:53, 22it/s]
+
4.633389e-03 42.2%┣████████████▎                ┫ 844/2.0k [00:38<00:52, 22it/s]
+
4.398341e-03 42.5%┣████████████▍                ┫ 851/2.0k [00:38<00:51, 22it/s]
+
4.186602e-03 42.9%┣████████████▌                ┫ 859/2.0k [00:38<00:51, 23it/s]
+
4.018980e-03 43.3%┣████████████▋                ┫ 867/2.0k [00:38<00:50, 23it/s]
+
3.830157e-03 43.7%┣████████████▊                ┫ 874/2.0k [00:38<00:49, 23it/s]
+
3.716610e-03 44.1%┣████████████▉                ┫ 882/2.0k [00:38<00:49, 23it/s]
+
3.570926e-03 44.5%┣█████████████                ┫ 891/2.0k [00:38<00:48, 23it/s]
+
3.401090e-03 44.9%┣█████████████                ┫ 899/2.0k [00:38<00:47, 23it/s]
+
3.295424e-03 45.3%┣█████████████▏               ┫ 906/2.0k [00:38<00:46, 24it/s]
+
3.188860e-03 45.7%┣█████████████▎               ┫ 914/2.0k [00:38<00:46, 24it/s]
+
3.114334e-03 46.1%┣█████████████▍               ┫ 922/2.0k [00:39<00:45, 24it/s]
+
3.056130e-03 46.5%┣█████████████▌               ┫ 930/2.0k [00:39<00:44, 24it/s]
+
2.983980e-03 46.9%┣█████████████▋               ┫ 939/2.0k [00:39<00:44, 24it/s]
+
2.893251e-03 47.3%┣█████████████▊               ┫ 946/2.0k [00:39<00:43, 24it/s]
+
2.775435e-03 47.7%┣█████████████▉               ┫ 954/2.0k [00:39<00:43, 25it/s]
+
2.681086e-03 48.1%┣██████████████               ┫ 962/2.0k [00:39<00:42, 25it/s]
+
2.509660e-03 48.5%┣██████████████               ┫ 970/2.0k [00:39<00:41, 25it/s]
+
2.379199e-03 48.9%┣██████████████▏              ┫ 978/2.0k [00:39<00:41, 25it/s]
+
2.276732e-03 49.3%┣██████████████▎              ┫ 987/2.0k [00:39<00:40, 25it/s]
+
2.174759e-03 49.7%┣██████████████▍              ┫ 995/2.0k [00:39<00:39, 25it/s]
+
2.064730e-03 50.1%┣██████████████              ┫ 1.0k/2.0k [00:39<00:39, 26it/s]
+
2.017029e-03 50.5%┣██████████████▏             ┫ 1.0k/2.0k [00:39<00:38, 26it/s]
+
1.924628e-03 50.9%┣██████████████▎             ┫ 1.0k/2.0k [00:39<00:38, 26it/s]
+
1.859485e-03 51.4%┣██████████████▍             ┫ 1.0k/2.0k [00:39<00:37, 26it/s]
+
1.802581e-03 51.8%┣██████████████▌             ┫ 1.0k/2.0k [00:39<00:37, 26it/s]
+
1.742935e-03 52.1%┣██████████████▋             ┫ 1.0k/2.0k [00:39<00:36, 26it/s]
+
1.661336e-03 52.6%┣██████████████▊             ┫ 1.1k/2.0k [00:39<00:36, 27it/s]
+
1.591441e-03 53.0%┣██████████████▉             ┫ 1.1k/2.0k [00:39<00:35, 27it/s]
+
1.445303e-03 53.4%┣███████████████             ┫ 1.1k/2.0k [00:39<00:34, 27it/s]
+
1.356997e-03 53.8%┣███████████████             ┫ 1.1k/2.0k [00:40<00:34, 27it/s]
+
1.277342e-03 54.3%┣███████████████▏            ┫ 1.1k/2.0k [00:40<00:33, 27it/s]
+
1.213682e-03 54.7%┣███████████████▎            ┫ 1.1k/2.0k [00:40<00:33, 28it/s]
+
1.160094e-03 55.1%┣███████████████▍            ┫ 1.1k/2.0k [00:40<00:32, 28it/s]
+
1.117127e-03 55.4%┣███████████████▌            ┫ 1.1k/2.0k [00:40<00:32, 28it/s]
+
1.071530e-03 55.8%┣███████████████▋            ┫ 1.1k/2.0k [00:40<00:32, 28it/s]
+
1.024973e-03 56.2%┣███████████████▊            ┫ 1.1k/2.0k [00:40<00:31, 28it/s]
+
9.910321e-04 56.6%┣███████████████▉            ┫ 1.1k/2.0k [00:40<00:31, 28it/s]
+
9.546126e-04 57.1%┣████████████████            ┫ 1.1k/2.0k [00:40<00:30, 29it/s]
+
9.340374e-04 57.5%┣████████████████            ┫ 1.2k/2.0k [00:40<00:30, 29it/s]
+
8.865975e-04 57.9%┣████████████████▏           ┫ 1.2k/2.0k [00:40<00:29, 29it/s]
+
8.344196e-04 58.4%┣████████████████▍           ┫ 1.2k/2.0k [00:40<00:29, 29it/s]
+
8.045959e-04 58.9%┣████████████████▌           ┫ 1.2k/2.0k [00:40<00:28, 29it/s]
+
7.910708e-04 59.3%┣████████████████▋           ┫ 1.2k/2.0k [00:40<00:28, 29it/s]
+
7.673729e-04 59.7%┣████████████████▊           ┫ 1.2k/2.0k [00:40<00:27, 30it/s]
+
7.362953e-04 60.1%┣████████████████▉           ┫ 1.2k/2.0k [00:40<00:27, 30it/s]
+
7.165144e-04 60.6%┣█████████████████           ┫ 1.2k/2.0k [00:40<00:26, 30it/s]
+
6.932469e-04 61.0%┣█████████████████           ┫ 1.2k/2.0k [00:40<00:26, 30it/s]
+
6.719131e-04 61.4%┣█████████████████▏          ┫ 1.2k/2.0k [00:41<00:25, 30it/s]
+
6.513871e-04 61.9%┣█████████████████▎          ┫ 1.2k/2.0k [00:41<00:25, 30it/s]
+
6.205348e-04 62.4%┣█████████████████▌          ┫ 1.2k/2.0k [00:41<00:25, 31it/s]
+
6.076863e-04 62.8%┣█████████████████▋          ┫ 1.3k/2.0k [00:41<00:24, 31it/s]
+
5.964071e-04 63.2%┣█████████████████▊          ┫ 1.3k/2.0k [00:41<00:24, 31it/s]
+
5.818213e-04 63.6%┣█████████████████▉          ┫ 1.3k/2.0k [00:41<00:23, 31it/s]
+
5.692658e-04 64.0%┣██████████████████          ┫ 1.3k/2.0k [00:41<00:23, 31it/s]
+
5.607410e-04 64.4%┣██████████████████          ┫ 1.3k/2.0k [00:41<00:23, 32it/s]
+
5.485883e-04 64.8%┣██████████████████▏         ┫ 1.3k/2.0k [00:41<00:22, 32it/s]
+
5.366961e-04 65.2%┣██████████████████▎         ┫ 1.3k/2.0k [00:41<00:22, 32it/s]
+
5.249423e-04 65.7%┣██████████████████▍         ┫ 1.3k/2.0k [00:41<00:21, 32it/s]
+
5.142230e-04 66.1%┣██████████████████▌         ┫ 1.3k/2.0k [00:41<00:21, 32it/s]
+
5.001330e-04 66.5%┣██████████████████▋         ┫ 1.3k/2.0k [00:41<00:21, 32it/s]
+
4.904572e-04 66.9%┣██████████████████▊         ┫ 1.3k/2.0k [00:41<00:20, 32it/s]
+
4.810546e-04 67.3%┣██████████████████▉         ┫ 1.3k/2.0k [00:41<00:20, 33it/s]
+
4.708195e-04 67.8%┣███████████████████         ┫ 1.4k/2.0k [00:41<00:20, 33it/s]
+
4.634700e-04 68.2%┣███████████████████         ┫ 1.4k/2.0k [00:41<00:19, 33it/s]
+
4.583471e-04 68.6%┣███████████████████▏        ┫ 1.4k/2.0k [00:41<00:19, 33it/s]
+
4.474134e-04 69.0%┣███████████████████▎        ┫ 1.4k/2.0k [00:41<00:19, 33it/s]
+
4.382568e-04 69.5%┣███████████████████▌        ┫ 1.4k/2.0k [00:42<00:18, 33it/s]
+
4.292046e-04 69.9%┣███████████████████▋        ┫ 1.4k/2.0k [00:42<00:18, 34it/s]
+
4.195929e-04 70.3%┣███████████████████▊        ┫ 1.4k/2.0k [00:42<00:18, 34it/s]
+
4.115266e-04 70.7%┣███████████████████▉        ┫ 1.4k/2.0k [00:42<00:17, 34it/s]
+
4.048179e-04 71.2%┣████████████████████        ┫ 1.4k/2.0k [00:42<00:17, 34it/s]
+
3.979605e-04 71.6%┣████████████████████        ┫ 1.4k/2.0k [00:42<00:17, 34it/s]
+
3.921296e-04 71.9%┣████████████████████▏       ┫ 1.4k/2.0k [00:42<00:16, 34it/s]
+
3.829934e-04 72.4%┣████████████████████▎       ┫ 1.4k/2.0k [00:42<00:16, 35it/s]
+
3.758210e-04 72.9%┣████████████████████▍       ┫ 1.5k/2.0k [00:42<00:16, 35it/s]
+
3.697143e-04 73.3%┣████████████████████▌       ┫ 1.5k/2.0k [00:42<00:15, 35it/s]
+
3.618927e-04 73.7%┣████████████████████▋       ┫ 1.5k/2.0k [00:42<00:15, 35it/s]
+
3.549504e-04 74.1%┣████████████████████▊       ┫ 1.5k/2.0k [00:42<00:15, 35it/s]
+
3.432485e-04 74.6%┣█████████████████████       ┫ 1.5k/2.0k [00:42<00:14, 35it/s]
+
3.307458e-04 75.0%┣█████████████████████       ┫ 1.5k/2.0k [00:42<00:14, 36it/s]
+
3.225473e-04 75.4%┣█████████████████████▏      ┫ 1.5k/2.0k [00:42<00:14, 36it/s]
+
3.141893e-04 75.9%┣█████████████████████▎      ┫ 1.5k/2.0k [00:42<00:13, 36it/s]
+
2.971918e-04 76.4%┣█████████████████████▍      ┫ 1.5k/2.0k [00:42<00:13, 36it/s]
+
2.891459e-04 76.8%┣█████████████████████▌      ┫ 1.5k/2.0k [00:42<00:13, 36it/s]
+
2.825005e-04 77.2%┣█████████████████████▋      ┫ 1.5k/2.0k [00:43<00:13, 36it/s]
+
2.772244e-04 77.6%┣█████████████████████▊      ┫ 1.6k/2.0k [00:43<00:12, 36it/s]
+
2.718808e-04 78.1%┣█████████████████████▉      ┫ 1.6k/2.0k [00:43<00:12, 37it/s]
+
2.677189e-04 78.5%┣██████████████████████      ┫ 1.6k/2.0k [00:43<00:12, 37it/s]
+
2.644979e-04 78.9%┣██████████████████████      ┫ 1.6k/2.0k [00:43<00:11, 37it/s]
+
2.627322e-04 79.2%┣██████████████████████▏     ┫ 1.6k/2.0k [00:43<00:11, 37it/s]
+
2.601727e-04 79.6%┣██████████████████████▎     ┫ 1.6k/2.0k [00:43<00:11, 37it/s]
+
2.575263e-04 79.9%┣██████████████████████▍     ┫ 1.6k/2.0k [00:43<00:11, 37it/s]
+
2.546319e-04 80.3%┣██████████████████████▌     ┫ 1.6k/2.0k [00:43<00:11, 37it/s]
+
2.490873e-04 80.7%┣██████████████████████▋     ┫ 1.6k/2.0k [00:43<00:10, 38it/s]
+
2.418100e-04 81.1%┣██████████████████████▊     ┫ 1.6k/2.0k [00:43<00:10, 38it/s]
+
2.357787e-04 81.6%┣██████████████████████▉     ┫ 1.6k/2.0k [00:43<00:10, 38it/s]
+
2.320793e-04 82.0%┣███████████████████████     ┫ 1.6k/2.0k [00:43<00:09, 38it/s]
+
2.299007e-04 82.4%┣███████████████████████     ┫ 1.6k/2.0k [00:43<00:09, 38it/s]
+
2.238167e-04 82.9%┣███████████████████████▏    ┫ 1.7k/2.0k [00:43<00:09, 38it/s]
+
2.161775e-04 83.3%┣███████████████████████▎    ┫ 1.7k/2.0k [00:43<00:09, 38it/s]
+
2.087232e-04 83.8%┣███████████████████████▌    ┫ 1.7k/2.0k [00:43<00:08, 39it/s]
+
2.034028e-04 84.2%┣███████████████████████▋    ┫ 1.7k/2.0k [00:43<00:08, 39it/s]
+
2.009472e-04 84.6%┣███████████████████████▊    ┫ 1.7k/2.0k [00:43<00:08, 39it/s]
+
1.981467e-04 85.1%┣███████████████████████▉    ┫ 1.7k/2.0k [00:44<00:08, 39it/s]
+
1.957445e-04 85.5%┣████████████████████████    ┫ 1.7k/2.0k [00:44<00:07, 39it/s]
+
1.921294e-04 85.9%┣████████████████████████    ┫ 1.7k/2.0k [00:44<00:07, 39it/s]
+
1.881253e-04 86.4%┣████████████████████████▏   ┫ 1.7k/2.0k [00:44<00:07, 40it/s]
+
1.847526e-04 86.9%┣████████████████████████▎   ┫ 1.7k/2.0k [00:44<00:07, 40it/s]
+
1.806075e-04 87.3%┣████████████████████████▍   ┫ 1.7k/2.0k [00:44<00:06, 40it/s]
+
1.759428e-04 87.7%┣████████████████████████▌   ┫ 1.8k/2.0k [00:44<00:06, 40it/s]
+
1.734236e-04 88.0%┣████████████████████████▋   ┫ 1.8k/2.0k [00:44<00:06, 40it/s]
+
1.706921e-04 88.5%┣████████████████████████▊   ┫ 1.8k/2.0k [00:44<00:06, 40it/s]
+
1.653823e-04 88.9%┣████████████████████████▉   ┫ 1.8k/2.0k [00:44<00:06, 40it/s]
+
1.628589e-04 89.3%┣█████████████████████████   ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
+
1.600854e-04 89.6%┣█████████████████████████   ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
+
1.572501e-04 90.1%┣█████████████████████████▎  ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
+
1.535314e-04 90.6%┣█████████████████████████▍  ┫ 1.8k/2.0k [00:44<00:05, 41it/s]
+
1.503157e-04 91.0%┣█████████████████████████▌  ┫ 1.8k/2.0k [00:44<00:04, 41it/s]
+
1.488655e-04 91.4%┣█████████████████████████▋  ┫ 1.8k/2.0k [00:44<00:04, 41it/s]
+
1.462340e-04 91.9%┣█████████████████████████▊  ┫ 1.8k/2.0k [00:44<00:04, 41it/s]
+
1.422826e-04 92.3%┣█████████████████████████▉  ┫ 1.8k/2.0k [00:44<00:04, 42it/s]
+
1.397570e-04 92.8%┣██████████████████████████  ┫ 1.9k/2.0k [00:45<00:03, 42it/s]
+
1.373555e-04 93.1%┣██████████████████████████  ┫ 1.9k/2.0k [00:45<00:03, 42it/s]
+
1.354739e-04 93.6%┣██████████████████████████▏ ┫ 1.9k/2.0k [00:45<00:03, 42it/s]
+
1.325553e-04 94.0%┣██████████████████████████▎ ┫ 1.9k/2.0k [00:45<00:03, 42it/s]
+
1.271589e-04 94.5%┣██████████████████████████▌ ┫ 1.9k/2.0k [00:45<00:03, 42it/s]
+
1.247256e-04 94.8%┣██████████████████████████▌ ┫ 1.9k/2.0k [00:45<00:02, 42it/s]
+
1.228197e-04 95.3%┣██████████████████████████▊ ┫ 1.9k/2.0k [00:45<00:02, 42it/s]
+
1.215318e-04 95.7%┣██████████████████████████▉ ┫ 1.9k/2.0k [00:45<00:02, 43it/s]
+
1.204135e-04 96.2%┣███████████████████████████ ┫ 1.9k/2.0k [00:45<00:02, 43it/s]
+
1.195659e-04 96.4%┣███████████████████████████ ┫ 1.9k/2.0k [00:45<00:02, 43it/s]
+
1.178891e-04 96.8%┣███████████████████████████ ┫ 1.9k/2.0k [00:45<00:01, 43it/s]
+
1.158785e-04 97.3%┣███████████████████████████▎┫ 1.9k/2.0k [00:45<00:01, 43it/s]
+
1.153884e-04 97.7%┣███████████████████████████▍┫ 2.0k/2.0k [00:45<00:01, 43it/s]
+
1.147710e-04 98.1%┣███████████████████████████▌┫ 2.0k/2.0k [00:45<00:01, 43it/s]
+
1.139623e-04 98.5%┣███████████████████████████▋┫ 2.0k/2.0k [00:45<00:01, 44it/s]
+
1.124524e-04 99.0%┣███████████████████████████▊┫ 2.0k/2.0k [00:45<00:00, 44it/s]
+
1.103420e-04 99.4%┣███████████████████████████▉┫ 2.0k/2.0k [00:45<00:00, 44it/s]
+
1.078737e-04 99.9%┣████████████████████████████┫ 2.0k/2.0k [00:45<00:00, 44it/s]
+
1.074057e-04 100.0%┣███████████████████████████┫ 2.0k/2.0k [00:45<00:00, 44it/s]

Compute the relative L2 error

using Integrals
 
 u_analytical(x,p) = sin.(2 * pi .* x) + 0.1 * sin.(50 * pi .* x)
 error(x,p) = abs2.(vec(phi([x;;],res.u)) .- u_analytical(x,p))
 
 relative_L2_error = solve(IntegralProblem(error,0,1),HCubatureJL(),reltol=1e-3,abstol=1e-3) ./ solve(IntegralProblem((x,p) -> abs2.(u_analytical(x,p)),0, 1),HCubatureJL(),reltol=1e-3,abstol=1e-3)
1-element Vector{Float64}:
- 6.601930909432112e-9
+ 6.5670689063348285e-9 diff --git a/dev/tutorials/result.png b/dev/tutorials/result.png index 3a878cdd..e17870be 100644 Binary files a/dev/tutorials/result.png and b/dev/tutorials/result.png differ diff --git a/dev/tutorials/result.svg b/dev/tutorials/result.svg index 20d80e55..9acc71cf 100644 --- a/dev/tutorials/result.svg +++ b/dev/tutorials/result.svg @@ -1,54 +1,52 @@ - + - + - + - + - - + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/result1.svg b/dev/tutorials/result1.svg index 818b4e6a..ab225d65 100644 --- a/dev/tutorials/result1.svg +++ b/dev/tutorials/result1.svg @@ -1,48 +1,48 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/result10.svg b/dev/tutorials/result10.svg index edc52d3d..73fd7e06 100644 --- a/dev/tutorials/result10.svg +++ b/dev/tutorials/result10.svg @@ -1,48 +1,48 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/result2.svg b/dev/tutorials/result2.svg index 5ef7fbc9..2c7aa6b0 100644 --- a/dev/tutorials/result2.svg +++ b/dev/tutorials/result2.svg @@ -1,48 +1,48 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/result3.svg b/dev/tutorials/result3.svg index 31af27e2..1f320315 100644 --- a/dev/tutorials/result3.svg +++ b/dev/tutorials/result3.svg @@ -1,54 +1,54 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/sod/1ecfe341.svg b/dev/tutorials/sod/1ecfe341.svg new file mode 100644 index 00000000..6d95c772 --- /dev/null +++ b/dev/tutorials/sod/1ecfe341.svg @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/sod/601f42e8.svg b/dev/tutorials/sod/601f42e8.svg deleted file mode 100644 index 56f7a049..00000000 --- a/dev/tutorials/sod/601f42e8.svg +++ /dev/null @@ -1,50 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/dev/tutorials/sod/index.html b/dev/tutorials/sod/index.html index 00539bf4..63542709 100644 --- a/dev/tutorials/sod/index.html +++ b/dev/tutorials/sod/index.html @@ -62,4 +62,4 @@ phi = pinn.phi p1 = plot(xs, [first(phi.u([t_max, x], θ.u)) for x in xs]; label="u(t=1,x)") p2 = plot!(xs, [first(phi.ρ([t_max, x], θ.ρ)) for x in xs]; label="ρ(t=1,x)") -p3 = plot!(xs, [first(phi.p([t_max, x], θ.p)) for x in xs]; label="p(t=1,x)")Example block output +p3 = plot!(xs, [first(phi.p([t_max, x], θ.p)) for x in xs]; label="p(t=1,x)")Example block output diff --git a/dev/tutorials/sol.png b/dev/tutorials/sol.png index 55f23791..2f8e3031 100644 Binary files a/dev/tutorials/sol.png and b/dev/tutorials/sol.png differ diff --git a/dev/tutorials/u.svg b/dev/tutorials/u.svg index 5da3bd37..3f5208bd 100644 --- a/dev/tutorials/u.svg +++ b/dev/tutorials/u.svg @@ -1,43 +1,43 @@ - + - + - + - + - + - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev/tutorials/uv.png b/dev/tutorials/uv.png index d519b695..96e80c71 100644 Binary files a/dev/tutorials/uv.png and b/dev/tutorials/uv.png differ diff --git a/dev/tutorials/velocity.png b/dev/tutorials/velocity.png index f00318cf..3e312f2e 100644 Binary files a/dev/tutorials/velocity.png and b/dev/tutorials/velocity.png differ diff --git a/dev/tutorials/waveinverse2/index.html b/dev/tutorials/waveinverse2/index.html index c5f52901..181b32e1 100644 --- a/dev/tutorials/waveinverse2/index.html +++ b/dev/tutorials/waveinverse2/index.html @@ -40,7 +40,7 @@ prob = Sophon.discretize(wave, pinn, sampler, strategy; additional_loss=additional_loss) @showprogress res = Optimization.solve(prob, BFGS(), maxiters=1000)
retcode: Failure
-u: ComponentVector{Float64}(u = (layer_1 = (weight = [0.1866745956296791 0.5588647531210649; 0.42447906724208256 1.0174364790416819; … ; 1.0751223200264266 0.47413518243044067; 0.23313351985433708 0.8796479085814862], bias = [0.2541686993518635; 0.20543747153898775; … ; -0.3285586224607518; -0.22729146365723693;;]), layer_2 = (weight = [0.2850152125393203 0.19817394398512114 … -0.3709122902879997 -0.6825853093651886; 0.42359575087043283 -0.0900362307928494 … -0.06570139241080698 -0.1794160345733994; … ; -0.3681603553929829 0.011134748927023295 … 0.5070070797688715 0.380876486144369; 0.4302329114019143 -0.6488965452677948 … 0.2327728557537271 -0.7167344902381557], bias = [-0.34788555002723154; 0.06363006967233681; … ; 0.4877859585715955; 0.007947232044147599;;]), layer_3 = (weight = [-0.37981374079888236 0.3071206144790822 … 0.40253749572936864 -0.47280030363227793; -0.28884829257563843 0.01644643573389052 … 0.39780743364337756 0.24816794638386805; … ; 0.19776347898420996 0.3985934321050157 … -0.5714752907221025 -0.08757763508167311; -0.5329704921970722 -0.11117218632996168 … -0.1518881034126511 0.5143956705034471], bias = [-0.1526616563165587; 0.05670620257173293; … ; -0.061141774018555924; -0.006156371309829822;;]), layer_4 = (weight = [0.2613273948141624 -0.28971242810429304 … -0.07799442737230644 0.1824980546742902], bias = [0.019172429827216196;;])), c = (layer_1 = (weight = [0.892497595642905; 2.467102349828347; … ; -0.5938730541815221; 0.8136200997729508;;], bias = [-0.25981412359786493; 0.04248924954346983; … ; 0.8308257395536408; -0.20565958071340223;;]), layer_2 = (weight = [0.25715293676392736 -0.07594911136332556 … -0.11926616197590141 -0.8379348752285464; 0.752175415178328 -0.12246281351985946 … -0.6290447720997762 0.8796345198084053; … ; -0.40547875919625803 0.3813046226804944 … -0.5659083969382147 -0.634255535335301; 0.36116509785337 -0.7217367860913075 … 0.6943388864671007 0.2981431093283752], bias = [-0.15897340561596546; 0.24218164779273343; … ; -0.18135422440227664; 0.40814187315447903;;]), layer_3 = (weight = [-0.8831537559946644 0.29675661927495045 … -0.6087868392830994 0.6969329027630532], bias = [0.9742098038792278;;])))

Let's visualize the predictted solution and inferred velocity

using CairoMakie
+u: ComponentVector{Float64}(u = (layer_1 = (weight = [-0.8782367410786727 -0.2130134721699831; 0.04593193258949229 -0.32176059265892654; … ; 0.08039857051512433 -0.2266086390206076; 1.087416687499944 -0.19155513581126166], bias = [-0.2031331529877659; 0.4160923005618338; … ; -0.12241298903523411; -0.052282015451398337;;]), layer_2 = (weight = [-0.05219838124444794 -0.543715775689933 … -0.01879702362298516 -0.10604232960605701; 0.7155426702335106 0.31713525097784057 … 0.4196932183292157 0.023381138705239876; … ; -0.28520747812788705 -0.6326840427188702 … 0.09800858813827179 0.050585171866761584; -0.3146316379007103 0.3094353295029374 … -0.16604953878881648 -0.2479345785449781], bias = [-0.3797069474597654; -0.19855955544460613; … ; -0.1982555008620044; 0.20033078903988674;;]), layer_3 = (weight = [0.441450883105721 0.4025048567889157 … -0.13707471635836174 -0.26408675646020763; -0.4024065926140786 0.39123093733132475 … -0.05704791953144302 0.2277543294932142; … ; -0.3820720866129521 -0.26298099300475747 … 0.19807405050560078 0.05171762873561167; 0.5116260145091547 0.04563044225402554 … 0.45988377277436976 0.5147802655890826], bias = [-0.29175434089782626; -0.08578416585073341; … ; -0.11534375327605803; 0.13884403222971953;;]), layer_4 = (weight = [-0.1697527612675748 0.07696698981501414 … -0.601261049876168 0.5782661654848649], bias = [0.36666178079888684;;])), c = (layer_1 = (weight = [-1.8284961621959452; 1.0254837577810167; … ; -2.6276949646944328; -2.1690603549702985;;], bias = [0.34693469977052793; -1.167420569334675; … ; 0.25006994507000874; -0.6233782742745321;;]), layer_2 = (weight = [-0.40952955815595515 0.6986232077049462 … 0.18169264483933054 -0.16104386127261505; 0.08229804454281914 0.3157290845070805 … 0.5784035055894368 -0.5220838531909279; … ; -0.6050128485932427 -0.5796366983946803 … -0.575172169866919 -0.6392098973364646; 0.36313479684066047 -0.455807603899309 … -0.8916629035872807 0.3556177235583554], bias = [0.0570132051427784; 0.12290082412183116; … ; -0.07965894756215389; -0.06037249851750244;;]), layer_3 = (weight = [0.7043143366627391 0.6668299485242543 … -1.266570373960742 -1.4495123918713688], bias = [0.6642586979087077;;])))

Let's visualize the predictted solution and inferred velocity

using CairoMakie
 
 ts = range(0, 1; length=100)
 xs = range(0, 1; length=100)
@@ -58,4 +58,4 @@
 Colorbar(fig[:, end+1], hm3)
 fig

fig, ax = lines(xs, c_pred)
 lines!(ax, xs, c_true)
-fig

+fig