Skip to content

Commit

Permalink
test: add surrogate model tests
Browse files Browse the repository at this point in the history
  • Loading branch information
fjebaker committed May 20, 2024
1 parent d93e464 commit 2beafab
Show file tree
Hide file tree
Showing 2 changed files with 55 additions and 9 deletions.
14 changes: 5 additions & 9 deletions docs/src/surrogate-models.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
# Surrogate models

```@setup surrogate_example
using SpectralFitting
using Plots
ENV["GKSwstype"]="nul"
Plots.default(show=false)
```

Surrogate models allow you to create fast or memory efficient approximations of model components, or assist in optimizing some objective function directly. SpectralFitting uses the [Surrogates.jl](https://github.com/SciML/Surrogates.jl) library of models, that yields pure-Julia surrogate models. Consequently, surrogate models also permit use of automatic differentiation in fitting, and are therefore powerful tools for improving fitting performance.

## Surrogates overview
Expand Down Expand Up @@ -42,6 +35,8 @@ Before we start, let us discuss a number of benefits the use of surrogate models
The performance of this model represents its complexity.

```@example surrogate_example
using SpectralFitting
energy = collect(range(0.1, 20.0, 200))
model = XS_PhotoelectricAbsorption()
Expand Down Expand Up @@ -91,6 +86,7 @@ length(harness.surrogate.x)
We can examine how well our surrogate reconstructs the model for a given test parameter:

```@example surrogate_example
using Plots
import Random # hide
Random.seed!(1) # hide
# random test value
Expand Down Expand Up @@ -130,7 +126,7 @@ Tight. We can also inspect the memory footprint of our model:

```@example surrogate_example
# in bytes
Base.summarysize(surrogate)
Base.summarysize(harness)
```
This may be reduced by lowering `maxiters` in [`optimize_accuracy!`](@ref) at the cost of decreasing faithfulness. However, compare this to the Fortran tabulated source file in the XSPEC source code, which is approximately 224 Kb -- about 15x larger. The surrogate models are considerably more portable at this level.

Expand All @@ -146,7 +142,7 @@ nothing # hide
Now that we have the surrogate model, we use [`SurrogateSpectralModel`](@ref) to wrap it into an [`AbstractSpectralModel`](@ref). The constructor also needs to know the model kind, have a copy of the model parameters, and know which symbols to represent the parameters with.

```@example surrogate_example
sm = make_model(harness)
sm = @code_warntype make_model(harness)
```

We can now use the familiar API and attempt to benchmark the performance:
Expand Down
50 changes: 50 additions & 0 deletions test/models/test-surrogate-models.jl
Original file line number Diff line number Diff line change
@@ -1 +1,51 @@
using Test
using SpectralFitting
import Random

lower_bounds = (0.1, 1e-3)
upper_bounds = (20.0, 3.0)

energy = collect(range(0.1, 20.0, 100))
model = XS_PhotoelectricAbsorption()

flux = similar(energy)[1:end-1]

surrogate = make_surrogate_function(
(x, y) -> SpectralFitting.RadialBasis(x, y, lower_bounds, upper_bounds),
model,
lower_bounds,
upper_bounds,
)

# number of points the surrogate has been trained on
length(surrogate.surrogate.x)

Random.seed!(1)

nh = 2.0
model.ηH.value = nh

f = invokemodel(energy, model)

= map(energy[1:end-1]) do e
v = (e, nh)
surrogate.surrogate(v)
end

@test sum(f .- f̂) -0.299 atol=1e-3

optimize_accuracy!(surrogate; maxiters = 50)

= map(energy[1:end-1]) do e
v = (e, nh)
surrogate.surrogate(v)
end

@test sum(f .- f̂) 0.196 atol=1e-3

sm = make_model(surrogate)

# smoke test these
invokemodel(collect(range(0.01, 10.0, 100)), sm)
new_model = sm * PowerLaw()
invokemodel(collect(range(0.01, 10.0, 1000)), new_model)

0 comments on commit 2beafab

Please sign in to comment.