Skip to content

Commit

Permalink
fixed docstrings
Browse files Browse the repository at this point in the history
  • Loading branch information
pasq-cat committed Jun 23, 2024
1 parent 6fe01a2 commit 0bba488
Showing 1 changed file with 20 additions and 15 deletions.
35 changes: 20 additions & 15 deletions src/utils.jl
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,17 @@ function outdim(model::Chain)::Number
end

"""
empirical_frequency(Y_val,array)
empirical_frequency(Y_cal, sampled_distributions)
FOR REGRESSION MODELS.
Given a calibration dataset (x_t, y_t) for i ∈ {1,...,T} and an array of predicted distributions, the function calculates the empirical frequency
phat_j = {y_t|F_t(y_t)<= p_j, t= 1,....,T}/T, where T is the number of calibration points, p_j is the confidence level and F_t is the
cumulative distribution function of the predicted distribution targeting y_t. The function was suggested by Kuleshov(2018) in https://arxiv.org/abs/1807.00263
cumulative distribution function of the predicted distribution targeting y_t.
The function was suggested by Kuleshov(2018) in https://arxiv.org/abs/1807.00263
Arguments:
Y_val: a vector of values y_t
array: an array of sampled distributions F(x_t) stacked column-wise.
-Y_cal: a vector of values y_t
-sampled_distributions: an array of sampled distributions F(x_t) stacked column-wise.
"""
function empirical_frequency(Y_cal, sampled_distributions)
quantiles = collect(0:0.05:1)
Expand All @@ -67,33 +69,35 @@ function empirical_frequency(Y_cal, sampled_distributions)
end

"""
sharpness(array)
sharpness(sampled_distributions)
FOR REGRESSION MODELS.
Given a calibration dataset (x_t, y_t) for i ∈ {1,...,T} and an array of predicted distributions, the function calculates the
sharpness of the predicted distributions, i.e., the average of the variances var(F_t) predicted by the forecaster for each x_t
The function was suggested by Kuleshov(2018) in https://arxiv.org/abs/1807.00263
Arguments:
sampled_distributions: an array of sampled distributions F(x_t) stacked column-wise.
-sampled_distributions: an array of sampled distributions F(x_t) stacked column-wise.
"""
function sharpness(sampled_distributions)
sharpness = mean(var.(sampled_distributions))
return sharpness
end

"""
empirical_frequency-classification(Y_val,array)
empirical_frequency-classification(y_binary, sampled_distributions)
FOR BINARY CLASSIFICATION MODELS.
Given a calibration dataset (x_t, y_t) for i ∈ {1,...,T} let p_t= H(x_t)∈[0,1] be the forecasted probability.
We group the p_t into intervals I-j for j= 1,2,...,m that form a partition of [0,1]. The function computes
the observed average p_j= T^-1_j ∑_{t:p_t ∈ I_j} y_j in each interval I_j.
The function was suggested by Kuleshov(2018) in https://arxiv.org/abs/1807.00263
Arguments:
y_binary: the array of outputs y_t numerically coded . 1 for the target class, 0 for the negative result.
-y_binary: the array of outputs y_t numerically coded: 1 for the target class, 0 for the negative result.
sampled_distributions: an array of sampled distributions stacked column-wise where in the first row
there is the probability for the target class y_1=1 and in the second row y_0=0.
-sampled_distributions: an array of sampled distributions stacked column-wise so that in the first row
there is the probability for the target class y_1 and in the second row the probability for the null class y_0.
"""
function empirical_frequency_binary_classification(y_binary, sampled_distributions)
pred_avg = collect(range(0; step=0.1, stop=0.9))
Expand All @@ -119,17 +123,18 @@ function empirical_frequency_binary_classification(y_binary, sampled_distributio
end

"""
sharpness-classification(array)
sharpness-classification(y_binary,sampled_distributions)
FOR BINARY CLASSIFICATION MODELS.
We can also assess sharpness by looking at the distribution of model predictions.When forecasts are sharp,
most predictions are close to 0 or 1; unsharp forecasters make predictions closer to 0.5.
most predictions are close to 0 or 1; not sharp forecasters make predictions closer to 0.5.
The function was suggested by Kuleshov(2018) in https://arxiv.org/abs/1807.00263
Arguments:
y_binary: the array of outputs y_t numerically coded . 1 for the target class, 0 for the negative result.
sampled_distributions: an array of sampled distributions stacked column-wise where in the first row
there is the probability for the target class y_1=1 and in the second row y_0=0.
-y_binary: the array of outputs y_t numerically coded . 1 for the target class, 0 for the negative result.
-sampled_distributions: an array of sampled distributions stacked column-wise so that in the first row
there is the probability for the target class y_1 and in the second row the probability for the null class y_0.
"""
function sharpness_classification(y_binary, sampled_distributions)
class_one = sampled_distributions[1, findall(y_binary .== 1)]
Expand Down

0 comments on commit 0bba488

Please sign in to comment.