Negative Variance

[ToennisStef]

I have a problem with the SingleTaskMultiFideltiyGP. When I train the model on the following training data:

train_x = tensor([[0.8292, 1.0000],
        [0.2114, 1.0000],
        [0.2767, 1.0000],
        [0.7806, 2.0000],
        [0.2269, 2.0000],
        [0.2821, 2.0000],
        [0.0266, 3.0000],
        [0.9936, 3.0000],
        [0.6579, 3.0000],
        [0.2719, 0.0000],
        [0.0060, 0.0000],
        [0.7104, 0.0000],
        [0.7519, 0.0000],
        [0.4547, 0.0000],
        [0.7556, 0.0000],
        [0.7525, 0.0000],
        [0.7598, 0.0000]])

train_y = tensor([[ 1.3867],
        [ 1.7781],
        [ 0.6019],
        [ 4.8816],
        [ 3.6580],
        [ 3.1307],
        [ 8.8638],
        [-7.7652],
        [ 4.7179],
        [ 0.0914],
        [-2.7281],
        [ 5.0289],
        [ 6.0057],
        [-0.5264],
        [ 6.0193],
        [ 6.0090],
        [ 6.0171]])

with the model:

model = SingleTaskMultiFidelityGP(train_X=M5_train_x, train_Y=M5_train_y, input_transform=Normalize(d=M5_train_x.shape[-1], bounds=i.combined_bounds_MT), outcome_transform=Standardize(m=1), data_fidelity=-1)
mll = ExactMarginalLogLikelihood(model.likelihood, model)
fit_gpytorch_mll(mll)

I get negative variance values.
The BO loops exits with a Runtime error when I try to optimize a FixedfeatureAcquisition function and infer the next candidate with this model.

The error message is:
RuntimeError: probability tensor contains eitherinf, nan or element < 0

I try to circumvent the problem by setting the cholesky_jitter to a different tolerance value and turning of the fast_computations setting:

with gpytorch.settings.cholesky_jitter(float_value=1e-4, double_value=1e-4), gpytorch.settings.fast_computations(covar_root_decomposition=False, log_prob=False, solves=False):

But i still can't infer a new candidate because of negative variance values.
Does somebody know how to handle this issue?

[esantorella]

1. Does the problem persist if you use double precision?
2. Could you provide a reproducible example, including defining i and optimizing the acquisition function?

[ToennisStef]

1. Yes i am using double precision.
2. And sure i do my best! here is a reproducable example:

import torch
from botorch.models import SingleTaskMultiFidelityGP
from botorch.models.transforms import Standardize, Normalize
from gpytorch.mlls import ExactMarginalLogLikelihood
from botorch import fit_gpytorch_mll
from botorch.optim.fit import fit_gpytorch_mll_torch
from botorch.acquisition.fixed_feature import FixedFeatureAcquisitionFunction
from botorch.acquisition import qExpectedImprovement
from botorch.optim import optimize_acqf


train_x = torch.tensor([[0.4126, 1.0000],
        [0.7846, 1.0000],
        [0.7066, 1.0000],
        [0.3987, 2.0000],
        [0.7349, 2.0000],
        [0.8890, 2.0000],
        [0.8129, 3.0000],
        [0.2790, 3.0000],
        [0.1793, 3.0000],
        [0.1284, 0.0000],
        [0.7539, 0.0000],
        [0.6298, 0.0000],
        [0.2346, 0.0000],
        [0.0000, 0.0000],
        [0.7457, 0.0000],
        [0.7473, 0.0000],
        [1.0000, 0.0000],
        [0.5396, 0.0000],
        [0.3345, 0.0000],
        [0.0322, 0.0000],
        [0.7545, 0.0000],
        [0.2562, 0.0000],
        [0.7520, 0.0000],
        [0.7520, 0.0000],
        [0.7519, 0.0000],
        [0.7519, 0.0000],
        [0.7519, 0.0000]], dtype=torch.float64)

train_y = torch.tensor([[-4.3258e-02],
        [ 2.5140e+00],
        [ 1.8705e+00],
        [ 2.4248e+00],
        [ 5.1550e+00],
        [-3.0696e+00],
        [ 3.9558e+00],
        [ 7.2421e+00],
        [ 8.6170e+00],
        [ 9.5370e-01],
        [ 6.0149e+00],
        [ 1.2806e+00],
        [ 3.2478e-01],
        [-3.0272e+00],
        [ 5.9525e+00],
        [ 5.9695e+00],
        [-1.5830e+01],
        [-9.4679e-01],
        [-7.4191e-07],
        [-1.4842e+00],
        [ 6.0167e+00],
        [ 1.7088e-01],
        [ 6.0064e+00],
        [ 6.0061e+00],
        [ 6.0058e+00],
        [ 6.0056e+00],
        [ 6.0054e+00]], dtype=torch.float64)


bounds = torch.tensor([[0., 0.],
        [1., 3.]], dtype=torch.float64)

bounds_x = torch.tensor([[0.],
        [1.]], dtype=torch.float64)

model = SingleTaskMultiFidelityGP(
        train_X=train_x, 
        train_Y=train_y, 
        input_transform=Normalize(d=train_x.shape[-1], bounds=bounds) , 
        outcome_transform=Standardize(m=1), 
        data_fidelity=-1
        )

mll = ExactMarginalLogLikelihood(model.likelihood, model)
try:
    fit_gpytorch_mll(mll)
except Exception as e:
    print(e)
    print("fit_gpytorch_mll failed")
    print("trying fit_gpytorch_mll_torch")
    fit_gpytorch_mll_torch(mll)
    
acq_func = FixedFeatureAcquisitionFunction(acq_function=qExpectedImprovement(model=model, best_f=train_y[train_x[:,1]==0].max()), 
                        d=1+1, 
                        columns=[1],
                        values=[0],
                        )


candidates, _ = optimize_acqf(
        acq_function=acq_func,
        bounds=bounds_x,
        q=1,
        num_restarts=20,
        raw_samples=512,
    )

the error message:

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Also the version information of botorch, gpytorch and torch:

botorch:  0.9.4
gpytorch: 1.11
torch: 2.1.1+cu118

[esantorella]

Thanks for the repro!

So, two things. One is that your setup seems a little off: 0 is the lowest fidelity, so you're maximizing expected improvement at the lowest fidelity. As per your other discussion post, you want to find the optimum at the highest fidelity. If you want to evaluate at the lowest fidelity and optimize for the highest, using project_to_target_fidelity as in the Multi-Fidelity tutorial will help. If you want to evaluate at the highest fidelity, you'd just change the fixed feature to 3. Also, you'll get better numerical performance by using the log variants of acquisition functions, such as qLogExpectedImprovement, or just LogExpectedImprovement if you're only generating one candidate at a time. I can reproduce your bug, but it goes away if you evaluate at the highest fidelity or use LogExpectedImprovement.

The second thing is that even with qLogExpectedImprovement, I'm still seeing the optimization fail. Numerical issues do arise in BoTorch, but they tend to make the optimization perform poorly rather than making it crash entirely. This seems like a bug, so I'm going to open an issue for that.

[ToennisStef]

I encoded my data that 0 is the highest fidelity.

Is 0 the lowest fidelity in the SingleTaskMultiFidelityGP model and does it change the whole inference of the model if i encode it that way? It is not stated in the API reference that there is this strict fidelity ordering (0 beeing lowest fidelity), and i thought it would not matter if i encode my data in a way that 0 is the highest fidelity and 3 is the lowest fidelity.
I did not know that there is this direct encoding in the SingleTaskMultiFidelityGP model that encodes 0 to be lowest fidelity. I thought it would just encode this "knowledge transfer" between the fidelities with the fidelities having an ordering but no specification on here the highest and where the lowest fidelity is.

Thank you i'll incorporate the variants of the acuisition function! And thank you for looking into the issue!

[esantorella]

Yeah, the documentation is not great here. I confirmed that the order in which the fidelities is encoded matters by running some simulations:

Repeating this many times, using a higher fidelity for the less noisy data tended to perform much better.

[ToennisStef]

Thank you so much! That is incredible insight! I would have definitely overlooked that!
I'll have to change my setup then!

Quick follow-up question i will have to fix the FixedFeatureAcquisitionFunction value to the value of my highest fidelity (in my case 4- discrete fidelities with the highest value being 3 results in -> FixedFeatureAcquisitionFunction(..., value=[3]) ), correct?
It does not matter that I normalize the inputs with the input_transform (transforming the highest fidelity value to being 1), correct?
I am getting a bit confused with the Normalization of the inputs.