What are crucial differences between qEHVI and qNEHVI.

[taikichi-pc]

Thank you for your kindness.
And you always reply quickly and concretely. I appreciated it.

Now I'd like to understand about acquisition function.
It is conceptual and theoretical differences between qN~ and q~.

When I use qEI or qEHVI and so on, I can select qNEI or qNEHVI.
In your tutorial, if q-batch>1, qN~ is recommended.

If you've already provided about this info., I'm sorry and let me know the docs, issues, discussions.

Thank you.

[eytan]

Hi, Thanks for the kind words! The “N” variants are designed to handle variable amounts of noise, including settings where there is a low signal to noise ratio like “A/B” tests in the internet industry. We developed NEI ( https://projecteuclid.org/journalArticle/Download?urlId=10.1214%2F18-BA1110) after observing that off the shelf BayesOpt packages would tend to resample the same points over and over again, especially in the batch setting. EI is defined as the expected improvement over the best observed function value. This makes sense for deterministic functions, but if observations are noisy, then you really know what your f_best is. What NEI does is integrate over possible values of f_best. qNEHVI follows a similar idea: the Pareto front observed is not the true Pareto front, but some noisy realization of it. It similarly becomes important to integrate over your uncertainty in the (in sample) PF when computing the hyper volume improvement. qNEHVI utilizes some nice tricks that makes it much more scalable than qEHVI wrt the batch size (parallelism), q. Empirically the performance is at least as good in the noiseless case and better than any other AF in the noisy case. For this reason we’d recommend qNEHVI as the default multi objective BO algorithm to use (and we default to it in Ax). For a more in depth explanation you can watch Sam give a talk on qNEHVI https://youtu.be/XOYo_TNo3Gw or read the paper https://arxiv.org/pdf/2105.08195.pdf

…

On Tue, Feb 7, 2023 at 10:50 PM kotaro_chato ***@***.***> wrote: Thank you for your kindness. And you always reply quickly and concretely. I appreciated it. Now I'd like to understand about acquisition function. It is conceptual and theoretical differences between qN~ and q~. When I use qEI or qEHVI and so on, I can select qNEI or qNEHVI. In your tutorial, if q-batch>1, qN~ is recommended. If you've already provided about this info., I'm sorry and let me know the docs, issues, discussions. Thank you. — Reply to this email directly, view it on GitHub <#1661>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAW34K53EN2GBLDA4KYHSTWWMJZ7ANCNFSM6AAAAAAUUX6ORI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[taikichi-pc]

Thank you so much. It make sense to me!