Atexit-safe Close

[yellowhatter]

Describe the feature

This issue to be done some day.

Here we used to have a discussion on nolocal Close implementation: #1632
There is an option to make Close nolocal and thus callable from atexit handler. However, currently we decided to merge mentioned Close PR as it is, without making close to be callable in atexit handler.
This issue is a reminder that we still have that option.