Support async callbacks for popup

[ahuang11]

Closes #6388

Screen.Recording.2024-09-26.at.8.41.32.AM.mov

[codecov]

Codecov Report

Attention: Patch coverage is 95.23810% with 3 lines in your changes missing coverage. Please review.

Project coverage is 88.49%. Comparing base (c233dd1) to head (9906a4a).
Report is 30 commits behind head on main.

Files with missing lines	Patch %	Lines
holoviews/tests/ui/bokeh/test_callback.py	81.25%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #6390    +/-   ##
========================================
  Coverage   88.49%   88.49%            
========================================
  Files         323      323            
  Lines       68180    68603   +423     
========================================
+ Hits        60338    60713   +375     
- Misses       7842     7890    +48     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ahuang11]

Maybe @philippjfr has more insights on self._skipped_partial_event

[ahuang11]

It works in my notebook, but I think when I serve it, I get this error with the async executor changes:

ERROR:tornado.application:Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOMainLoop object at 0x1069b5db0>>, <Task finished name='Task-92' coro=<PopupMixin._process_selection_event() done, defined at [/Users/ahuang/repos/holoviews/holoviews/plotting/bokeh/callbacks.py:676](https://file+.vscode-resource.vscode-cdn.net/Users/ahuang/repos/holoviews/holoviews/plotting/bokeh/callbacks.py:676)> exception=RuntimeError('_pending_writes should be non-None when we have a document lock, and we should have the lock when the document changes')>)
Traceback (most recent call last):
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/tornado/ioloop.py", line 738, in _run_callback
    ret = callback()
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/tornado/ioloop.py", line 762, in _discard_future_result
    future.result()
  File "/Users/ahuang/repos/holoviews/holoviews/plotting/bokeh/callbacks.py", line 728, in _process_selection_event
    self._panel.visible = True
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/core/has_props.py", line 336, in __setattr__
    return super().__setattr__(name, value)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/core/property/descriptors.py", line 332, in __set__
    self._set(obj, old, value, setter=setter)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/core/property/descriptors.py", line 620, in _set
    self._trigger(obj, old, value, hint=hint, setter=setter)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/core/property/descriptors.py", line 698, in _trigger
    obj.trigger(self.name, old, value, hint, setter)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/model/model.py", line 571, in trigger
    super().trigger(descriptor.name, old, new, hint=hint, setter=setter)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/util/callback_manager.py", line 186, in trigger
    self.document.callbacks.notify_change(cast(Model, self), attr, old, new, hint, setter, invoke)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 251, in notify_change
    self.trigger_on_change(event)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 423, in trigger_on_change
    invoke_with_curdoc(doc, invoke_callbacks)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 453, in invoke_with_curdoc
    return f()
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 422, in invoke_callbacks
    cb(event)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/document/callbacks.py", line 278, in <lambda>
    self._change_callbacks[receiver] = lambda event: event.dispatch(receiver)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/document/events.py", line 352, in dispatch
    super().dispatch(receiver)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/document/events.py", line 218, in dispatch
    cast(DocumentPatchedMixin, receiver)._document_patched(self)
  File "/Users/ahuang/miniconda3/envs/holoviews/lib/python3.10/site-packages/bokeh/server/session.py", line 244, in _document_patched
    raise RuntimeError("_pending_writes should be non-None when we have a document lock, and we should have the lock when the document changes")
RuntimeError: _pending_writes should be non-None when we have a document lock, and we should have the lock when the document changes

[ahuang11]

Ok I'm stuck; can't seem to figure a way to get around that document lock; works in notebook, but not server

[ahuang11]

Nice! It works in both notebook and serve

Just for my own knowledge, wondering what document.session_context is and why add_next_tick_callback is needed vs state.execute?

if self.plot.document.session_context:
                self.plot.document.add_next_tick_callback(self._process_selection_partial_event)
            else:
                state.execute(self._process_selection_partial_event)

[philippjfr]

Just for my own knowledge, wondering what document.session_context is and why add_next_tick_callback is needed vs state.execute?

I think state.execute should swap it's logic so it does use add_next_tick_callback for coroutine functions when schedule=True. doc.session_context is an indicator that we are running in the context of a server session (though it's not quite right and should actually use doc.session_context and doc.session_context.server_context.

[ahuang11]

Okay I didn't quite catch that besides doc.session_context means server while self.plot.comm means notebook.

However, I don't quite understand what "should actually use doc.session_context and doc.session_context.server_context"

And for this,

state.execute should swap it's logic so it does use add_next_tick_callback for coroutine functions when schedule=True

are you saying that we should drop state.execute() and replace with add_next_tick_callback? How do you check schedule=True?

[philippjfr]

However, I don't quite understand what "should actually use doc.session_context and doc.session_context.server_context"

I meant that the code here should be updated to check that both are present.

are you saying that we should drop state.execute() and replace with add_next_tick_callback? How do you check schedule=True?

No I'm saying that pn.state.execute should be changed so that, if you pass schedule=True it uses add_next_tick_callback internally.