Worker running code on the main thread (#22)

README.md

@@ -27,10 +27,7 @@ working as expected
 
 ### New content
 
-[pyconsole](/src/packages/pyconsole/README.md) demonstrates the integration of a
-nice python console to 3dsMax.
-
-![Console](/src/packages/pyconsole/doc/pyconsole.png)
+[mxthread](/src/packages/mxthread/README.md) demonstrates a worker running code on the main thread.
 
 ### Samples
 
@@ -49,13 +46,14 @@ the Python version in the best Python way known to us. An example of this is tha
 - Output Object Data to File [speedsheet](/src/packages/speedsheet/README.md)
 - Create a quick video preview [quickpreview](/src/packages/quickpreview/README.md)
 - Access the Z-Depth Channel [zdepthchannel](/src/packages/zdepthchannel/README.md)
-- Integrate a Python Console [pyconsole](/src/packages/pyconsole/README.md)
 
 ## Python Examples that don't come from maxscript howtos
 
 - Update a progressbar from a Python thread [threadprogressbar](/src/packages/threadprogressbar/README.md)
 - Create a single instance modal dialog [singleinstancedlg](/src/packages/singleinstancedlg/README.md)
 - Add menu items to open documentation pages in the web browser [inbrowserhelp](/src/packages/inbrowserhelp/README.md)
+- Integrate a Python Console [pyconsole](/src/packages/pyconsole/README.md)
+- Run code on thre main thread [mxthread](/src/packages/mxthread/README.md)
 
 ## Python Samples
 

src/packages/mxthread/LICENSE

@@ -0,0 +1 @@
+Copyright (c) 2020 Autodesk, all rights reserved.

src/packages/mxthread/README.md

@@ -0,0 +1,176 @@
+# HowTo: mxthread
+
+Can a python thread queue some work to be executed by the main thread, wait for the execution
+to complete and then get the result of running this code?
+
+This question was asked on a forum. I will now try to show how this can work.
+
+
+### Overview of how this can be solved
+
+3ds Max integrates Qt and Qt provides many tools to deal with threads. The method presented
+here may not be the simplest or the best but it is entirely based on Qt mechanisms.
+
+
+Disclaimer: It is difficult to work with threads and even more difficult to work with
+threads in 3dsMax (ex: prints are not shown in the listener window). So I would not necessarily
+recommand this kind of approach. It can nevertheless be useful to adapt a library to multiple
+DCCs (or other reasons). 
+
+#### Slots and Signals
+If an object that has a main thread affinity creates a slot, and that a worker 
+emits a signal on this slot, the signal will be processed on the main thread. This does
+not provide a way for the worker to retrieve the result of running this code.
+
+#### QWaitCondition
+A thread can wait on a QWaitCondition that another thread will raise.
+
+#### Approach
+
+We provide a way to bundle a function to execute in an object that is passed by a signal
+to a slot that is serviced by the main thread. The bundle/payload also contains a wait
+condition that the main thread raises when the payload has been executed. The return value
+of the code is added to the payload so that the worker can retrieve it. Exceptions are also
+propagated.
+
+## The "test case"
+
+To make things clearer, here is the sample program that we will use to test the maxthread
+module.
+
+We import 2 funcitons from maxthread: `on_main_thread` which allows to decorate a function
+so that it will always run on the main thread, and `main_thread_print` which is an already
+decorated function that prints on the main thread.
+
+The run function of the Worker (that is a QThread) is what the worker thread does: it essentially
+run stuff on the main thread.
+
+This sample can be saved in a "testmxthread.py" file and then run in 3dsMax.
+
+
+```python
+from mxthread import on_main_thread, main_thread_print
+from pymxs import runtime as rt
+from PySide2.QtCore import QThread
+
+
+class Worker(QThread):
+    """
+    Worker thread doing various things with maxtrhead.
+    """
+    def __init__(self, name="worker"):
+        QThread.__init__(self)
+        self.setObjectName(name)
+
+    def run(self):
+        # use a function that was already decorated with on_main_thread
+        main_thread_print(f"hello from thread {self.objectName()}")
+
+        # create our own function decorated with on_main_thread
+        @on_main_thread
+        def do_pymxs_stuff():
+            print("resetting the max file")
+            # reset the max file (so that the scene is empty)
+            rt.resetMaxFile(rt.name("noprompt"))            
+            # we are on main thread so we can use print and it will work
+            print("creating 3 boxes on main thread")
+            # pymxs stuff can only work on the main thread. Well no problem we are on the main thread:
+            rt.box(width=1, height=1, depth=1, position=rt.Point3(0,0,0))
+            rt.box(width=1, height=1, depth=1, position=rt.Point3(0,0,2))
+            rt.box(width=1, height=1, depth=1, position=rt.Point3(0,0,4))
+
+            # make 3ds Max aware that the views are dirtied
+            rt.redrawViews()
+
+            return 3
+        # call our main thread function 
+        res = do_pymxs_stuff()
+        main_thread_print(f"our main thread function returned {res}")
+
+        # create another function that will throw something
+        # (to show that exceptions are propagated)
+        @on_main_thread
+        def do_faulty_stuff():
+            # we are on main thread so we can use print and it will work
+            a = 2
+            b = 0
+            return a / b
+        try:
+            res = do_faulty_stuff()
+            main_thread_print(f"The function will fail, this will never be displayed")
+        except Exception as e:
+            main_thread_print(f"our main thread function raised:  {e}")
+
+
+        # use a lambda instead
+        on_main_thread(lambda : print("hello from lambda"))()
+
+        main_thread_print(f"we are done, the sample ran correctly!")
+
+
+# Name the main thread
+QThread.currentThread().setObjectName("main_thread")
+# create a worker
+worker = Worker("worker_thread")
+worker.start()
+# Note: we cannot wait this worker here. This will create a deadlock.
+# The worker executes stuff on the main thread and we are on the main thread.
+# But to convince ourselves, we can print something here and the man thread
+# calls initiated by the worker will all happen after this
+print("--- Worker thread calls to the main thread will run after this")
+
+```
+
+Running this sample will display this:
+
+```
+--- Worker thread calls to the main thread will run after this
+hello from thread worker_thread
+resetting the max file
+creating 3 boxes on main thread
+our main thread function returned 3
+our main thread function raised:  division by zero
+hello from lambda
+we are done, the sample ran correctly!
+```
+
+## How to use mxthread
+
+To create a function that will be executed on the main thread (no matter what thread calls the function),
+the function needs to be decorated with `@on_main_thread`, as shown here:
+
+```
+@on_main_thread
+def do_faulty_stuff():
+    # we are on main thread so we can use print and it will work
+    a = 2
+    b = 0
+    return a / b
+```
+
+Decorated functions can return values and throw exceptions and in both cases this
+behaves normally from the thread that calls the function.
+
+### gotcha
+
+The most important gotcha is that the main thread cannot wait for its worker (this
+will create a deadlock). The worker should also be kept in a variable until it completes.
+
+## The implementation
+
+The implementation of mxthread can be found in [mxtrhead/__init__.py](mxthread/__init__.py).
+The code is abundantly commented.
+
+- `on_main_thread` is a decorator that makes a function runnable on the main thread
+
+- `main_thread_print` is a function that uses `on_main_thread` to make the main thread print
+something (in 3dsMax print does not work from a worker thread)
+
+- RunnableWaitablePayload is an object that is passed by a worker thread using the `RUNNABLE_PAYLOAD_SIGNAL`
+to the `RUNNABLE_PAYLOAD_SLOT` that runs on the main thread. This payload object contains
+the function that needs to be called on the main thread. After the function runs it contains
+the return value of the function or an exception if an exception was raised. It also contains
+a QWaitCondition. The caller worker waits for this QWaitCondition and the main thread triggers
+it when the function has been executed.
+
+

src/packages/mxthread/mxthread/__init__.py

@@ -0,0 +1,127 @@
+"""
+    Provide a way to decorate python functions so that they are always executed in
+    the main thread of 3dsMax. The functions can throw exceptions and return values
+    and this is propagated to the caller in the thread.
+"""
+import sys
+import os
+import functools
+from PySide2.QtCore import QObject, Slot, Signal, QThread, QMutex, QWaitCondition, QTimer
+from PySide2.QtWidgets import QApplication
+#pylint: disable=W0703,R0903,R0201
+
+class RunnableWaitablePayload():
+    """
+    Wrap a function call as a payload that will be emitted
+    to a slot owned by the main thread. The main thread will execute
+    the function call and package the return value in the payload.
+    The payload also contains a wait condition that the main thread will
+    signal when the payload was executed. The worker thread (that creates
+    the payload) will wait for this wait condition and then retrieve the
+    return value from the function.
+    """
+    def __init__(self, todo):
+        """
+        Initialize the payload.
+        todo is the function to execute, that takes no arguments but
+        that can return a value.
+        """
+        self.todo = todo
+        self.todo_exception = None
+        self.todo_return_value = None
+        self.wcnd = QWaitCondition()
+        self.mutex = QMutex()
+
+    def wait_for_todo_function_to_complete_on_main_thread(self):
+        """
+        Wait for the pending operation to complete
+        Returns the value returned by the todo function (the function
+        to execute in this payload).
+        """
+        self.mutex.lock()
+        # queue the thing to do on the main thread
+        RUNNABLE_PAYLOAD_SIGNAL.sig.emit(self)
+        # while waiting the QWaitCondition unlocks the mutex
+        # and relocks it when the wait completes
+        self.wcnd.wait(self.mutex)
+        self.mutex.unlock()
+        # if the payload failed, propagate this to the thread
+        if self.todo_exception:
+            raise self.todo_exception
+        # otherwise return the result
+        return self.todo_return_value
+
+    def run_todo_function(self):
+        """
+        Run the todo function of payload.
+        This will add the return of the todo function to the payload as "todo_return_value".
+        """
+        self.mutex.lock()
+        try:
+            self.todo_return_value = self.todo()
+        except Exception as exception:
+            self.todo_exception = exception
+        self.wcnd.wakeAll()
+        self.mutex.unlock()
+
+class RunnableWaitablePayloadSignal(QObject):
+    """
+    Creates a signal that can be used to send RunnableWaitablePyaloads to
+    the main thread for execution.
+    """
+    sig = Signal(RunnableWaitablePayload)
+
+# Create the Slots that will receive signals
+class PayloadSlot(QObject):
+    """
+    Slot for function submission on the main thread.
+    """
+    def __init__(self):
+        """
+        An object that owns a slot.
+        This object's affinity is the main thread so that signals it receives
+        will run on the main thread.
+        """
+        QObject.__init__(self)
+        self.moveToThread(QApplication.instance().thread())
+
+    @Slot(RunnableWaitablePayload)
+    def run(self, ttd):
+        """
+        Run the slot payload.
+        """
+        ttd.run_todo_function()
+
+RUNNABLE_PAYLOAD_SLOT = PayloadSlot()
+
+# connect the payload signal to the payload slot
+RUNNABLE_PAYLOAD_SIGNAL = RunnableWaitablePayloadSignal()
+RUNNABLE_PAYLOAD_SIGNAL.sig.connect(RUNNABLE_PAYLOAD_SLOT.run)
+
+def run_on_main_thread(todo):
+    """
+    Run code on the main thread.
+    Returns the return value of the todo code. If this is called
+    from the main thread, todo is immediately called.
+    """
+    if QThread.currentThread() is QApplication.instance().thread():
+        return todo()
+    ttd = RunnableWaitablePayload(todo)
+    return ttd.wait_for_todo_function_to_complete_on_main_thread()
+
+def on_main_thread(func):
+    """
+    Decorate a function to make it always run on the main thread.
+    """
+    # preserve docstring of the wrapped function
+    @functools.wraps(func)
+    def decorated(*args, **kwargs):
+        return run_on_main_thread(lambda: func(*args, **kwargs))
+    return decorated
+
+@on_main_thread
+def main_thread_print(*args, **kwargs):
+    """
+    Print on the main thread.
+    """
+    return print(*args, **kwargs)

src/packages/mxthread/setup.py

@@ -0,0 +1,15 @@
+import setuptools
+
+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
+setuptools.setup(
+    name="mxthread-autodesk",
+    version="0.0.1",
+    description="mxthread sample",
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://git.autodesk.com/windish/maxpythontutorials",
+    packages=setuptools.find_packages(),
+    python_requires='>=3.7'
+)