acryl_datahub_dagster_plugin is incompatible with Dagster 1.9.5

[jtv8]

Describe the bug
Deploying a user code deployment using acryl_datahub_dagster_plugin with Dagster 1.9.5 results in a fatal error with the following log trace:

2024-12-13 15:14:21 +0000 - dagster.code_server - INFO - Starting Dagster code server for module dataplatform_databricks_demo on port 3030 in process 1
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/code_pointer.py", line 134, in load_python_module
    return importlib.import_module(module_name)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/src/dataplatform_databricks_demo/__init__.py", line 1, in <module>
    from .dagster_databricks_step_launcher import defs
  File "/src/dataplatform_databricks_demo/dagster_databricks_step_launcher.py", line 19, in <module>
    from datahub_dagster_plugin.sensors.datahub_sensors import (
  File "/usr/local/lib/python3.10/site-packages/datahub_dagster_plugin/sensors/datahub_sensors.py", line 31, in <module>
    from dagster._core.definitions.sensor_definition import (
ImportError: cannot import name 'RawSensorEvaluationFunctionReturn' from 'dagster._core.definitions.sensor_definition' (/usr/local/lib/python3.10/site-packages/dagster/_core/definitions/sensor_definition.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/dagster", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/dagster/_cli/__init__.py", line 50, in main
    cli(auto_envvar_prefix=ENV_PREFIX)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/dagster/_cli/api.py", line 793, in grpc_command
    api_servicer = DagsterApiServer(
  File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/server.py", line 421, in __init__
    self._loaded_repositories: Optional[LoadedRepositories] = LoadedRepositories(
  File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/server.py", line 255, in __init__
    loadable_targets = get_loadable_targets(
  File "/usr/local/lib/python3.10/site-packages/dagster/_grpc/utils.py", line 50, in get_loadable_targets
    else loadable_targets_from_python_module(module_name, working_directory)
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/workspace/autodiscovery.py", line 30, in loadable_targets_from_python_module
    module = load_python_module(
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/code_pointer.py", line 139, in load_python_module
    raise DagsterImportError(
dagster._core.errors.DagsterImportError: Encountered ImportError: `cannot import name 'RawSensorEvaluationFunctionReturn' from 'dagster._core.definitions.sensor_definition' (/usr/local/lib/python3.10/site-packages/dagster/_core/definitions/sensor_definition.py)` while importing module dataplatform_databricks_demo. Local modules were resolved using the working directory `/src`. If another working directory should be used, please explicitly specify the appropriate path using the `-d` or `--working-directory` for CLI based targets or the `working_directory` configuration option for workspace targets. 

To Reproduce
Steps to reproduce the behavior:

1. Install Dagster 1.9.5 on any platform
2. Deploy the sample user code deployment at https://datahubproject.io/docs/lineage/dagster/, using dagster==1.9.5 as a dependency

Expected behavior
The user code deployment should run successfully and be visible in the Dagster UI.

Adding an asset to the user code deployment and materializing it from the Dagster UI should result in a corresponding dataset being created in DataHub (this may already have been broken in previous Dagster 1.9.x releases but I cannot verify for sure).

Screenshots
N/A

Desktop (please complete the following information):
N/A

Additional context
The root cause appears to be relying on non-public types in dagster._core that have been eliminated during refactoring. Specifically RawSensorEvaluationFunctionReturn was replaced by SensorReturnTypesUnion in this commit: dagster-io/dagster@41b30b6

[hsheth2]

Fixed by #12121.