[Platform] Add platform pluggable framework

[wangxiyuan]

See RFC: #11162

This PR is the first step of the RFC. More PR will be added until reach the RFC goal.

What is Done:

• Added PlatformRegister class. Provide some APIs for plugin to register and initialize out-of-tree platform
• Added an example about how to register a new out-of-tree platform plugin

Next Step:

• Communitor pluggable support.
• Refactor to solve the problem that platform hard code in common place

How to use:

1. The user implement his platform library, for example, the name called vllm_new_device_plugin.
2. install vLLM and the plugin package

   pip install vllm vllm_new_device_plugin
   

3. use vLLM the same as before. Then the inference will run on the new device.

   import vllm
   vllm.LLM(xxx)
   

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[wangxiyuan]

Some test failed due to the current_platform is called before initialization. For example

vllm/vllm/engine/arg_utils.py

Line 117 in 69ba344

block_size: int = 16 if not current_platform.is_hpu() else 128

It needs refactor.

Fixed here: #11225

[wangxiyuan]

Test OK on CPU device. Not sure the TPU CI failure related to this PR. Will take a look farther later.

[DarkLight1337]

I think you will lose type information this way since the methods of Platform aren't annotated in CurrentPlatform.

[wangxiyuan]

yeah, this is a quick work to make the plugin run. Will fill up the code format、code comment、docstrings etc. later

[DarkLight1337]

Is there a reason why you created this CurrentPlatform class? I think it would be simpler to just make refresh_current_platform a function in the global namespace and rename the current global _current_platform to current_platform.

[wangxiyuan]

current_platform is loaded in many place when import vllm. Once it's imported by other module, the value in the other module would not be changed even though it's changed in platform. Here is an example:https://gist.github.com/wangxiyuan/5af186668ccc41ec1d47694146a8fd1f

There are some way to fix the problem:

1. do not import current_platform when import vllm. This way will change a lot of file, it's not recommend.

The file need be changed.

#       modified:   vllm/config.py
#       modified:   vllm/distributed/parallel_state.py
#       modified:   vllm/engine/arg_utils.py
#       modified:   vllm/executor/ray_utils.py
#       modified:   vllm/model_executor/guided_decoding/__init__.py
#       modified:   vllm/model_executor/models/registry.py
#       modified:   vllm/model_executor/utils.py
#       modified:   vllm/spec_decode/metrics.py
#       modified:   vllm/usage/usage_lib.py
#       modified:   vllm/utils.py
#       modified:   vllm/worker/model_runner_base.py
#       modified:   vllm/worker/worker_base.py

2. Like this PR does, create a dynamic load mechanism

[DarkLight1337]

Thanks for the explanation! I understand now. Can you add this explanation to the docstring for future readers?

[wangxiyuan]

sure, will update in next patch set

[MengqingCao]

Tested the latest code on my Tesla T4 GPU, everything works fine. Thx!

[wangxiyuan]

This PR is ready for review now. @DarkLight1337 @simon-mo @youkaichao