[RFC]: Create VllmState to save immutable args in VllmConfig

[MengqingCao]

Motivation.

The VllmConfig becomes huge as more and more features are added in vLLM. But some of that I think is immutable to users, thus not suitable to be as a configure, e.g. worker_cls, and other private args.

Inspired by this comment from Nick, I think creating a dataclass VllmState could help making VllmConfig lighter and safer for users to specify.

Proposed Change.

I think VllmState is similar to VllmConfig, composed by similar member vars as VllmConfig.

@dataclass
class VllmState:
    model_state: ModelState
    cache_state: CacheState
    parallel_state: ParallelState
    scheduler_state: SchedulerState
    device_state: DeviceState
    load_state: LoadState
    lora_state: Optional[LoRAState]
    speculative_state: Optional[SpeculativeState]
    decoding_state: Optional[DecodingState]
    observability_state: Optional[ObservabilityState]
    prompt_adapter_state: Optional[PromptAdapterState]
    quant_state: Optional[QuantizationState]
    compilation_state: CompilationState

And the initial member of VllmState will absorb the immutable args of the corresponding members in VllmConfig as its member variables. And in the meantime, xxxConfig will also be a member of xxxState.

VllmState will be constructed at the begining of the initialize of LLMEngine, and we could just use vllm_state as argument for executor/worker/model_runner/model.
Maybe other modules are missing, plz help me add if you know it

I'm sorting out the immutable vars of all the xxxConfig, I'll update this info in a table later. And there is a tiny example for ParallelState:

@dataclass
class ParallelState:
    parallel_config: ParallelConfig
    placement_group: Optional["PlacementGroup"]
    worker_cls: str
    world_size: int
    rank: int

Feedback Period.

No response

CC List.

cc @youkaichao @njhill @WoosukKwon @DarkLight1337

Any Other Things.

[Help wanted]

There are quite a lot args in VllmConfig, and only a few are marked as private. Is there any other way for me to find out the immutable member vars for users in VllmConfig? Or you can just post the vars you think should put into VllmState here. Thanks!

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.