Code

Adding cascade inference to vLLM#10011

Code

Open

raywanb wants to merge 17 commits intovllm-project:mainvllm-project/vllm:mainfrom raywanb:ray/cascade_infer_eagerraywanb/vllm:ray/cascade_infer_eagerCopy head branch name to clipboard

+770-202

Commits

Commits on Nov 1, 2024

• eager mode

  raywanb

  committed
  ef7b292

  Copy full SHA for ef7b292

• test

  raywanb

  committed
  78f0cbe

  Copy full SHA for 78f0cbe

• clean up

  raywanb

  committed
  3f6c458

  Copy full SHA for 3f6c458

Commits on Nov 4, 2024

• cleanup

  raywanb

  committed
  84b8be9

  Copy full SHA for 84b8be9

• cleanup code

  raywanb

  committed
  7babe4f

  Copy full SHA for 7babe4f

Commits on Nov 5, 2024

• Merge remote-tracking branch 'origin/main' into ray/cascade_infer_eager

  raywanb

  committed
  23dfbc2

  Copy full SHA for 23dfbc2

Commits on Nov 19, 2024

• adds chunked prefill support, but haven't cleaned up code

  raywanb

  committed
  a908e77

  Copy full SHA for a908e77

• change layer to level

  raywanb

  committed
  9064ec3

  Copy full SHA for 9064ec3

• remove comment

  raywanb

  committed
  2637e09

  Copy full SHA for 2637e09

• merge

  raywanb

  committed
  b9508c1

  Copy full SHA for b9508c1

• typo

  raywanb

  committed
  29950bc

  Copy full SHA for 29950bc

Commits on Nov 22, 2024

• formatting

  raywanb

  committed
  555de93

  Copy full SHA for 555de93

• sliding window

  raywanb

  committed
  3616ac6

  Copy full SHA for 3616ac6

• typing

  raywanb

  committed
  8e763f0

  Copy full SHA for 8e763f0

Commits on Nov 29, 2024

• add warning for logits and scale

  raywanb

  committed
  81d3f43

  Copy full SHA for 81d3f43

Commits on Dec 29, 2024

• Merge remote-tracking branch 'origin' into ray/cascade_infer_eager

  raywanb

  committed
  ed463da

  Copy full SHA for ed463da

Commits on Dec 31, 2024

• Merge remote-tracking branch 'origin' into ray/cascade_infer_eager

  raywanb

  committed
  42c931d

  Copy full SHA for 42c931d