Code

logits API support#67

Code

Merged

zhiyuan8 merged 25 commits intomainNexaAI/nexa-sdk:mainfrom qingying-logitsNexaAI/nexa-sdk:qingying-logitsCopy head branch name to clipboardSep 12, 2024

+177-41

Commits

Commits on Aug 29, 2024

• implement logits and logprobs output, add debug print statements

  qmeng222

  committed
  d4e88d3

Commits on Aug 30, 2024

• debug (in progress)

  qmeng222

  committed
  f833649

• Merge branch 'main' of https://github.com/NexaAI/nexa-sdk into qingying-logits

  qmeng222

  committed
  946da23

Commits on Sep 2, 2024

• implement the expected structure in response body

  qmeng222

  committed
  e69c81c

Commits on Sep 3, 2024

• debug null logprobs in Nexa SDK server

  qmeng222

  committed
  20f10a6

• add additional intermediate print statements for enhanced debugging

  qmeng222

  committed
  e79a043

• handle and accumulate logprobs for completion format

  qmeng222

  committed
  3bad104

Commits on Sep 4, 2024

• clean code

  qmeng222

  committed
  09f0d5c

• update NexaTextInference class to handle logprobs

  qmeng222

  committed
  12c1904

• add logprobs functionality to Llama class

  qmeng222

  committed
  7c15aa0

• troubleshoot the CI issue

  qmeng222

  committed
  c4cd6fb

• solve pr conflict: update completion chunk format for streaming

  qmeng222

  committed
  5a090c9

• Merge remote-tracking branch 'origin/main' into qingying-logits

  qmeng222

  committed
  9fc5b98

Commits on Sep 5, 2024

• fix logprobs handling consistency in llama.py

  qmeng222

  committed
  fcdfc6c

• implement logprobs and top_logprobs functionality

  qmeng222

  committed
  21d5cbc

• fix TypeError in create_completion by conditionally passing top_logprobs

  qmeng222

  committed
  89780b9

• add top_logprobs parameter to Llama.create_completion method

  qmeng222

  committed
  9ba5684

• ensure logprobs is being passed as a boolean throughout the call chain

  qmeng222

  committed
  49615ec

• set logprobs: true and top_logprobs: 4 by default for both the /v1/completions and /v1/chat/completions endpoints

  qmeng222

  committed
  1c77636

• remove redundant logprobs CLI argument

  qmeng222

  committed
  a37f501

• comment out top_logprobs CLI argument

  qmeng222

  committed
  329993f

Commits on Sep 10, 2024

• remove intermediate printouts

  qmeng222

  committed
  771b301

Commits on Sep 11, 2024

• minor changes: uncomment and reset defaults

  qmeng222

  committed
  ce20a9e

• clean code

  qmeng222

  committed
  e1ee1b6

• Merge branch 'main' of https://github.com/NexaAI/nexa-sdk into qingying-logits

  JoyboyBrian

  committed
  08ba622