Skip to content

Commit

Permalink
server : handle "logprobs" field with false value (ggerganov#9871)
Browse files Browse the repository at this point in the history
Co-authored-by: Gimling <[email protected]>
  • Loading branch information
VoidIsVoid and Gimling authored Oct 14, 2024
1 parent 13dca2a commit a89f75e
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions examples/server/utils.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -360,9 +360,9 @@ static json oaicompat_completion_params_parse(

// Handle "logprobs" field
// TODO: The response format of this option is not yet OAI-compatible, but seems like no one really using it; We may need to fix it in the future
if (body.contains("logprobs")) {
if (json_value(body, "logprobs", false)) {
llama_params["n_probs"] = json_value(body, "top_logprobs", 20);
} else if (body.contains("top_logprobs")) {
} else if (body.contains("top_logprobs") && !body.at("top_logprobs").is_null()) {
throw std::runtime_error("top_logprobs requires logprobs to be set to true");
}

Expand Down

0 comments on commit a89f75e

Please sign in to comment.