Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Numerically stable logit pred #35

Merged
merged 10 commits into from
Nov 27, 2024
Merged

Numerically stable logit pred #35

merged 10 commits into from
Nov 27, 2024

Conversation

dr-david
Copy link
Collaborator

The get_logit_predictions function was computing $\text{logit}(\text{softmax}(x))$ which is not stable for values of $\text{softmax}(x) \approx 0 \text{ or } \approx 1$

I rewrote it as:

$$ \text{logit}(\text{softmax}(x)_i) = \log(\text{softmax}(x)_i) - \log(1 - \text{softmax}(x)_i) $$

Expanding further:

$$ \text{logit}(\text{softmax}(x)_i) = x_i - \text{logSumExp}(x) - \left(\text{logSumExp}(x_{-i}) - \text{logSumExp}(x)\right) = x_i - \text{logSumExp}(x_{-i}) $$

Copy link
Member

@pawel-czyz pawel-czyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

src/covvfit/_quasimultinomial.py Show resolved Hide resolved
@dr-david dr-david merged commit 1cdfe83 into main Nov 27, 2024
1 check passed
@dr-david dr-david deleted the logit_pred branch November 27, 2024 17:54
dr-david added a commit that referenced this pull request Nov 28, 2024
* evaluation workflow

* fixed data expands

* solve lineage overcounting

* Save current changes for testing older version

* compatibility with refactor #27

* added fitted plots (in progress)

* numerically stable logit predictions

* Generalize the function

---------

Co-authored-by: Paweł Czyż <[email protected]>
@pawel-czyz pawel-czyz restored the logit_pred branch January 16, 2025 13:51
@pawel-czyz pawel-czyz deleted the logit_pred branch January 16, 2025 13:52
pawel-czyz pushed a commit that referenced this pull request Jan 16, 2025
* evaluation workflow

* fixed data expands

* solve lineage overcounting

* Save current changes for testing older version

* compatibility with refactor #27

* added fitted plots (in progress)

* numerically stable logit predictions

* Generalize the function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants