Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0 value handling #345

Merged
merged 23 commits into from
Nov 18, 2024
Merged

0 value handling #345

merged 23 commits into from
Nov 18, 2024

Conversation

JuliaS92
Copy link
Collaborator

Major changes:

  • added raplacement of ==0 values with nan as optional processing step (replacing step in data completeness filter)
  • added dropping of none finite features as optional processing step (replacing dropping of all 0 step)
  • rewrote the ttest to always operate on log transformed data for fold change and p-value calculation (p-values were calculated in untransformed space if not done durign preprocessing)

Tests were updated accordingly, but subset output changed due to the dropping of values. (Before, rows of nans and 0s were retained). Since this step is also used in batch correction the result changed.

TODO: Figure out which undisplayable p-values make it to the volcano plot function, as the axis of the fold change is wider than it needs to be.

@JuliaS92
the t-test now assumes log-normally distributed data and does log-tra…
0bb6202 
…nsformation prior to p-value calculation if it wasn't done before. nas are omitted, p-values are rebustly calculated (pandas can't handle 0)
@JuliaS92
Add 0 replacement as explicit preprocessing step and remove it from d…
8c4839f 
…ata completeness filtering.
@JuliaS92
remove all nan instead of all 0 rows
828a8a5
@JuliaS92
repr of new step
4b60d43
@JuliaS92
Add 0 replacement to wrapper code
481a1df
@JuliaS92
fix data completeness test
2987636
@JuliaS92
Make dropping of unmesured features robust, optional and transparent …
c50ec46 
…in the code
@JuliaS92
Fix subset test to only look at sample dimension
ae84d61
@JuliaS92
Fix batch correction test with major todo, removed normalization step…
9d5503c 
…, as this will change in the future.
@JuliaS92
Add optional step to interface
96fdfc4
mschwoer
mschwoer reviewed Oct 8, 2024
View reviewed changes
alphastats/DataSet.py Outdated Show resolved Hide resolved
alphastats/gui/utils/preprocessing_helper.py Outdated Show resolved Hide resolved
alphastats/plots/VolcanoPlot.py Show resolved Hide resolved
alphastats/plots/VolcanoPlot.py Show resolved Hide resolved
alphastats/plots/VolcanoPlot.py Show resolved Hide resolved
alphastats/statistics/DifferentialExpressionAnalysis.py Outdated Show resolved Hide resolved
alphastats/statistics/DifferentialExpressionAnalysis.py Outdated Show resolved Hide resolved
tests/gui/test_04_preprocessing.py Show resolved Hide resolved
alphastats/DataSet_Preprocess.py Outdated Show resolved Hide resolved
@mschwoer mschwoer merged commit f8f3ce2 into development Nov 18, 2024
5 checks passed
@mschwoer mschwoer deleted the 0-value-handling branch November 18, 2024 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@boopthesnoot boopthesnoot boopthesnoot left review comments

@mschwoer mschwoer mschwoer approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@JuliaS92 @boopthesnoot @mschwoer