Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Second merge of main to spatial-main #246

Merged
merged 47 commits into from
Nov 11, 2024
Merged

Conversation

kaitejohnson
Copy link
Collaborator

@kaitejohnson kaitejohnson commented Nov 5, 2024

This closes #215. An additional PR to merge the latest updates in main (still need to update infection feedback parameters), but has other changes. @cbernalz if this looks good to you then I will go ahead and merge! @arranhamlet Again tagging you for awareness

gvegayon and others added 30 commits August 30, 2024 12:35
* Starting off refactoring (expected to fail) [skip ci]

* Adding new method

* Fixing bug in fit_model (was exploiting scoping)

* Updating docs (fixing S3 methods)

* 49 output class creation (#59)

* add a space

* add first test of first check

* add tests for all of the check/assert functions

* run precommit

* check bug in passing output of checkmate to cliabort

* initial tests of preprocess_ww_data

* add custum utils function for autoescaping brackets to pass to glue

* add a bunch of tests for preprocessing wastewater data

* add one more test of site lab indexing

* fix bugs caught in CI

* fix lab site spacing

* fix spacing in name again

* add test to hospital admissions preprocessing

* add additional test to ensure character to indexing of sites and labs

* remove bug in expected number of unique lab site indices

* add tests to make sure data is daily and test to checkers

* add a bunch of validation checks to the joint datasets and the user specifications

* replace with new way of getting stan data

* fix examples, add test, add warning

* fix examples, add test, add warning

* change from hosp -> count everywhere except stan and  vignette/examples

* add tests for pmfs

* fix bugs in documentation

* add padding value as a function arg

* change pmf size check to a warning not an error

* fix bug

* make initialization function more generic

* update changelog

* modify to test

* fix typo from merge

* fix parsing of cmdstan object

* change parsing of fit obj

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* some tweaks to checkers

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* fix documentation

* fix typo

* fix typo

* change outputs from wwinference() function

* fix typos, add documentation

* fix bug missing stan args

* exclude t columns in data join

* fix vignette bug

* add the ww_output documentation

* document ...

* fix missing comma

* move documentation of params around

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* change syntax and filenames

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/wwinference.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/wwinference.R

Co-authored-by: Dylan H. Morris <[email protected]>

* change naming and internal checking

* change syntax

* move around documentation

* fix check

* fix tests, fix documentation

* rename assert function to specify within a certain frame

* add element to text

* fix bug in function name

* tweak to inference function

* fix two bugs

* adjust tests based on updated get stan data function which breaks up generation of input data

* Update get_stan_data.R example

* update documentation after fixing example

* add example to wwinference wrapper function

* attempt to move around documentation for wwinference methods

* play around with the documentation of the default and the S3 method functions

* export S3 method function

* add back in exporting functions to get input data formatted for stan

* make first argument of function have same name as class object

* fix bug in how max generation time is found

* update vignette to explain wwinference_fit class object vs explicit function calling, add diagnostics and show both ways

* fix naming blocks adding comma when needed

* dont export autoescape brackets function

* fix same bug

* update test and preprocessing to count at LOD values at below LOD

* fix internal call to diagnostic flags function

* Update R/validate.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_preprocess_count_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* implement DMs suggestions

* run pre-commit

* export default functions

* Add test-coverage.yaml from epinowcast

* remove test coverage

* remove example, function not exported

* export default function

* export both diagnostics functions

* add documentation of additional arguments

* Update R/validate.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/validate.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/validate.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_preprocess_count_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_preprocess_ww_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/validate.R

Co-authored-by: Dylan H. Morris <[email protected]>

* manually input some suggestions

* Update tests/testthat/test_checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_checkers.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_preprocess_count_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_preprocess_count_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_preprocess_count_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* add more checknames

* run pre-commit locally

* fix typo

* add some very minimal tests

* fix wwinference function

* fix bug

* fix bug

* Update tests/testthat/test_preprocess_count_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update tests/testthat/test_preprocess_count_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* run pre-commit locally

* fix bugs in tests

* fix error in tests

* move forecast date, calib time, horizon time to args to wrapper function

* fix hosp only example in vignette

* fix error in example

* add dont run to examples

* check -> expect in checkmate, confirm tests pass locally

---------

Co-authored-by: Dylan H. Morris <[email protected]>
Co-authored-by: George G. Vega Yon <[email protected]>

* Making pre-commit happy

* Reworking cross-references and print method

* Removing copy of fit_model

* Fixing function call

* Addressing PR comments

* Forgot to save some changes

* Change output names (#86)

* change names of outputs of wwinference wrapper function

* fix a few other missed replacements

* fix pre-commit

* Fixing R CMD check

* Pre-commit

* Removing diagnostics_summary

---------

Co-authored-by: George G. Vega Yon <[email protected]>

* Update vignettes/wwinference.Rmd

Co-authored-by: Kaitlyn Johnson <[email protected]>

* Update vignettes/wwinference.Rmd

Co-authored-by: Kaitlyn Johnson <[email protected]>

* Adding example of summary and print in the vignette. Addressing some minor comments

* fix test for expected names after changing function args

* set seed in tests

---------

Co-authored-by: Kaitlyn Johnson <[email protected]>
Co-authored-by: Dylan H. Morris <[email protected]>
Co-authored-by: kaitejohnson <[email protected]>
* Starting to use .data and others

* Removing more warnings

* Think almost all issues are now solved

* License warning and passing params as expected

* Removing  prefix

* Fixing note on license and news file

* Using str2lang in spread_draws

* Update R/get_draws_df.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Fixing R CMD check

* fixed intercept in figures

* Update R/generate_simulated_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Apply suggestions from code review by @dylanhmorris

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/get_stan_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* remove call to utils::globalVariables()

* Update R/preprocessing.R

* Update R/generate_simulated_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/preprocessing.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/get_stan_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

---------

Co-authored-by: Dylan H. Morris <[email protected]>
Co-authored-by: Kaitlyn Johnson <[email protected]>
Co-authored-by: kaitejohnson <[email protected]>
* add a space

* update hierarchical estimate of sigma_site

* update prior table

* run pre-commit

* update comment when transforming to site level standard deviations

* add to change log

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* Update model_definition.md

Co-authored-by: Dylan H. Morris <[email protected]>

* Update model_definition.md

Co-authored-by: Dylan H. Morris <[email protected]>

* Update model_definition.md

Co-authored-by: Dylan H. Morris <[email protected]>

* update notation for mode and sd of stdevs

* Update model_definition.md

Co-authored-by: Dylan H. Morris <[email protected]>

* Update model_definition.md

Co-authored-by: Dylan H. Morris <[email protected]>

* Update model_definition.md

Co-authored-by: Dylan H. Morris <[email protected]>

* tweaks to formatting

* Update model_definition.md

Co-authored-by: Dylan H. Morris <[email protected]>

---------

Co-authored-by: Dylan H. Morris <[email protected]>
* fix typo in indicate ww exclusions documentation

* fix typos/language in vignette

* Update R/preprocessing.R

Co-authored-by: Chirag Kumar <[email protected]>

* update docs

---------

Co-authored-by: Chirag Kumar <[email protected]>
* hot fix to readme

* Update NEWS.md

* run pre-commit

* Update NEWS.md (#144)

* run pre-commit locally

* Update NEWS.md

---------

Co-authored-by: George G. Vega Yon <[email protected]>
* Adding new class and method (expected to fail)

* Addressing issues with names (expected to fail)

* Adding the what parameter to the docs

* Addressing final bits. Now need the test

* Adding plot method as a wrapper

* Adding some tests

* Fixing test and setting default y=NULL in plot

* Adding some lines in the vignette to explain the plot method works on wwinference_fit_draws

* Addressing review comments

* Typo in length function

* Reverting R/sysdata.rda and ensuring tests run properly

* Reverting sysdata (again)

* Better print and fixing test

* Fixing tests
* update readme with logo

* swap to svg

* use use package

* adjust size and remove extra text

* try adding new logo

* fix title

* fix title again

* delete old logos
…158)

* update validate to warn if sum(site_pop)>total pop

* modify to center around the reference pop

* temporary change to stan file path for troubleshooting

* model compiles

* reorder pops by size, reindex subpops to sites, add switch for include_ww = 0

* wip rmd

* reindex labsites
 + other changes

* ensure the sum(sites)<total_pop case works

* workaround to handle include_ww = 0 in stan data

* add documentation, fix vignette

* fix preprocessing to order by site pop, add a test for this

* add tests for the hosp only and no aux site cases

* add a test of null data being passed in

* tweaks to print methods and get draws function

* tweak diagnostics, make sure hosp logic works as expected

* switch diagnostics, fix inits, add error message if req ww for hosp only model

* update vignette package data

* update test data

* add log shift from reference pop to central dynamic

* add m prior to params and stan data, fix inits bug

* update test data

* fix preprocessing test to order in terms of site pops

* fix model diagnostics functions

* m should be centered around 1!

* fix inits

* m is log scale, it should be centered around 0

* fix inits

* update test data

* edit subpop definition in model defn

* run pre-commit

* fix arrange

* edit model definition to explain reference subpop

* run pre-commit locally

* fix example

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* Update inst/stan/wwinference.stan

Co-authored-by: Dylan H. Morris <[email protected]>

* add ofsets to intercept and growth rate of unobserved infection process

* update test data running on WSL2

* Change how offsets are handled (#168)

* Update model file to handle offsets slightly differently, clarify parameter name comments

* Fix missing close paren

* Fix variable name

* Fix more variable names

* Remove separate handling of reference pop, fix a few more bugs

* Update docs

* Fix check for warning in get_stan_data test

* Better fix for test_get_stan_data

* Fail more informatively if test_ww_model fails to fit entirely

* Further customize the fitting failure message for informativeness

* Update get stan data with new variable names

* Add new variable names to example_params.toml

* Fix indexing and initialization

* Update test data

* add test of no ww model

* add conditional for inits, add test for no ww

* tweak prreprocessing to handle no wastewater case, add tests for all cases

* update testing data

* Update R/get_stan_data.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/get_draws.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/validate.R

Co-authored-by: Dylan H. Morris <[email protected]>

* Update R/validate.R

Co-authored-by: Dylan H. Morris <[email protected]>

* fix initialization

* update language around the sum(sites)>pop

* run pre-commit locally

* whoops, fix init

* aux site -> aux subpop

* add site_to_subpop map to get_subpop_data function

* create vectors to pass to stan using the subpopulation mappings

* revert to original initialization, use index explicitly in df column name

* remove old comments

* add functions for making spines in wwinference

* move spine functions to get stan data file

* update docs

* fix fxn input

* Fix typo

* refactor handling of sites, subpops, ww data indices interally, commented code, expect to fail

* include lod vals in plots

* fix get stan data to be all based on mappings

* fix tests to take in all inputs to get stan data

* fix lab_site_subpop_spine fxn

* first pass fix postprocessing

* minor tweaks

* update expected column names from get_draws

* update test data

* fix labsite to subpop spine handling, add docs for get ww indices and vals

---------

Co-authored-by: Dylan H. Morris <[email protected]>
Co-authored-by: Dylan H. Morris <[email protected]>
kaitejohnson and others added 17 commits October 2, 2024 14:16
* Adding the developer mode (see if it works)

* Updating action to build twice with caching

* Fixing concurrency and workflow graph

* Fixing action

* Wrong option passed to gh release list

* Adding missing token

* Debugging

* Debugging gh release list

* Debugging gh release list v2

* Trying a different strategu

* Trying a different strategy v2

* Using jq to extract the tag info

* Another try

* Printing releases

* Trying a different strategy

* Was using the wrong pipe

* Properly using the caching

* Switching the version

* Adding person in construction icon

* Adding minor tweaks: auto dev mode and rename cache key

* Adding more links to the site and enforcing buit on new _pkgdown config

* Fixing hashing step

* Was pointing to the wrong yml

* Ensuring hashing and usage of _pkgdown.yml

* Leveraging sparse checkout

* Ensuring where the pkg is thrown

* Correcting sed

* Fixing my bash

* Devel is main and adding toggle button
* Adding validation of records per site

* Updated news

* Addressing co-pilot hallucination

* Explicit call to dplyr::n()

* testing data had multiple site pops per site!

* Better error and adding a test to catch the error.

---------

Co-authored-by: Kaitlyn Johnson <[email protected]>
* Hot fix!

* There was probably a single site!

* Correcting site name

* fix cbind, dont want duplicat column names, use seq_len but differently

* Update NEWS.md

Co-authored-by: Dylan H. Morris <[email protected]>

---------

Co-authored-by: Kaitlyn Johnson <[email protected]>
Co-authored-by: Kaitlyn Johnson <[email protected]>
Co-authored-by: Dylan H. Morris <[email protected]>
* swap order of plotting so calib data shows up

* fix plot function
@kaitejohnson kaitejohnson marked this pull request as ready for review November 6, 2024 18:23
@kaitejohnson kaitejohnson changed the title Merge main to spatial2 Second merge of main to spatial-main Nov 6, 2024
Copy link
Collaborator

@cbernalz cbernalz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! So, this should be the latest changes from main so I can do my results @kaitejohnson?

@kaitejohnson
Copy link
Collaborator Author

kaitejohnson commented Nov 11, 2024

This looks good! So, this should be the latest changes from main so I can do my results @kaitejohnson?

Unfortunately, the changes to the priors on the infection feedback and the RW R(t) step size have not yet been merged into the package defaults for wwinference (see this PR CDCgov/wastewater-informed-covid-forecasting#206 and #236 (comment)). I will make sure this gets done on Tuesday.

@kaitejohnson kaitejohnson merged commit fe94da5 into spatial-main Nov 11, 2024
5 checks passed
@kaitejohnson kaitejohnson deleted the merge-main-to-spatial2 branch November 11, 2024 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants