Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfixes for model2netcdf.ED2() #2985

Merged
merged 66 commits into from
Aug 22, 2022
Merged

Bugfixes for model2netcdf.ED2() #2985

merged 66 commits into from
Aug 22, 2022

Conversation

Aariq
Copy link
Collaborator

@Aariq Aariq commented Jul 25, 2022

Description

Went down a bit of a rabbit hole on this one...

Fixes several bugs related to model2netcdf.ED2(). The end goal is to fix #2974, but ended up having to deal with other issues along the way.

  • Added tests for PEcAn.utils::mstmipvar() where there were none
  • Refactors PEcAn.utils::mstmipvar()
  • Changes default values in mstmipvar() to NULL instead of NA (why this is a good practice: https://stackoverflow.com/a/29620701). This is to fix PEcAn.utils::mstmipvar() condition has length > 1 error #2981
  • Replaces test ED2 output for in tests/testthat/data so the example includes 2 PFTs
  • Adds a pecan_checked.xml for testing the settings argument in test/testthat/data
  • Fixes existing tests (closing ED tests are broken #1329) except for
  • Fixes issue with model2netcdf.ED2() only reading one PFT (model2netcdf.ED2() only converts data for first PFT #2974) (and adds tests)
    • Looks like there was a time when functions looked for a character vector for pfts and the change to using the settings$pfts list was never completed.
  • Set up the pftmapping dataset with usethis::use_data() and documented it. Surprised that it still gives a "global variables" type warning without explicitly assigning it 🤷‍♂️
  • Also ran usethis::use_data_raw() to set up infrastructure to keep pftmapping.csv and convert to package dataset.

Motivation and Context

model2netcdf.ED2() was broken

Review Time Estimate

  • Immediately
  • Within one week
  • When possible

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Unsure if new NULL defaults for mstmipvar args will cause problems anywhere else

Checklist:

  • My change requires a change to the documentation.
  • My name is in the list of CITATION.cff
  • I have updated the CHANGELOG.md.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Should I add a NEWS file to PEcAn.ED2?
Should I increment the version of PEcAn.ED2?

@Aariq Aariq marked this pull request as ready for review July 25, 2022 21:10
@Aariq
Copy link
Collaborator Author

Aariq commented Aug 1, 2022

@dlebauer, I'd like to look at the resulting .nc files with you and figure out if they're correct before merging this PR.

@Aariq Aariq marked this pull request as ready for review August 1, 2022 18:53
@Aariq
Copy link
Collaborator Author

Aariq commented Aug 1, 2022

.nc outputs don't have PFT as a dimension for variables that probably should have that dimension, but I think this is a separate issue for a different PR, so this PR is ready now.

@Aariq Aariq added the Status: Ready Pull request ready for merge, or issue ready to be closed label Aug 2, 2022
@Aariq Aariq requested a review from infotroph August 8, 2022 20:38
Copy link
Member

@dlebauer dlebauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow. Nice work!!! 👏🏻👏🏻👏🏻👏🏻
Thanks for taking the time to understand and improve how these functions work and to re-enable and add tests!

models/ed/R/model2netcdf.ED2.R Outdated Show resolved Hide resolved
@dlebauer
Copy link
Member

dlebauer commented Aug 9, 2022

all test / check errors are 'Error: These files have changed since checkout. Check for out-of-sync autogenerated files, or for untracked files written inside the working tree'

TODO

  • run ./scripts/generate_dependencies.R
  • commit any resulting changes to docker/depends/pecan.depends.R

@Aariq Aariq linked an issue Aug 10, 2022 that may be closed by this pull request
@Aariq Aariq removed the request for review from infotroph August 22, 2022 14:16
@dlebauer dlebauer merged commit 22874d3 into develop Aug 22, 2022
@dlebauer dlebauer deleted the model2netcdf-ed2 branch August 22, 2022 21:37
Copy link
Member

@infotroph infotroph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aariq Sorry again for my long delay in reviewing -- As long as I typed them already, I'm posting the incomplete comments that I started before this was merged but then failed to post in time for them to be useful to you 😬

None of my comments are showstoppers, and in fact as I type this I realize my proposed dir.create additions in the tests aren't strictly needed because the unzip below them does the directory creation for us.

docker/depends/pecan.depends.R Outdated Show resolved Hide resolved
models/ed/DESCRIPTION Show resolved Hide resolved
models/ed/data-raw/pftmapping-csv.R Show resolved Hide resolved
test_that("Met conversion runs without error", {
outdir <- tempfile() #creates a directory
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tempfile gives a name but doesn't actually create a file there!

Suggested change
outdir <- tempfile() #creates a directory
outdir <- tempfile()
dir.create(outdir, showWarnings = FALSE)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like getting files and directories mixed up. Might be:

outfile <- tempfile()
file.create(outfile, showWarnings = FALSE)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the intention here was to create a directory and I was instructed (by someone, forgot who) to use tempfile() to do this. I think a better way is probably to use a sub-directory in tempdir(). Might change it in the next PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the good thing is that you are guaranteed this way to have a unique tempfolder, while if you create it you need to keep track how often this function has been called to make sure the folder you create is unique.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unique to the R session, but doesn't change between calls

Copy link
Member

@infotroph infotroph Aug 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aariq I think I'm the one who said to use tempfile(), which does default to being inside the session temp directory... the gotcha is that what it returns is "here's a path that will be unique if you create a file-or-directory there" rather than a path it has created.

The other, maybe bigger gotcha is that tempdir() and tempfile() are such similarly named functions that do such completely different things. That's one of my top nominees for items that should be in the Inferno but aren't yet.

context("check output from model2netcdf.ED2")
library(stringr)
#set up tempdir
outdir <- tempfile()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
outdir <- tempfile()
outdir <- tempfile()
dir.create(outdir, showWarnings = FALSE)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Ready Pull request ready for merge, or issue ready to be closed
Projects
None yet
4 participants