From aba2f0a8f169d842e50c475b3f4b899b97d5a8a5 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Tue, 4 Oct 2022 20:47:19 -0500 Subject: [PATCH 01/20] more context for choosing packages @skadauke, does this address your comments in https://github.com/OuhscBbmc/REDCapR/issues/417#issuecomment-1264385479 --- vignettes/longitudinal-and-repeating.Rmd | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/vignettes/longitudinal-and-repeating.Rmd b/vignettes/longitudinal-and-repeating.Rmd index aacd9b8a..589252c3 100644 --- a/vignettes/longitudinal-and-repeating.Rmd +++ b/vignettes/longitudinal-and-repeating.Rmd @@ -27,9 +27,13 @@ knitr::opts_chunk$set( Background ================================================================== -If your REDCap project is longitudinal or contains repeating measures, a single call to the API (or a single export through the browser) will return a dataset that is not readily analyzed. Instead, the dataset will resemble Table 5. This isn't because of a software bug, but because you haven't told the software how you would like it structured. There isn't a great way to jam this multidimensional space into a rectangle of points. Our advice for querying REDCap is the same as querying any database system: request datasets that have a natural "grain" and assemble them as to best fit your analyses. +This vignette pertains to reading REDCap records from a project that (a) has longitudinal events or (b) has a repeating measure. The first section conceptually discusses how REDCap stores complex structures. The remaining sections describe how to best retrieve complex structures with the [REDCapTidyieR](https://chop-cgtdataops.github.io/REDCapTidieR) and [REDCapR]((https://OuhscBbmc.github.io/REDCapR) packages. -If your project is not longitudinal and has not repeating measures, you can stop reading and simply call REDCapR's [`redcap_read()`](https://ouhscbbmc.github.io/REDCapR/reference/redcap_read.html). +* If you are new to R or REDCap, consider start with the [Typical REDCap Workflow for a Data Analyst](https://ouhscbbmc.github.io/REDCapR/articles/workflow-read.html) and [Basic REDCapR Operations](https://ouhscbbmc.github.io/REDCapR/articles/BasicREDCapROperations.html) vignettes and then return to this document. +* If you are reading from a *simple* project, just call REDCapR's [`redcap_read()`](https://ouhscbbmc.github.io/REDCapR/reference/redcap_read.html). +* If you want to perform some other operation (such as writing records to REDCap), review the [Reference of REDCapR functions](https://ouhscbbmc.github.io/REDCapR/reference/index.html) to see what is currently available. + +If your REDCap project is longitudinal or contains repeating measures, a single call to the API (or a single export through the browser) will return a dataset that is not readily analyzed. Instead, the dataset will resemble Table 5. This isn't because of a software bug, but because you haven't told the software how you would like the data structured. There isn't a good way to jam this multidimensional space into a rectangle of points. Our advice for querying REDCap is the same as querying any database system: request separate datasets that have a natural "grain" and assemble them as to fit your analyses. Illustration of How Data Points are Structured ================================================================== @@ -115,7 +119,7 @@ When these measurements are added to REDCap's observation table, it resembles Ta As mentioned above, there isn't a universally good way to coerce Tables 1, 3a, and 3b into a single rectangle because the rows represent different things. Or from REDCap's perspective, there's not a good transformation of `redcap_data` (*i.e.*, Table 4) that is appropriate for most statistical programs. -When forced to combine the different entities, the best option is probably Table 5. We call this a "block dataset", borrowing from linear algebra's term [block matrix](https://mathworld.wolfram.com/BlockMatrix.html). You can see the mishmash of tables masquerading as a unified dataset. However it's obvious the rows lack the conceptual coherency of Tables 1, 3a, & 3b. +When forced to combine the different entities, the best option is probably Table 5. We call this a "block dataset", borrowing from linear algebra's [block matrix](https://mathworld.wolfram.com/BlockMatrix.html) term. You can see the mishmash of tables masquerading as a unified dataset. However it's obvious the rows lack the conceptual coherency of Tables 1, 3a, & 3b. **Table 5: mishmashed grain** is how the points are returned from REDCap if you request the data in a single call. @@ -141,7 +145,7 @@ Table 5's primary limitation is that a block dataset is not understood by analys Table 5's secondary limitation is inefficiency. The empty cells aren't computationally free. Every cell must be queried from the database and concatenated in REDCap's web server in order to return Table 5 in the plain-text csv, json, or xml format. In our simple example, more than half of the block dataset's cells are wasted. The emptiness frequently exceeds 90% in real-world REDCap projects (because they tend to have many more variables and repeating instances). The emptiness always exceeds 99.9% in real-world EMRs. -For this reason, REDCap and EMR design their observation table to resemble the computational structure of a [sparse matrix](https://en.wikipedia.org/wiki/Sparse_matrix). The only important difference is that REDCap's unspecified cells are interpreted as null/empty, while a sparse matrix's unspecified cells are interpreted as zero. +For this reason, REDCap and EMR design their observation table to resemble the computational structure of a [sparse matrix](https://en.wikipedia.org/wiki/Sparse_matrix). (The only important difference is that REDCap's unspecified cells are interpreted as null/empty, while a sparse matrix's unspecified cells are interpreted as zero.) > In the case of a sparse matrix, substantial memory requirement reductions can be realized by storing only the non-zero entries. Depending on the number and distribution of the non-zero entries, different data structures can be used and yield huge savings in memory when compared to the basic approach. The trade-off is that accessing the individual elements becomes more complex and additional structures are needed to be able to recover the original matrix unambiguously. @@ -171,20 +175,23 @@ Under the hood, REDCapTidieR calls REDCapR multiple times. Choosing between the Approaches ------------------------------------------------------------------ -We recommend calling [REDCapTidieR](https://chop-cgtdataops.github.io/REDCapTidieR/) in many scenarios, such as: +When retrieving data from REDCap, we recommend calling [REDCapTidieR](https://chop-cgtdataops.github.io/REDCapTidieR/) in many scenarios, such as: -* you are new to managing data with R, or +* you are new to managing or analyzing data with R, or * your analyses will require most of the dataset's rows or columns, or -* you'd benefit from some of the auxiliary information in [REDCapTidieR's supertibble](https://chop-cgtdataops.github.io/REDCapTidieR/articles/REDCapTidieR.html#tidying-redcap-exports), such as instrument's structure. +* you'd benefit from some of the auxiliary information in [REDCapTidieR's supertibble](https://chop-cgtdataops.github.io/REDCapTidieR/articles/REDCapTidieR.html#tidying-redcap-exports), such as the instrument's structure. However we recommend calling [REDCapR](https://ouhscbbmc.github.io/REDCapR/) in other scenarios. It could be worth calling REDCapR multiple times if: -* you are comfortable with managing data with R, or +* you are performing some operation other than retrieving/reading data from REDCap, +* you are comfortable with managing and analyzing data with R, or * your analyses require only a fraction of the data (such as (a) you need only the first event, or (b) the analyses don't involve most of the instruments), or * you want to specify the variables' data types with [`readr::cols()`](https://readr.tidyverse.org/reference/cols.html). If in doubt, start with REDCapTidieR. Escalate to REDCapR if your download time is too long and might be decreased by reducing the information retrieved from the server and transported across the network. +And of course many scenarios are solved best with a combination of both packages, such as (a) [REDCapR](https://ouhscbbmc.github.io/REDCapR) populates the initial demographics in REDCap, (b) research staff enter measures collected from patients over time, (c) [REDCapTidieR](https://chop-cgtdataops.github.io/REDCapTidieR/) retrieves the complete longitudinal dataset, (d) [dplyr](https://dplyr.tidyverse.org/) joins the tibbles, and finally (e) [lme4](https://cran.r-project.org/web/packages/lme4/vignettes/lmer.pdf) tests hypotheses involving [patient trajectories](https://datascienceplus.com/analysing-longitudinal-data-multilevel-growth-models-i/) over time. + Escalating to REDCapR ------------------------------------------------------------------ From 193a743c674b10f56d2d7a254b18c45b9149d8f1 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Tue, 4 Oct 2022 22:47:07 -0500 Subject: [PATCH 02/20] fix problem with output interacting with CLI decorations --- vignettes/BasicREDCapROperations.Rmd | 3 ++- vignettes/TroubleshootingApiCalls.Rmd | 9 +++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/vignettes/BasicREDCapROperations.Rmd b/vignettes/BasicREDCapROperations.Rmd index e2b2d2cb..8cf74b75 100644 --- a/vignettes/BasicREDCapROperations.Rmd +++ b/vignettes/BasicREDCapROperations.Rmd @@ -26,7 +26,8 @@ library(magrittr) requireNamespace("kableExtra") opts_chunk$set( - comment = NA, + collapse = TRUE, + comment = "#>", tidy = FALSE ) diff --git a/vignettes/TroubleshootingApiCalls.Rmd b/vignettes/TroubleshootingApiCalls.Rmd index e11f6492..2c3c94dd 100644 --- a/vignettes/TroubleshootingApiCalls.Rmd +++ b/vignettes/TroubleshootingApiCalls.Rmd @@ -11,6 +11,15 @@ vignette: > # - images/EmailVerified.png --- +```{r} +#| include = FALSE +knitr::opts_chunk$set( + collapse = TRUE, + comment = "#>", + tidy = FALSE +) +``` + There are many links in the pipeline between your institution's [REDCap](https://www.project-redcap.org/) server and the API user. When the end result is unsuccessful, this document should help narrow the location of the possible problem. The first two sections will be relevant to almost any language interacting with the API. The remaining sections are possibly relevant only to your language (*e.g.*, Python, R, PHP, SAS, bash), or your software library such as [redcapAPI](https://github.com/nutterb/redcapAPI) and [REDCapR](https://ouhscbbmc.github.io/REDCapR/) in R, [phpcap](https://github.com/iuredcap/phpcap) in PHP, and [PyCap](https://pycap.readthedocs.io/en/latest/) in Python). Language Agnostic From 970ec0aed2a15d9f2103624f9b399c3c3457fa90 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Wed, 5 Oct 2022 21:03:56 -0500 Subject: [PATCH 03/20] test project for repeating vignette ref #417 --- inst/misc/example.credentials | 1 + inst/test-data/vignette-repeating/data.csv | 13 +++++++++++++ inst/test-data/vignette-repeating/dictionary.csv | 9 +++++++++ 3 files changed, 23 insertions(+) create mode 100644 inst/test-data/vignette-repeating/data.csv create mode 100644 inst/test-data/vignette-repeating/dictionary.csv diff --git a/inst/misc/example.credentials b/inst/misc/example.credentials index 83b6b78c..33274b89 100644 --- a/inst/misc/example.credentials +++ b/inst/misc/example.credentials @@ -34,3 +34,4 @@ redcap_uri,username,project_id,token,comment "https://bbmc.ouhsc.edu/redcap/api/","myusername","2634","8F5313CAA266789F560D79EFCEE2E2F1","Validation Types" "https://bbmc.ouhsc.edu/redcap/api/","myusername","3003","1F2EC7059AC339DFDCD5800225DC7A95","Blank for Gray Status" "https://bbmc.ouhsc.edu/redcap/api/","myusername","3074","5007DC786DBE39CE77ED8DD0C68069A6","Checkboxes 1" +"https://bbmc.ouhsc.edu/redcap/api/","myusername","3181","22C3FF1C8B08899FB6F86D91D874A159","Vignette: Longitudinal & Repeating Measures" diff --git a/inst/test-data/vignette-repeating/data.csv b/inst/test-data/vignette-repeating/data.csv new file mode 100644 index 00000000..8e478eb8 --- /dev/null +++ b/inst/test-data/vignette-repeating/data.csv @@ -0,0 +1,13 @@ +record_id,redcap_repeat_instrument,redcap_repeat_instance,height,weight,bmi,demographics_complete,sbp,dbp,blood_pressure_complete,lab,conc,laboratory_complete +1,,,1.0,11.0,111.0,2,,,,,, +1,blood_pressure,1,,,,,1.1,11.1,2,,, +1,blood_pressure,2,,,,,1.2,11.2,2,,, +1,blood_pressure,3,,,,,1.3,11.3,2,,, +1,laboratory,1,,,,,,,,aa1,"1.1 ppm",2 +1,laboratory,2,,,,,,,,aa2,"11.2 ppm",2 +2,,,2.0,22.0,222.0,2,,,,,, +2,blood_pressure,1,,,,,2.1,22.1,2,,, +2,blood_pressure,2,,,,,2.2,22.2,2,,, +2,blood_pressure,3,,,,,2.3,22.3,2,,, +2,laboratory,1,,,,,,,,bb1,"2.1 ppm",2 +2,laboratory,2,,,,,,,,bb2,"2.2 ppm",2 diff --git a/inst/test-data/vignette-repeating/dictionary.csv b/inst/test-data/vignette-repeating/dictionary.csv new file mode 100644 index 00000000..c19904c6 --- /dev/null +++ b/inst/test-data/vignette-repeating/dictionary.csv @@ -0,0 +1,9 @@ +"Variable / Field Name","Form Name","Section Header","Field Type","Field Label","Choices, Calculations, OR Slider Labels","Field Note","Text Validation Type OR Show Slider Number","Text Validation Min","Text Validation Max",Identifier?,"Branching Logic (Show field only if...)","Required Field?","Custom Alignment","Question Number (surveys only)","Matrix Group Name","Matrix Ranking?","Field Annotation" +record_id,demographics,,text,"Record ID",,,,,,,,,,,,, +height,demographics,,text,"patient height",,,number_1dp,1,100,,,y,,,,, +weight,demographics,,text,"patient weight",,,number_1dp,1,300,,,y,,,,, +bmi,demographics,,text,"patient bmi",,,number_1dp,0,300,,,y,,,,, +sbp,blood_pressure,,text,"systolic blood pressure",,,number,1,300,,,y,,,,, +dbp,blood_pressure,,text,"diastolic blood pressure",,,number,1,300,,,y,,,,, +lab,laboratory,,text,"lab value",,,,,,,,,,,,, +conc,laboratory,,text,concentration,,,,,,,,,,,,, From 7ec035d07bc7538e8e5b695b5a0b7971c8721e65 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Thu, 6 Oct 2022 14:59:10 -0500 Subject: [PATCH 04/20] include new project in the tests ref #417 --- .../project-info-read/all-test-projects.R | 292 ++++++++++-------- 1 file changed, 156 insertions(+), 136 deletions(-) diff --git a/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R b/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R index 788d9ed3..f28c30aa 100644 --- a/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R +++ b/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R @@ -1,7 +1,7 @@ structure(list(project_id = c(153L, 212L, 213L, 268L, 690L, 691L, 753L, 817L, 977L, 998L, 999L, 1396L, 1400L, 1425L, 1490L, 2545L, 2545L, 2593L, 2597L, 2603L, 2626L, 2627L, 2629L, 2630L, 2631L, -2632L, 2634L, 3003L, 3074L), project_title = c("REDCapR Target Simple Static -see https://github.com/OuhscBbmc/REDCapR", +2632L, 2634L, 3003L, 3074L, 3181L), project_title = c("REDCapR Target Simple Static -see https://github.com/OuhscBbmc/REDCapR", "REDCapR Target Longitudinal Arm -see https://github.com/OuhscBbmc/REDCapR", "REDCapR Target Simple -Write Data", "REDCapR Russian -see https://github.com/OuhscBbmc/REDCapR", "REDCapR Target Empty Rows -see https://github.com/OuhscBbmc/REDCapR", @@ -20,142 +20,162 @@ structure(list(project_id = c(153L, 212L, 213L, 268L, 690L, 691L, "REDCapR Longitudinal Single Arm", "REDCapR Decimal Commas and Dots", "REDCapR Decimal Comma", "REDCapR Decimal Dots", "REDCapR Validation Types", "REDCapR Target Simple with Gray Form Status --see https://github.com/OuhscBbmc/REDCapR", -"REDCapR Checkboxes 1"), creation_time = structure(c(1385762300, -1409508475, 1409510046, 1424542707, 1503371677, 1503372285, 1512140287, -1520372784, 1535128221, 1536708074, 1536717011, 1570739027, 1571112561, -1573579416, 1579464868, 1632500924, 1632500924, 1635560455, 1635866052, -1636077368, 1637557792, 1637557976, 1637874534, 1637941852, 1637959298, -1637959358, 1638248591, 1660433618, 1662155863), tzone = "America/Chicago", class = c("POSIXct", +"REDCapR Checkboxes 1", "REDCapR Vignette: Repeating Measures" +), creation_time = structure(c(1385762300, 1409508475, 1409510046, +1424542707, 1503371677, 1503372285, 1512140287, 1520372784, 1535128221, +1536708074, 1536717011, 1570739027, 1571112561, 1573579416, 1579464868, +1632500924, 1632500924, 1635560455, 1635866052, 1636077368, 1637557792, +1637557976, 1637874534, 1637941852, 1637959298, 1637959358, 1638248591, +1660433618, 1662155863, 1665019453), tzone = "America/Chicago", class = c("POSIXct", "POSIXt")), production_time = structure(c(NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, -NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), tzone = "America/Chicago", class = c("POSIXct", -"POSIXt")), in_production = c(FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), project_language = c("English", -"English", "English", "English", "English", "English", "English", -"English", "English", "English", "English", "English", "English", -"English", "English", "English", "English", "English", "English", -"English", "English", "English", "English", "English", "English", -"English", "English", "English", "English"), purpose = c(0L, -0L, 0L, 0L, 0L, 0L, 4L, 4L, 0L, 4L, 0L, 1L, 4L, 4L, 0L, 3L, 3L, -4L, 4L, 4L, 4L, 4L, 4L, 3L, 3L, 3L, 3L, 0L, 4L), purpose_other = c(NA, -NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, "Testing REDCapR API library (in R)", -NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, -NA), project_notes = c(NA, NA, "see https://github.com/OuhscBbmc/REDCapR", -NA, NA, NA, NA, NA, "Generated & contributed by Peter Higgins at University of Michigan School of Medicine", -"Helps test functions when the record name is not numeric", NA, -"Contains values that might create problems for the API recipient", -"Repeating instruments", "Has variables or descriptions that might cause problems with the API", -"see https://github.com/OuhscBbmc/REDCapR", NA, NA, "Another very wide dataset. Supports https://github.com/OuhscBbmc/REDCapR/issues/335", -"REDCapR -- see https://github.com/OuhscBbmc/REDCapR/issues/335", -"Repeating instruments See https://github.com/OuhscBbmc/REDCapR", -"REDCapR Test Suite: delete records from a single-arm project", -"REDCapR Test Suite: delete records from a multiple-arm project", -"Longitudinal project with only one arm", NA, NA, NA, "REDCapR using Validation Types", -"Created for \nhttps://github.com/OuhscBbmc/REDCapR/issues/386 & https://github.com/OuhscBbmc/REDCapR/pull/389", -"Variety of f checkboxes"), custom_record_label = c(NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_), secondary_unique_field = c(NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_), is_longitudinal = c(FALSE, -TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, TRUE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE), has_repeating_instruments_or_events = c(FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE -), surveys_enabled = c(FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE), scheduling_enabled = c(FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE), record_autonumbering_enabled = c(TRUE, FALSE, TRUE, FALSE, -TRUE, TRUE, TRUE, TRUE, TRUE, FALSE, FALSE, TRUE, TRUE, TRUE, -TRUE, FALSE, FALSE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, -TRUE, TRUE, TRUE, TRUE, TRUE), randomization_enabled = c(FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE), ddp_enabled = c(FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, -FALSE, FALSE, FALSE, FALSE, FALSE), project_irb_number = c(NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_), project_grant_number = c(NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_), project_pi_firstname = c(NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_), project_pi_lastname = c(NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_), display_today_now_button = c(TRUE, -TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, -TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, -TRUE, TRUE, TRUE, TRUE, TRUE, TRUE), missing_data_codes = c(NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, -NA_character_, NA_character_, NA_character_), external_modules = c("cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks", -"cross_project_piping,date_validation_action_tags,form_status_tweaks" -), bypass_branching_erase_field_prompt = c("0", "0", "0", "0", -"0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", -"0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0")), row.names = c(NA, --29L), class = c("tbl_df", "tbl", "data.frame")) +NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_ +), tzone = "America/Chicago", class = c("POSIXct", "POSIXt")), + in_production = c(FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), project_language = c("English", + "English", "English", "English", "English", "English", "English", + "English", "English", "English", "English", "English", "English", + "English", "English", "English", "English", "English", "English", + "English", "English", "English", "English", "English", "English", + "English", "English", "English", "English", "English"), purpose = c(0L, + 0L, 0L, 0L, 0L, 0L, 4L, 4L, 0L, 4L, 0L, 1L, 4L, 4L, 0L, 3L, + 3L, 4L, 4L, 4L, 4L, 4L, 4L, 3L, 3L, 3L, 3L, 0L, 4L, 3L), + purpose_other = c(NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, + NA, "Testing REDCapR API library (in R)", NA, NA, NA, NA, + NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA), + project_notes = c(NA, NA, "see https://github.com/OuhscBbmc/REDCapR", + NA, NA, NA, NA, NA, "Generated & contributed by Peter Higgins at University of Michigan School of Medicine", + "Helps test functions when the record name is not numeric", + NA, "Contains values that might create problems for the API recipient", + "Repeating instruments", "Has variables or descriptions that might cause problems with the API", + "see https://github.com/OuhscBbmc/REDCapR", NA, NA, "Another very wide dataset. Supports https://github.com/OuhscBbmc/REDCapR/issues/335", + "REDCapR -- see https://github.com/OuhscBbmc/REDCapR/issues/335", + "Repeating instruments See https://github.com/OuhscBbmc/REDCapR", + "REDCapR Test Suite: delete records from a single-arm project", + "REDCapR Test Suite: delete records from a multiple-arm project", + "Longitudinal project with only one arm", NA, NA, NA, "REDCapR using Validation Types", + "Created for \nhttps://github.com/OuhscBbmc/REDCapR/issues/386 & https://github.com/OuhscBbmc/REDCapR/pull/389", + "Variety of f checkboxes", "https://github.com/OuhscBbmc/REDCapR/blob/main/vignettes/longitudinal-and-repeating.Rmd" + ), custom_record_label = c(NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_ + ), secondary_unique_field = c(NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_ + ), is_longitudinal = c(FALSE, TRUE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, TRUE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), has_repeating_instruments_or_events = c(FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, TRUE), surveys_enabled = c(FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), + scheduling_enabled = c(FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), record_autonumbering_enabled = c(TRUE, + FALSE, TRUE, FALSE, TRUE, TRUE, TRUE, TRUE, TRUE, FALSE, + FALSE, TRUE, TRUE, TRUE, TRUE, FALSE, FALSE, TRUE, TRUE, + TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, + TRUE), randomization_enabled = c(FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), + ddp_enabled = c(FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), project_irb_number = c(NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_), project_grant_number = c(NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_ + ), project_pi_firstname = c(NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_ + ), project_pi_lastname = c(NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_ + ), display_today_now_button = c(TRUE, TRUE, TRUE, TRUE, TRUE, + TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, + TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, + TRUE, TRUE, TRUE, TRUE, TRUE), missing_data_codes = c(NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_, NA_character_, NA_character_, NA_character_, + NA_character_), external_modules = c("cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks", + "cross_project_piping,date_validation_action_tags,form_status_tweaks" + ), bypass_branching_erase_field_prompt = c("0", "0", "0", + "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", + "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0", + "0", "0", "0")), row.names = c(NA, -30L), class = c("tbl_df", +"tbl", "data.frame")) From 2be50c3f40db610df0ccac5dca4af2884674ac53 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Thu, 6 Oct 2022 23:30:57 -0500 Subject: [PATCH 05/20] update spelling list --- inst/WORDLIST | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/inst/WORDLIST b/inst/WORDLIST index 5d80901e..f3385d15 100644 --- a/inst/WORDLIST +++ b/inst/WORDLIST @@ -51,6 +51,7 @@ REDCapCon REDCapR's REDCapTidieR REDCapTidieR's +REDCapTidyieR RODBC RODBCext SSL @@ -83,6 +84,7 @@ dataset's datetime dbp dp +dplyr dropdown eav edu @@ -103,6 +105,8 @@ interpretable io ish json +lme +magrittr manageably mdy mishmashed @@ -136,6 +140,7 @@ tibble tibbles tidyr tidyverse +unsecure upcast uploadable uri From f145373b19024ee330955b705ec2f80bc0a2e140 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Fri, 7 Oct 2022 20:53:35 -0500 Subject: [PATCH 06/20] reflect new labels ref #417 --- .../project-info-read/all-test-projects.R | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R b/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R index f28c30aa..3ca1c999 100644 --- a/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R +++ b/inst/test-data/specific-redcapr/project-info-read/all-test-projects.R @@ -60,26 +60,20 @@ NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_ "Longitudinal project with only one arm", NA, NA, NA, "REDCapR using Validation Types", "Created for \nhttps://github.com/OuhscBbmc/REDCapR/issues/386 & https://github.com/OuhscBbmc/REDCapR/pull/389", "Variety of f checkboxes", "https://github.com/OuhscBbmc/REDCapR/blob/main/vignettes/longitudinal-and-repeating.Rmd" - ), custom_record_label = c(NA_character_, NA_character_, + ), custom_record_label = c(NA, NA, NA, NA, NA, NA, NA, NA, + NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, + NA, NA, NA, NA, NA, NA, "[height]; [weight]; [bmi]"), secondary_unique_field = c(NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, - NA_character_, NA_character_, NA_character_, NA_character_ - ), secondary_unique_field = c(NA_character_, NA_character_, - NA_character_, NA_character_, NA_character_, NA_character_, - NA_character_, NA_character_, NA_character_, NA_character_, - NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, NA_character_, - NA_character_, NA_character_, NA_character_, NA_character_, - NA_character_, NA_character_, NA_character_, NA_character_, - NA_character_, NA_character_, NA_character_, NA_character_ - ), is_longitudinal = c(FALSE, TRUE, FALSE, FALSE, FALSE, + NA_character_), is_longitudinal = c(FALSE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, - FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, TRUE, - FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), has_repeating_instruments_or_events = c(FALSE, + FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, + TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE), has_repeating_instruments_or_events = c(FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, From 4eba8cb4cf8113b1945b8db4aed133903974823b Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Fri, 7 Oct 2022 21:37:01 -0500 Subject: [PATCH 07/20] define & register `knit_print.data.frame()` ref #417 --- vignettes/longitudinal-and-repeating.Rmd | 97 ++++++++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/vignettes/longitudinal-and-repeating.Rmd b/vignettes/longitudinal-and-repeating.Rmd index 589252c3..9f367c7b 100644 --- a/vignettes/longitudinal-and-repeating.Rmd +++ b/vignettes/longitudinal-and-repeating.Rmd @@ -17,11 +17,32 @@ vignette: > ```{r} #| include = FALSE +library(knitr) knitr::opts_chunk$set( collapse = TRUE, comment = "#>", tidy = FALSE ) + +knit_print.data.frame <- function(x, ...) { + # See https://cran.r-project.org/package=knitr/vignettes/knit_print.html +# stop() + x %>% + # rmarkdown::print.paged_df() %>% + kable( + col.names = gsub("_", " ", colnames(.)), + format = "html" + ) %>% + kableExtra::kable_styling( + bootstrap_options = c("striped", "hover", "condensed", "responsive"), + full_width = FALSE + ) %>% + c("", "", .) %>% + paste(collapse = "\n") %>% + asis_output() +} +# register the method +registerS3method("knit_print", "data.frame", knit_print.data.frame) ``` Background @@ -158,6 +179,20 @@ Our advice is to start before Table 5 is assembled --retrieve the information in Two approaches are appropriate for most scenarios: (a) a single call to REDCapTidieR's [`redcap_read_tidy()`](https://chop-cgtdataops.github.io/REDCapTidieR/reference/read_redcap_tidy.html) or (b) multiple calls to REDCapR's [`redcap_read()`](https://ouhscbbmc.github.io/REDCapR/reference/redcap_read.html). +The code in the vignette requires the magrittr package for the `%>%` (alternatively you can use `|>` if you're using R 4.0.2 or later). It also uses the credentials for practice/unsecure dataset. Please see [Part 2 - Retrieve Protected Token](https://ouhscbbmc.github.io/REDCapR/articles/workflow-read.html#part-2---retrieve-protected-token) of the [Typical REDCap Workflow for a Data Analyst](https://ouhscbbmc.github.io/REDCapR/articles/workflow-read.html) vignette for approaches for securing a REDCap token. + +```{r retrieve-credential} +# Support pipes +library(magrittr) + +# Retrieve token +path_credential <- system.file("misc/example.credentials", package = "REDCapR") +credential <- REDCapR::retrieve_credential_local( + path_credential = path_credential, + project_id = 3181 +) +``` + Single REDCapTidieR Calls ------------------------------------------------------------------ @@ -172,6 +207,68 @@ Under the hood, REDCapTidieR calls REDCapR multiple times. {Will Beasley will write this section.} + +To retrieve the credentials for the first project listed above, pass the value of "153" to `project_id`. + +```{r redcapr-demographics} +col_types_demographics <- + readr::cols_only( + record_id = readr::col_integer(), + height = readr::col_double(), + weight = readr::col_double(), + bmi = readr::col_double() + # redcap_repeat_instrument = readr::col_character(), + # redcap_repeat_instance = readr::col_integer(), + # demographics_complete = readr::col_integer() + ) + +ds_demographic <- + REDCapR::redcap_read( + redcap_uri = credential$redcap_uri, + token = credential$token, + forms = "demographics", + col_types = col_types_demographics, + verbose = FALSE, + )$data + +ds_demographic + +# col_types_blood_pressure <- +# readr::cols_only( +# record_id = readr::col_integer(), +# redcap_repeat_instrument = readr::col_character(), +# redcap_repeat_instance = readr::col_integer(), +# sbp = readr::col_double(), +# dbp = readr::col_double(), +# # blood_pressure_complete = readr::col_integer() +# ) +# +# ds_blood_pressure <- +# REDCapR::redcap_read( +# redcap_uri = credential$redcap_uri, +# token = credential$token, +# forms = "blood_pressure", +# # col_types = col_types_blood_pressure, +# verbose = FALSE, +# )$data +# +# ds_blood_pressure + +``` + + +```{r redcapr-block} +ds_block <- + REDCapR::redcap_read( + redcap_uri = credential$redcap_uri, + token = credential$token, + col_types = readr::cols(.default = readr::col_character()), + verbose = FALSE, + )$data + +ds_block +``` + Choosing between the Approaches ------------------------------------------------------------------ From a17fc63c65dc10974b223a7ebcb8cc4e505255b6 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 00:14:57 -0500 Subject: [PATCH 08/20] test that forms work for read & read_oneshot ref #431 --- .../specify-forms-only-1st.R | 12 +++ .../read-oneshot/specify-forms-only-1st.R | 24 ++++++ .../read-oneshot/specify-forms-only-2nd.R | 17 ++++ tests/testthat/test-read-batch-simple.R | 82 +++++++++++++++++++ tests/testthat/test-read-oneshot.R | 50 +++++++++++ 5 files changed, 185 insertions(+) create mode 100644 inst/test-data/specific-redcapr/read-batch-simple/specify-forms-only-1st.R create mode 100644 inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-1st.R create mode 100644 inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-2nd.R diff --git a/inst/test-data/specific-redcapr/read-batch-simple/specify-forms-only-1st.R b/inst/test-data/specific-redcapr/read-batch-simple/specify-forms-only-1st.R new file mode 100644 index 00000000..4853d091 --- /dev/null +++ b/inst/test-data/specific-redcapr/read-batch-simple/specify-forms-only-1st.R @@ -0,0 +1,12 @@ +structure(list(record_id = c(1, 2, 3, 4, 5), name_first = c("Nutmeg", +"Tumtum", "Marcus", "Trudy", "John Lee"), name_last = c("Nutmouse", +"Nutmouse", "Wood", "DAG", "Walker"), address = c("14 Rose Cottage St.\nKenning UK, 323232", +"14 Rose Cottage Blvd.\nKenning UK 34243", "243 Hill St.\nGuthrie OK 73402", +"342 Elm\nDuncanville TX, 75116", "Hotel Suite\nNew Orleans LA, 70115" +), telephone = c("(405) 321-1111", "(405) 321-2222", "(405) 321-3333", +"(405) 321-4444", "(405) 321-5555"), email = c("nutty@mouse.com", +"tummy@mouse.comm", "mw@mwood.net", "peroxide@blonde.com", "left@hippocket.com" +), dob = structure(c(12294, 12121, -13051, -6269, -5375), class = "Date"), + age = c(11, 11, 80, 61, 59), sex = c(0, 1, 1, 0, 1), demographics_complete = c(2, + 2, 2, 2, 2)), row.names = c(NA, -5L), class = c("spec_tbl_df", +"tbl_df", "tbl", "data.frame")) diff --git a/inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-1st.R b/inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-1st.R new file mode 100644 index 00000000..6e88b29c --- /dev/null +++ b/inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-1st.R @@ -0,0 +1,24 @@ +structure(list(record_id = c(1, 2, 3, 4, 5), name_first = c("Nutmeg", +"Tumtum", "Marcus", "Trudy", "John Lee"), name_last = c("Nutmouse", +"Nutmouse", "Wood", "DAG", "Walker"), address = c("14 Rose Cottage St.\nKenning UK, 323232", +"14 Rose Cottage Blvd.\nKenning UK 34243", "243 Hill St.\nGuthrie OK 73402", +"342 Elm\nDuncanville TX, 75116", "Hotel Suite\nNew Orleans LA, 70115" +), telephone = c("(405) 321-1111", "(405) 321-2222", "(405) 321-3333", +"(405) 321-4444", "(405) 321-5555"), email = c("nutty@mouse.com", +"tummy@mouse.comm", "mw@mwood.net", "peroxide@blonde.com", "left@hippocket.com" +), dob = structure(c(12294, 12121, -13051, -6269, -5375), class = "Date"), + age = c(11, 11, 80, 61, 59), sex = c(0, 1, 1, 0, 1), demographics_complete = c(2, + 2, 2, 2, 2)), row.names = c(NA, -5L), spec = structure(list( + cols = list(record_id = structure(list(), class = c("collector_double", + "collector")), name_first = structure(list(), class = c("collector_character", + "collector")), name_last = structure(list(), class = c("collector_character", + "collector")), address = structure(list(), class = c("collector_character", + "collector")), telephone = structure(list(), class = c("collector_character", + "collector")), email = structure(list(), class = c("collector_character", + "collector")), dob = structure(list(format = ""), class = c("collector_date", + "collector")), age = structure(list(), class = c("collector_double", + "collector")), sex = structure(list(), class = c("collector_double", + "collector")), demographics_complete = structure(list(), class = c("collector_double", + "collector"))), default = structure(list(), class = c("collector_guess", + "collector")), delim = ","), class = "col_spec"), class = c("spec_tbl_df", +"tbl_df", "tbl", "data.frame")) diff --git a/inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-2nd.R b/inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-2nd.R new file mode 100644 index 00000000..395727f0 --- /dev/null +++ b/inst/test-data/specific-redcapr/read-oneshot/specify-forms-only-2nd.R @@ -0,0 +1,17 @@ +structure(list(race___1 = c(0, 0, 0, 0, 1), race___2 = c(0, 0, +0, 1, 0), race___3 = c(0, 1, 0, 0, 0), race___4 = c(0, 0, 1, +0, 0), race___5 = c(1, 1, 1, 1, 0), race___6 = c(0, 0, 0, 0, +1), ethnicity = c(1, 1, 0, 1, 2), interpreter_needed = c(0, 0, +1, NA, 0), race_and_ethnicity_complete = c(2, 0, 2, 2, 2)), row.names = c(NA, +-5L), spec = structure(list(cols = list(race___1 = structure(list(), class = c("collector_double", +"collector")), race___2 = structure(list(), class = c("collector_double", +"collector")), race___3 = structure(list(), class = c("collector_double", +"collector")), race___4 = structure(list(), class = c("collector_double", +"collector")), race___5 = structure(list(), class = c("collector_double", +"collector")), race___6 = structure(list(), class = c("collector_double", +"collector")), ethnicity = structure(list(), class = c("collector_double", +"collector")), interpreter_needed = structure(list(), class = c("collector_double", +"collector")), race_and_ethnicity_complete = structure(list(), class = c("collector_double", +"collector"))), default = structure(list(), class = c("collector_guess", +"collector")), delim = ","), class = "col_spec"), class = c("spec_tbl_df", +"tbl_df", "tbl", "data.frame")) diff --git a/tests/testthat/test-read-batch-simple.R b/tests/testthat/test-read-batch-simple.R index 9df6cd80..5160a610 100644 --- a/tests/testthat/test-read-batch-simple.R +++ b/tests/testthat/test-read-batch-simple.R @@ -255,6 +255,88 @@ test_that("specify-forms", { expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object2$data, "tbl") }) +test_that("specify-forms-only-1st", { + testthat::skip_on_cran() + path_expected <- "test-data/specific-redcapr/read-batch-simple/specify-forms-only-1st.R" + desired_forms <- c("demographics") + expected_outcome_message <- "\\d+ records and 10 columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." + + ########################### + ## Default Batch size + expect_message( + regexp = expected_outcome_message, + returned_object1 <- redcap_read(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms) + ) + + if (update_expectation) save_expected(returned_object1$data, path_expected) + expected_data_frame <- retrieve_expected(path_expected) + + expect_equal(returned_object1$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object1$data) + expect_true(returned_object1$success) + expect_match(returned_object1$status_codes, regexp="200", perl=TRUE) + expect_true(returned_object1$records_collapsed=="", "A subset of records was not requested.") + expect_true(returned_object1$fields_collapsed=="", "A subset of fields was not requested.") + expect_true(returned_object1$filter_logic=="", "A filter was not specified.") + expect_match(returned_object1$outcome_messages, regexp=expected_outcome_message, perl=TRUE) + expect_s3_class(returned_object1$data, "tbl") + + ########################### + ## Tiny Batch size + expect_message( + returned_object2 <- redcap_read(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms, batch_size=2), + regexp = expected_outcome_message + ) + + expect_equal(returned_object2$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object2$data) + expect_true(returned_object2$success) + expect_match(returned_object2$status_codes, regexp="200", perl=TRUE) + expect_true(returned_object2$records_collapsed=="", "A subset of records was not requested.") + expect_true(returned_object2$fields_collapsed=="", "A subset of fields was not requested.") + expect_true(returned_object2$filter_logic=="", "A filter was not specified.") + expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) + expect_s3_class(returned_object2$data, "tbl") +}) +# test_that("specify-forms-only-2nd", { +# testthat::skip_on_cran() +# path_expected <- "test-data/specific-redcapr/read-batch-simple/specify-forms-only-2nd.R" +# desired_forms <- c("race_and_ethnicity") +# expected_outcome_message <- "\\d+ records and 19 columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." +# +# ########################### +# ## Default Batch size +# expect_message( +# regexp = expected_outcome_message, +# returned_object1 <- redcap_read(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms) +# ) +# +# if (update_expectation) save_expected(returned_object1$data, path_expected) +# expected_data_frame <- retrieve_expected(path_expected) +# +# expect_equal(returned_object1$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object1$data) +# expect_true(returned_object1$success) +# expect_match(returned_object1$status_codes, regexp="200", perl=TRUE) +# expect_true(returned_object1$records_collapsed=="", "A subset of records was not requested.") +# expect_true(returned_object1$fields_collapsed=="", "A subset of fields was not requested.") +# expect_true(returned_object1$filter_logic=="", "A filter was not specified.") +# expect_match(returned_object1$outcome_messages, regexp=expected_outcome_message, perl=TRUE) +# expect_s3_class(returned_object1$data, "tbl") +# +# ########################### +# ## Tiny Batch size +# expect_message( +# returned_object2 <- redcap_read(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms, batch_size=2), +# regexp = expected_outcome_message +# ) +# +# expect_equal(returned_object2$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object2$data) +# expect_true(returned_object2$success) +# expect_match(returned_object2$status_codes, regexp="200", perl=TRUE) +# expect_true(returned_object2$records_collapsed=="", "A subset of records was not requested.") +# expect_true(returned_object2$fields_collapsed=="", "A subset of fields was not requested.") +# expect_true(returned_object2$filter_logic=="", "A filter was not specified.") +# expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) +# expect_s3_class(returned_object2$data, "tbl") +# }) test_that("raw", { testthat::skip_on_cran() path_expected <- "test-data/specific-redcapr/read-batch-simple/raw.R" diff --git a/tests/testthat/test-read-oneshot.R b/tests/testthat/test-read-oneshot.R index 55930022..91c5903b 100644 --- a/tests/testthat/test-read-oneshot.R +++ b/tests/testthat/test-read-oneshot.R @@ -224,6 +224,56 @@ test_that("specify-forms", { expect_s3_class(returned_object$data, "tbl") }) +test_that("specify-forms-only-1st", { + testthat::skip_on_cran() + path_expected <- "test-data/specific-redcapr/read-oneshot/specify-forms-only-1st.R" + desired_forms <- c("demographics") + expected_outcome_message <- "\\d+ records and \\d+ columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." + + expect_message( + regexp = expected_outcome_message, + returned_object <- redcap_read_oneshot(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms) + ) + + if (update_expectation) save_expected(returned_object$data, path_expected) + expected_data_frame <- retrieve_expected(path_expected) + + expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) + expect_equal(returned_object$status_code, expected=200L) + expect_equal(returned_object$raw_text, expected="", ignore_attr = TRUE) # dput(returned_object$raw_text) + expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") + expect_true(returned_object$fields_collapsed=="", "A subset of fields was not requested.") + expect_true(returned_object$filter_logic=="", "A filter was not specified.") + expect_match(returned_object$outcome_message, regexp=expected_outcome_message, perl=TRUE) + expect_true(returned_object$success) + + expect_s3_class(returned_object$data, "tbl") +}) +test_that("specify-forms-only-2nd", { + testthat::skip_on_cran() + path_expected <- "test-data/specific-redcapr/read-oneshot/specify-forms-only-2nd.R" + desired_forms <- c("race_and_ethnicity") + expected_outcome_message <- "\\d+ records and \\d+ columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." + + expect_message( + regexp = expected_outcome_message, + returned_object <- redcap_read_oneshot(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms) + ) + + if (update_expectation) save_expected(returned_object$data, path_expected) + expected_data_frame <- retrieve_expected(path_expected) + + expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) + expect_equal(returned_object$status_code, expected=200L) + expect_equal(returned_object$raw_text, expected="", ignore_attr = TRUE) # dput(returned_object$raw_text) + expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") + expect_true(returned_object$fields_collapsed=="", "A subset of fields was not requested.") + expect_true(returned_object$filter_logic=="", "A filter was not specified.") + expect_match(returned_object$outcome_message, regexp=expected_outcome_message, perl=TRUE) + expect_true(returned_object$success) + + expect_s3_class(returned_object$data, "tbl") +}) test_that("force-character-type", { testthat::skip_on_cran() path_expected <- "test-data/specific-redcapr/read-oneshot/force-character-type.R" From 8d0a9ca8d9c2652058dca09c0c0b6b181d376786 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 00:18:48 -0500 Subject: [PATCH 09/20] pull out name of record_id in case ...it needs to be forcefully added in the future --- R/redcap-read.R | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/R/redcap-read.R b/R/redcap-read.R index cf771fed..273c3069 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -287,11 +287,15 @@ redcap_read <- function( stop(error_message) } + record_id_name <- metadata$data$field_name[id_position] + # if (!grepl(paste0("\\b", record_id_name, "\\b"), forms_collapsed)) + # forms_collapsed <- paste0(record_id_name, ",", forms_collapsed) + initial_call <- REDCapR::redcap_read_oneshot( redcap_uri = redcap_uri, token = token, records_collapsed = records_collapsed, - fields_collapsed = metadata$data$field_name[id_position], + fields_collapsed = record_id_name, # forms_collapsed = forms_collapsed, events_collapsed = events_collapsed, filter_logic = filter_logic, From ebfec936c8e135c84beab1396bf766e345051d8e Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 12:21:08 -0500 Subject: [PATCH 10/20] add more examples to documentation --- R/redcap-metadata-read.R | 10 ++++++ man/redcap_metadata_read.Rd | 10 ++++++ vignettes/longitudinal-and-repeating.Rmd | 41 ++++++++++++------------ 3 files changed, 41 insertions(+), 20 deletions(-) diff --git a/R/redcap-metadata-read.R b/R/redcap-metadata-read.R index 8347a037..45a7e99f 100644 --- a/R/redcap-metadata-read.R +++ b/R/redcap-metadata-read.R @@ -64,8 +64,18 @@ #' @examples #' \dontrun{ #' uri <- "https://bbmc.ouhsc.edu/redcap/api/" +#' +#' # A simple project (pid 153) #' token <- "9A81268476645C4E5F03428B8AC3AA7B" #' REDCapR::redcap_metadata_read(redcap_uri=uri, token=token) +#' +#' # A longitudinal project (pid 212) +#' token <- "0434F0E9CF53ED0587847AB6E51DE762" +#' REDCapR::redcap_metadata_read(redcap_uri=uri, token=token) +#' +#' # A repeating measures (pid 3181) +#' token <- "22C3FF1C8B08899FB6F86D91D874A159" +#' REDCapR::redcap_metadata_read(redcap_uri=uri, token=token) #' } #' @importFrom magrittr %>% diff --git a/man/redcap_metadata_read.Rd b/man/redcap_metadata_read.Rd index f097bcf5..9b2350c2 100644 --- a/man/redcap_metadata_read.Rd +++ b/man/redcap_metadata_read.Rd @@ -78,8 +78,18 @@ one field in the project's dataset. \examples{ \dontrun{ uri <- "https://bbmc.ouhsc.edu/redcap/api/" + +# A simple project (pid 153) token <- "9A81268476645C4E5F03428B8AC3AA7B" REDCapR::redcap_metadata_read(redcap_uri=uri, token=token) + +# A longitudinal project (pid 212) +token <- "0434F0E9CF53ED0587847AB6E51DE762" +REDCapR::redcap_metadata_read(redcap_uri=uri, token=token) + +# A repeating measures (pid 3181) +token <- "22C3FF1C8B08899FB6F86D91D874A159" +REDCapR::redcap_metadata_read(redcap_uri=uri, token=token) } } \references{ diff --git a/vignettes/longitudinal-and-repeating.Rmd b/vignettes/longitudinal-and-repeating.Rmd index 9f367c7b..9aa0cd95 100644 --- a/vignettes/longitudinal-and-repeating.Rmd +++ b/vignettes/longitudinal-and-repeating.Rmd @@ -233,29 +233,30 @@ ds_demographic <- ds_demographic -# col_types_blood_pressure <- -# readr::cols_only( -# record_id = readr::col_integer(), -# redcap_repeat_instrument = readr::col_character(), -# redcap_repeat_instance = readr::col_integer(), -# sbp = readr::col_double(), -# dbp = readr::col_double(), -# # blood_pressure_complete = readr::col_integer() -# ) -# -# ds_blood_pressure <- -# REDCapR::redcap_read( -# redcap_uri = credential$redcap_uri, -# token = credential$token, -# forms = "blood_pressure", -# # col_types = col_types_blood_pressure, -# verbose = FALSE, -# )$data -# -# ds_blood_pressure +col_types_blood_pressure <- + readr::cols( + record_id = readr::col_integer(), + redcap_repeat_instrument = readr::col_character(), + redcap_repeat_instance = readr::col_integer(), + sbp = readr::col_double(), + dbp = readr::col_double(), + # blood_pressure_complete = readr::col_integer() + ) + +ds_blood_pressure <- + REDCapR::redcap_read_oneshot( + redcap_uri = credential$redcap_uri, + token = credential$token, + fields = "redcord_id", + forms = "blood_pressure", + # col_types = col_types_blood_pressure, + verbose = FALSE, + )$data +ds_blood_pressure ``` +If for some reason you need the block dataset through the API, one call will retrieve it. ```{r redcapr-block} ds_block <- From 68737cb3a4113dc02adfc71e8c7ad721c1b96502 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 12:41:27 -0500 Subject: [PATCH 11/20] add boolean `plumbing` variable to internal metadata fx ref #439 --- R/redcap-metadata-coltypes.R | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/R/redcap-metadata-coltypes.R b/R/redcap-metadata-coltypes.R index c2277951..aaf142c6 100644 --- a/R/redcap-metadata-coltypes.R +++ b/R/redcap-metadata-coltypes.R @@ -106,7 +106,7 @@ #' the project manager selected the "integer" validation option, all those #' "abcd" values remain untouched. #' -#' This is one reason `redcap_metadata_coltypes` prints it suggestions to the console. +#' This is one reason `redcap_metadata_coltypes()` prints it suggestions to the console. #' It allows the developer to adjust the specifications to match the values #' returned by the API. The the "abcd" scenario, consider (a) changing the type #' from `col_integer` to `col_character`, (b) excluding the trash values, @@ -344,8 +344,14 @@ redcap_metadata_internal <- function( dplyr::ungroup() %>% dplyr::arrange(.data$form_order, .data$field_order_within_form) %>% dplyr::select(-.data$form_order, -.data$field_order_within_form) %>% - tibble::add_row(d_again, .after = 1) + tibble::add_row(d_again, .after = 1) %>% + dplyr::mutate( + plumbing = (.data$field_name %in% c("redcap_event_name", "redcap_repeat_instrument", "redcap_repeat_instance")) + ) + + d_meta$plumbing[1] <- TRUE # The first field should always be the "record" identifier. + # The types of variables that are in metadata, but NOT variables: # setdiff(d_meta$field_name_base, d_var$original_field_name) # [1] "signature" "file_upload" "descriptive" @@ -451,7 +457,8 @@ redcap_metadata_internal <- function( # .data$padding1, # .data$padding2, .data$aligned, - .data$field_name_base + .data$field_name_base, + .data$plumbing ) decimal_period_any <- any(d_meta$vt %in% c("number", "number_1dp", "number_2dp", "number_3dp", "number_4dp" )) @@ -476,3 +483,14 @@ redcap_metadata_internal <- function( repeating = d_proj$has_repeating_instruments_or_events[1] ) } + +# uri <- "https://bbmc.ouhsc.edu/redcap/api/" +# +# A simple project (pid 153) +# REDCapR:::redcap_metadata_internal(uri, "9A81268476645C4E5F03428B8AC3AA7B")$d_variable +# +# A longitudinal project (pid 212) +# REDCapR:::redcap_metadata_internal(uri, "0434F0E9CF53ED0587847AB6E51DE762")$d_variable +# +# # A repeating measures (pid 3181) +# REDCapR:::redcap_metadata_internal(uri, "22C3FF1C8B08899FB6F86D91D874A159")$d_variable From 1df198ae3b280b9b2ae5f0a9b899fb1e8f34b40d Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 14:54:41 -0500 Subject: [PATCH 12/20] temporarily removing *_collapsed paramters ref #440 --- NAMESPACE | 1 - NEWS.md | 17 +++++ R/redcap-event-instruments.R | 2 +- R/redcap-metadata-read.R | 20 +++--- R/redcap-project.R | 8 --- R/redcap-read-eav-oneshot.R | 44 ++++++------ R/redcap-read-oneshot-eav.R | 50 +++++++------- R/redcap-read-oneshot.R | 44 ++++++------ R/redcap-read.R | 68 +++++++++---------- R/utilities.R | 30 ++++---- R/validate.R | 22 +++--- man/collapse_vector.Rd | 10 +-- man/redcap_metadata_coltypes.Rd | 2 +- man/redcap_metadata_read.Rd | 8 --- man/redcap_project.Rd | 4 -- man/redcap_read.Rd | 16 ----- man/redcap_read_eav_oneshot.Rd | 16 ----- man/redcap_read_oneshot.Rd | 16 ----- man/redcap_read_oneshot_eav.Rd | 20 +----- man/validate.Rd | 7 -- .../testthat/test-utilities-collapse_vector.R | 18 ++--- tests/testthat/test-validate.R | 16 ++--- vignettes/BasicREDCapROperations.Rmd | 32 ++------- vignettes/longitudinal-and-repeating.Rmd | 22 +++--- 24 files changed, 189 insertions(+), 304 deletions(-) diff --git a/NAMESPACE b/NAMESPACE index 1ba68b45..a492d864 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -41,7 +41,6 @@ export(retrieve_credential_local) export(retrieve_credential_mssql) export(sanitize_token) export(validate_field_names) -export(validate_field_names_collapsed) export(validate_for_write) export(validate_no_logical) importFrom(magrittr,"%>%") diff --git a/NEWS.md b/NEWS.md index 6222ec9f..b8e8c7aa 100644 --- a/NEWS.md +++ b/NEWS.md @@ -21,6 +21,23 @@ These features are not yet on CRAN. Install with `remotes::install_github("Ouhs For more information, read the short chapter in [*R for Data Science*](https://r4ds.had.co.nz/tibbles.html). +* The `*_collapsed` parameters are deprecated. When your want to limit on records/fields/forms/events, pass the vector of characters, not the scalar character separated by commas (which I think everyone does already). In other words use `c("demographics", "blood_pressure")` instead of `"demographics,blood_pressure"`. + + Here are the relationships between the four pairs of variables: + + ```r + records_collapsed <- collapse_vector(records , records_collapsed) + fields_collapsed <- collapse_vector(fields , fields_collapsed) + forms_collapsed <- collapse_vector(forms , forms_collapsed) + events_collapsed <- collapse_vector(events , events_collapsed) + ``` + + If someone is using the *_collapsed parameter, they can programmatically convert it to a vector like: + + ```r + field_names <- trimws(unlist(strsplit(field_names_collapsed, ","))) + ``` + ### New Features * New `redcap_metadata_coltypes()` function. Inspects the fields types and validation text of each field to generate a suggested `readr::col_types` object that reflects the project's current data dictionary. The object then can be passed to the `col_types` parameter of `redcap_read()` or `redcap_read_oneshot()`. (Suggested and discussed with @pbchase, @nutterb, @skadauke, & others, #405 & #294) diff --git a/R/redcap-event-instruments.R b/R/redcap-event-instruments.R index f0c9a211..77cbe767 100644 --- a/R/redcap-event-instruments.R +++ b/R/redcap-event-instruments.R @@ -83,7 +83,7 @@ redcap_event_instruments <- function( token = token, content = "formEventMapping", format = "csv", - "arms[0]" = collapse_vector(arms, NULL) + "arms[0]" = collapse_vector(arms) ) col_types <- readr::cols( diff --git a/R/redcap-metadata-read.R b/R/redcap-metadata-read.R index 45a7e99f..79185084 100644 --- a/R/redcap-metadata-read.R +++ b/R/redcap-metadata-read.R @@ -15,12 +15,12 @@ #' project. Required. #' @param forms An array, where each element corresponds to the REDCap form #' of the desired fields. Optional. -#' @param forms_collapsed A single string, where the desired forms are -#' separated by commas. Optional. +# @param forms_collapsed A single string, where the desired forms are +# separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -#' @param fields_collapsed A single string, where the desired field names are -#' separated by commas. Optional. +# @param fields_collapsed A single string, where the desired field names are +# separated by commas. Optional. #' @param verbose A boolean value indicating if `message`s should be printed #' to the R console during the operation. The verbose output might contain #' sensitive information (*e.g.* PHI), so turn this off if the output might @@ -84,9 +84,9 @@ redcap_metadata_read <- function( redcap_uri, token, forms = NULL, - forms_collapsed = "", + # forms_collapsed = "", fields = NULL, - fields_collapsed = "", + # fields_collapsed = "", verbose = TRUE, config_options = NULL, handle_httr = NULL @@ -98,12 +98,12 @@ redcap_metadata_read <- function( validate_field_names(fields, stop_on_error = TRUE) token <- sanitize_token(token) - fields_collapsed <- collapse_vector(fields , fields_collapsed) - forms_collapsed <- collapse_vector(forms , forms_collapsed) + fields_collapsed <- collapse_vector(fields) + forms_collapsed <- collapse_vector(forms) verbose <- verbose_prepare(verbose) - if (1L <= nchar(fields_collapsed)) - validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) + # if (1L <= nchar(fields_collapsed)) + # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) post_body <- list( token = token, diff --git a/R/redcap-project.R b/R/redcap-project.R index 40fa8562..d00db925 100644 --- a/R/redcap-project.R +++ b/R/redcap-project.R @@ -59,13 +59,9 @@ redcap_project <- setRefClass( batch_size = 100L, interbatch_delay = 0, records = NULL, - records_collapsed = "", fields = NULL, - fields_collapsed = "", forms = NULL, - forms_collapsed = "", events = NULL, - events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, @@ -89,13 +85,9 @@ redcap_project <- setRefClass( redcap_uri = redcap_uri, token = token, records = records, - records_collapsed = records_collapsed, fields = fields, - fields_collapsed = fields_collapsed, forms = forms, - forms_collapsed = forms_collapsed, events = events, - events_collapsed = events_collapsed, raw_or_label = raw_or_label, raw_or_label_headers = raw_or_label_headers, export_checkbox_label = export_checkbox_label, diff --git a/R/redcap-read-eav-oneshot.R b/R/redcap-read-eav-oneshot.R index 313a81f2..32c033d5 100644 --- a/R/redcap-read-eav-oneshot.R +++ b/R/redcap-read-eav-oneshot.R @@ -14,20 +14,20 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -#' @param records_collapsed A single string, where the desired ID values -#' are separated by commas. Optional. +# @param records_collapsed A single string, where the desired ID values +# are separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -#' @param fields_collapsed A single string, where the desired field names are -#' separated by commas. Optional. +# @param fields_collapsed A single string, where the desired field names are +# separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' form. Optional. -#' @param forms_collapsed A single string, where the desired form names are -#' separated by commas. Optional. +# @param forms_collapsed A single string, where the desired form names are +# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -#' @param events_collapsed A single string, where the desired event names are -#' separated by commas. Optional. +# @param events_collapsed A single string, where the desired event names are +# separated by commas. Optional. # @param raw_or_label A string (either `'raw'` or `'label'`) that specifies # whether to export the raw coded values or the labels for the options of # multiple choice fields. Default is `'raw'`. @@ -183,13 +183,13 @@ redcap_read_eav_oneshot <- function( redcap_uri, token, records = NULL, - records_collapsed = "", + # records_collapsed = "", fields = NULL, - fields_collapsed = "", + # fields_collapsed = "", forms = NULL, - forms_collapsed = "", + # forms_collapsed = "", events = NULL, - events_collapsed = "", + # events_collapsed = "", # raw_or_label = "raw", # raw_or_label_headers = "raw", # export_checkbox_label = FALSE, @@ -214,13 +214,13 @@ redcap_read_eav_oneshot <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic(records , any.missing=TRUE , min.len=0) - checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) # checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) # checkmate::assert_subset( raw_or_label , c("raw", "label")) # checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -246,17 +246,17 @@ redcap_read_eav_oneshot <- function( validate_field_names(fields, stop_on_error = TRUE) token <- sanitize_token(token) - records_collapsed <- collapse_vector(records , records_collapsed) - fields_collapsed <- collapse_vector(fields , fields_collapsed) - forms_collapsed <- collapse_vector(forms , forms_collapsed) - events_collapsed <- collapse_vector(events , events_collapsed) + records_collapsed <- collapse_vector(records) + fields_collapsed <- collapse_vector(fields) + forms_collapsed <- collapse_vector(forms) + events_collapsed <- collapse_vector(events) filter_logic <- filter_logic_prepare(filter_logic) datetime_range_begin<- dplyr::coalesce(strftime(datetime_range_begin, "%Y-%m-%d %H:%M:%S"), "") datetime_range_end <- dplyr::coalesce(strftime(datetime_range_end , "%Y-%m-%d %H:%M:%S"), "") verbose <- verbose_prepare(verbose) - if (1L <= nchar(fields_collapsed) ) - validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) + # if (1L <= nchar(fields_collapsed) ) + # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) post_body <- list( token = token, diff --git a/R/redcap-read-oneshot-eav.R b/R/redcap-read-oneshot-eav.R index 0b2c3a9a..ffaaaaec 100644 --- a/R/redcap-read-oneshot-eav.R +++ b/R/redcap-read-oneshot-eav.R @@ -14,20 +14,20 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -#' @param records_collapsed A single string, where the desired ID values -#' are separated by commas. Optional. +# @param records_collapsed A single string, where the desired ID values +# are separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired #' project field. Optional. -#' @param fields_collapsed A single string, where the desired field names -#' are separated by commas. Optional. +# @param fields_collapsed A single string, where the desired field names +# are separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' field. Optional. -#' @param forms_collapsed A single string, where the desired form names are -#' separated by commas. Optional. +# @param forms_collapsed A single string, where the desired form names are +# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -#' @param events_collapsed A single string, where the desired event names are -#' separated by commas. Optional. +# @param events_collapsed A single string, where the desired event names are +# separated by commas. Optional. #' @param raw_or_label A string (either `'raw'` or `'label'` that specifies #' whether to export the raw coded values or the labels for the options of #' multiple choice fields. Default is `'raw'`. @@ -131,7 +131,7 @@ #' #' # Return only records with IDs of 1 and 3 #' desired_records_v1 <- c(1, 3) -#' ds_some_rows_v1 <- REDCapR::redcap_read_oneshot_eav( +#' ds_some_rows_v1 <- REDCapR:::redcap_read_oneshot_eav( #' redcap_uri = uri, #' token = token, #' records = desired_records_v1 @@ -139,7 +139,7 @@ #' #' # Return only the fields record_id, name_first, and age #' desired_fields_v1 <- c("record_id", "name_first", "age") -#' ds_some_fields_v1 <- redcap_read_oneshot_eav( +#' ds_some_fields_v1 <- REDCapR:::redcap_read_oneshot_eav( #' redcap_uri = uri, #' token = token, #' fields = desired_fields_v1 @@ -154,13 +154,13 @@ redcap_read_oneshot_eav <- function( redcap_uri, token, records = NULL, - records_collapsed = "", + # records_collapsed = "", fields = NULL, - fields_collapsed = "", + # fields_collapsed = "", forms = NULL, - forms_collapsed = "", + # forms_collapsed = "", events = NULL, - events_collapsed = "", + # events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", # placeholder: exportCheckboxLabel @@ -184,13 +184,13 @@ redcap_read_oneshot_eav <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic(records , any.missing=TRUE , min.len=0) - checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) + # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE ) - checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) + # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE ) - checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) + # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE ) - checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) checkmate::assert_subset( raw_or_label , c("raw", "label")) checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -214,18 +214,18 @@ redcap_read_oneshot_eav <- function( validate_field_names(fields, stop_on_error = TRUE) token <- sanitize_token(token) - records_collapsed <- collapse_vector(records , records_collapsed) - fields_collapsed <- collapse_vector(fields , fields_collapsed) - forms_collapsed <- collapse_vector(forms , forms_collapsed) - events_collapsed <- collapse_vector(events , events_collapsed) + records_collapsed <- collapse_vector(records) + fields_collapsed <- collapse_vector(fields) + forms_collapsed <- collapse_vector(forms) + events_collapsed <- collapse_vector(events) export_data_access_groups <- ifelse(export_data_access_groups, "true", "false") filter_logic <- filter_logic_prepare(filter_logic) datetime_range_begin <- dplyr::coalesce(strftime(datetime_range_begin, "%Y-%m-%d %H:%M:%S"), "") datetime_range_end <- dplyr::coalesce(strftime(datetime_range_end , "%Y-%m-%d %H:%M:%S"), "") verbose <- verbose_prepare(verbose) - if (1L <= nchar(fields_collapsed) ) - validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) + # if (1L <= nchar(fields_collapsed) ) + # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) post_body <- list( token = token, @@ -268,7 +268,7 @@ redcap_read_oneshot_eav <- function( REDCapR::redcap_metadata_read( redcap_uri, token, - forms_collapsed = forms_collapsed, + forms = forms, handle_httr = handle_httr )$data diff --git a/R/redcap-read-oneshot.R b/R/redcap-read-oneshot.R index 91ecda8b..084ec4f4 100644 --- a/R/redcap-read-oneshot.R +++ b/R/redcap-read-oneshot.R @@ -13,20 +13,20 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -#' @param records_collapsed A single string, where the desired ID values -#' are separated by commas. Optional. +# @param records_collapsed A single string, where the desired ID values +# are separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -#' @param fields_collapsed A single string, where the desired field names are -#' separated by commas. Optional. +# @param fields_collapsed A single string, where the desired field names are +# separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' form. Optional. -#' @param forms_collapsed A single string, where the desired form names are -#' separated by commas. Optional. +# @param forms_collapsed A single string, where the desired form names are +# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -#' @param events_collapsed A single string, where the desired event names are -#' separated by commas. Optional. +# @param events_collapsed A single string, where the desired event names are +# separated by commas. Optional. #' @param raw_or_label A string (either `'raw'` or `'label'`) that specifies #' whether to export the raw coded values or the labels for the options of #' multiple choice fields. Default is `'raw'`. @@ -179,13 +179,13 @@ redcap_read_oneshot <- function( redcap_uri, token, records = NULL, - records_collapsed = "", + # records_collapsed = "", fields = NULL, - fields_collapsed = "", + # fields_collapsed = "", forms = NULL, - forms_collapsed = "", + # forms_collapsed = "", events = NULL, - events_collapsed = "", + # events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, @@ -210,13 +210,13 @@ redcap_read_oneshot <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic(records , any.missing=TRUE , min.len=0) - checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) checkmate::assert_subset( raw_or_label , c("raw", "label")) checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -242,17 +242,17 @@ redcap_read_oneshot <- function( validate_field_names(fields, stop_on_error = TRUE) token <- sanitize_token(token) - records_collapsed <- collapse_vector(records , records_collapsed) - fields_collapsed <- collapse_vector(fields , fields_collapsed) - forms_collapsed <- collapse_vector(forms , forms_collapsed) - events_collapsed <- collapse_vector(events , events_collapsed) + records_collapsed <- collapse_vector(records) + fields_collapsed <- collapse_vector(fields) + forms_collapsed <- collapse_vector(forms) + events_collapsed <- collapse_vector(events) filter_logic <- filter_logic_prepare(filter_logic) datetime_range_begin<- dplyr::coalesce(strftime(datetime_range_begin, "%Y-%m-%d %H:%M:%S"), "") datetime_range_end <- dplyr::coalesce(strftime(datetime_range_end , "%Y-%m-%d %H:%M:%S"), "") verbose <- verbose_prepare(verbose) - if (1L <= nchar(fields_collapsed) ) - validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) + # if (1L <= nchar(fields_collapsed) ) + # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) post_body <- list( token = token, diff --git a/R/redcap-read.R b/R/redcap-read.R index 273c3069..2050f8a5 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -26,20 +26,20 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -#' @param records_collapsed A single string, where the desired ID values are -#' separated by commas. Optional. +# @param records_collapsed A single string, where the desired ID values are +# separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -#' @param fields_collapsed A single string, where the desired field names are -#' separated by commas. Optional. +# @param fields_collapsed A single string, where the desired field names are +# separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' form. Optional. -#' @param forms_collapsed A single string, where the desired form names are -#' separated by commas. Optional. +# @param forms_collapsed A single string, where the desired form names are +# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -#' @param events_collapsed A single string, where the desired event names are -#' separated by commas. Optional. +# @param events_collapsed A single string, where the desired event names are +# separated by commas. Optional. #' @param raw_or_label A string (either `'raw'` or `'label'` that specifies #' whether to export the raw coded values or the labels for the options of #' multiple choice fields. Default is `'raw'`. @@ -198,10 +198,10 @@ redcap_read <- function( continue_on_error = FALSE, redcap_uri, token, - records = NULL, records_collapsed = "", - fields = NULL, fields_collapsed = "", - forms = NULL, forms_collapsed = "", - events = NULL, events_collapsed = "", + records = NULL, # records_collapsed = "", + fields = NULL, # fields_collapsed = "", + forms = NULL, # forms_collapsed = "", + events = NULL, # events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, @@ -227,13 +227,13 @@ redcap_read <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic( records , any.missing=TRUE, min.len=0) - checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) + # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) checkmate::assert_subset( raw_or_label , c("raw", "label")) checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -259,15 +259,15 @@ redcap_read <- function( validate_field_names(fields, stop_on_error = TRUE) token <- sanitize_token(token) - records_collapsed <- collapse_vector(records , records_collapsed) - fields_collapsed <- collapse_vector(fields , fields_collapsed) - forms_collapsed <- collapse_vector(forms , forms_collapsed) - events_collapsed <- collapse_vector(events , events_collapsed) + # records_collapsed <- collapse_vector(records , records_collapsed) + # fields_collapsed <- collapse_vector(fields , fields_collapsed) + # forms_collapsed <- collapse_vector(forms , forms_collapsed) + # events_collapsed <- collapse_vector(events , events_collapsed) filter_logic <- filter_logic_prepare(filter_logic) verbose <- verbose_prepare(verbose) - if (1L <= nchar(fields_collapsed)) - validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) + # if (1L <= nchar(fields_collapsed)) + # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) start_time <- Sys.time() @@ -294,10 +294,10 @@ redcap_read <- function( initial_call <- REDCapR::redcap_read_oneshot( redcap_uri = redcap_uri, token = token, - records_collapsed = records_collapsed, - fields_collapsed = record_id_name, - # forms_collapsed = forms_collapsed, - events_collapsed = events_collapsed, + records = records, + fields = record_id_name, + # forms = forms, + events = events, filter_logic = filter_logic, datetime_range_begin = datetime_range_begin, datetime_range_end = datetime_range_end, @@ -362,9 +362,9 @@ redcap_read <- function( redcap_uri = redcap_uri, token = token, records = selected_ids, - fields_collapsed = fields_collapsed, - events_collapsed = events_collapsed, - forms_collapsed = forms_collapsed, + fields = fields, + events = events, + forms = forms, raw_or_label = raw_or_label, raw_or_label_headers = raw_or_label_headers, export_checkbox_label = export_checkbox_label, @@ -412,7 +412,7 @@ redcap_read <- function( lst_batch[[i]] <- read_result$data success_combined <- success_combined & read_result$success - rm(read_result) # Admittedly overkill defensiveness. + # rm(read_result) # Admittedly overkill defensiveness. } ds_stacked <- dplyr::bind_rows(lst_batch) @@ -461,10 +461,10 @@ redcap_read <- function( status_codes = status_code_combined, outcome_messages = outcome_message_combined, # data_types = data_types, - records_collapsed = records_collapsed, - fields_collapsed = fields_collapsed, - forms_collapsed = forms_collapsed, - events_collapsed = events_collapsed, + records_collapsed = collapse_vector(records), + fields_collapsed = read_result$fields_collapsed, # From the last call + forms_collapsed = read_result$forms_collapsed, # From the last call + events_collapsed = read_result$events_collapsed, # From the last call filter_logic = filter_logic, datetime_range_begin= datetime_range_begin, datetime_range_end = datetime_range_end, diff --git a/R/utilities.R b/R/utilities.R index 893afa87..05f4cb9b 100644 --- a/R/utilities.R +++ b/R/utilities.R @@ -80,8 +80,8 @@ replace_nas_with_explicit <- function( #' (presumably right before the return value is passed to the API) #' #' @param elements An array of values. Can be `NULL`. Required. -#' @param collapsed A single character element, where the values are separated -#' by commas. Can be `NULL`. Required. +# @param collapsed A single character element, where the values are separated +# by commas. Can be `NULL`. Required. #' #' @return A single character element, where the values are separated by #' commas. Can be blank. (*i.e.*, `""`). @@ -91,25 +91,19 @@ replace_nas_with_explicit <- function( #' #' @examples #' library(REDCapR) # Load the package into the current R session. -#' REDCapR:::collapse_vector(elements = NULL , collapsed = NULL) -#' REDCapR:::collapse_vector(elements = letters, collapsed = NULL) -#' REDCapR:::collapse_vector(elements = NULL , collapsed = "4,5,6") +#' REDCapR:::collapse_vector(elements = NULL ) +#' REDCapR:::collapse_vector(elements = letters) ## We're intentionally not exporting this function. -collapse_vector <- function(elements, collapsed) { - checkmate::assert_character(collapsed, len=1, any.missing=TRUE, null.ok=TRUE) +collapse_vector <- function(elements) { + checkmate::assert_vector(elements, any.missing=FALSE, null.ok=TRUE) - if ((is.null(collapsed) || length(collapsed) == 0L) || all(nchar(collapsed) == 0L)) { - - # This is an empty string if `elements` (eg, fields`) is NULL. - collapsed <- dplyr::if_else( - is.null(elements), - "", - paste0(elements, collapse = ",") - ) - } - - collapsed + # This is an empty string if `elements` (eg, fields`) is NULL. + dplyr::if_else( + is.null(elements), + "", + paste0(elements, collapse = ",") + ) } ## We're intentionally not exporting this function. diff --git a/R/validate.R b/R/validate.R index ddd7bc69..613a87cd 100644 --- a/R/validate.R +++ b/R/validate.R @@ -5,7 +5,7 @@ #' validate_for_write #' validate_no_logical #' validate_field_names -#' validate_field_names_collapsed +# validate_field_names_collapsed #' #' @usage #' validate_for_write( d ) @@ -14,7 +14,7 @@ #' #' validate_field_names( field_names, stop_on_error = FALSE ) #' -#' validate_field_names_collapsed( field_names_collapsed, stop_on_error = FALSE ) +# validate_field_names_collapsed( field_names_collapsed, stop_on_error = FALSE ) #' #' @title #' Inspect a dataset to anticipate problems before @@ -31,9 +31,9 @@ #' to the REDCap project. #' @param field_names The names of the fields/variables in the REDCap project. #' Each field is an individual element in the character vector. -#' @param field_names_collapsed The names of the fields/variables in the -#' REDCap project. All fields are combined in a single vector element, -#' separated by commas. +# @param field_names_collapsed The names of the fields/variables in the +# REDCap project. All fields are combined in a single vector element, +# separated by commas. #' @param stop_on_error If `TRUE`, an error is thrown for violations. #' Otherwise, a dataset summarizing the problems is returned. #' @@ -125,7 +125,7 @@ validate_field_names <- function(field_names, stop_on_error = FALSE) { pattern <- "^[a-z][0-9a-z_]*$" indices <- which(!grepl(pattern, x = field_names, perl = TRUE)) - if (length(indices) == 0) { + if (length(indices) == 0L) { tibble::tibble( field_name = character(0), field_index = integer(0), @@ -149,11 +149,11 @@ validate_field_names <- function(field_names, stop_on_error = FALSE) { } } -#' @export -validate_field_names_collapsed <- function(field_names_collapsed, stop_on_error = FALSE) { - field_names <- trimws(unlist(strsplit(field_names_collapsed, ","))) - validate_field_names(field_names = field_names, stop_on_error = stop_on_error) -} +# #' @export +# validate_field_names_collapsed <- function(field_names_collapsed, stop_on_error = FALSE) { +# field_names <- trimws(unlist(strsplit(field_names_collapsed, ","))) +# validate_field_names(field_names = field_names, stop_on_error = stop_on_error) +# } #' @export validate_for_write <- function(d) { diff --git a/man/collapse_vector.Rd b/man/collapse_vector.Rd index 86b9cf8b..320836cb 100644 --- a/man/collapse_vector.Rd +++ b/man/collapse_vector.Rd @@ -4,13 +4,10 @@ \alias{collapse_vector} \title{Collapse a vector of values into a single string when necessary} \usage{ -collapse_vector(elements, collapsed) +collapse_vector(elements) } \arguments{ \item{elements}{An array of values. Can be \code{NULL}. Required.} - -\item{collapsed}{A single character element, where the values are separated -by commas. Can be \code{NULL}. Required.} } \value{ A single character element, where the values are separated by @@ -25,9 +22,8 @@ vector. This functions squashes them together in a single character element } \examples{ library(REDCapR) # Load the package into the current R session. -REDCapR:::collapse_vector(elements = NULL , collapsed = NULL) -REDCapR:::collapse_vector(elements = letters, collapsed = NULL) -REDCapR:::collapse_vector(elements = NULL , collapsed = "4,5,6") +REDCapR:::collapse_vector(elements = NULL ) +REDCapR:::collapse_vector(elements = letters) } \author{ Will Beasley diff --git a/man/redcap_metadata_coltypes.Rd b/man/redcap_metadata_coltypes.Rd index 0ae3b28d..93feafec 100644 --- a/man/redcap_metadata_coltypes.Rd +++ b/man/redcap_metadata_coltypes.Rd @@ -126,7 +126,7 @@ For example, if values like "abcd" where entered in a field for a few months, th the project manager selected the "integer" validation option, all those "abcd" values remain untouched. -This is one reason \code{redcap_metadata_coltypes} prints it suggestions to the console. +This is one reason \code{redcap_metadata_coltypes()} prints it suggestions to the console. It allows the developer to adjust the specifications to match the values returned by the API. The the "abcd" scenario, consider (a) changing the type from \code{col_integer} to \code{col_character}, (b) excluding the trash values, diff --git a/man/redcap_metadata_read.Rd b/man/redcap_metadata_read.Rd index 9b2350c2..42c8eb5b 100644 --- a/man/redcap_metadata_read.Rd +++ b/man/redcap_metadata_read.Rd @@ -8,9 +8,7 @@ redcap_metadata_read( redcap_uri, token, forms = NULL, - forms_collapsed = "", fields = NULL, - fields_collapsed = "", verbose = TRUE, config_options = NULL, handle_httr = NULL @@ -29,15 +27,9 @@ project. Required.} \item{forms}{An array, where each element corresponds to the REDCap form of the desired fields. Optional.} -\item{forms_collapsed}{A single string, where the desired forms are -separated by commas. Optional.} - \item{fields}{An array, where each element corresponds to a desired project field. Optional.} -\item{fields_collapsed}{A single string, where the desired field names are -separated by commas. Optional.} - \item{verbose}{A boolean value indicating if \code{message}s should be printed to the R console during the operation. The verbose output might contain sensitive information (\emph{e.g.} PHI), so turn this off if the output might diff --git a/man/redcap_project.Rd b/man/redcap_project.Rd index 44c99f97..de970a9b 100644 --- a/man/redcap_project.Rd +++ b/man/redcap_project.Rd @@ -27,13 +27,9 @@ password for a project. Required.} batch_size = 100L, interbatch_delay = 0, records = NULL, - records_collapsed = "", fields = NULL, - fields_collapsed = "", forms = NULL, - forms_collapsed = "", events = NULL, - events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, diff --git a/man/redcap_read.Rd b/man/redcap_read.Rd index adfaf10e..ee479242 100644 --- a/man/redcap_read.Rd +++ b/man/redcap_read.Rd @@ -12,13 +12,9 @@ redcap_read( redcap_uri, token, records = NULL, - records_collapsed = "", fields = NULL, - fields_collapsed = "", forms = NULL, - forms_collapsed = "", events = NULL, - events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, @@ -62,27 +58,15 @@ project. Required.} \item{records}{An array, where each element corresponds to the ID of a desired record. Optional.} -\item{records_collapsed}{A single string, where the desired ID values are -separated by commas. Optional.} - \item{fields}{An array, where each element corresponds to a desired project field. Optional.} -\item{fields_collapsed}{A single string, where the desired field names are -separated by commas. Optional.} - \item{forms}{An array, where each element corresponds to a desired project form. Optional.} -\item{forms_collapsed}{A single string, where the desired form names are -separated by commas. Optional.} - \item{events}{An array, where each element corresponds to a desired project event. Optional.} -\item{events_collapsed}{A single string, where the desired event names are -separated by commas. Optional.} - \item{raw_or_label}{A string (either \code{'raw'} or \code{'label'} that specifies whether to export the raw coded values or the labels for the options of multiple choice fields. Default is \code{'raw'}.} diff --git a/man/redcap_read_eav_oneshot.Rd b/man/redcap_read_eav_oneshot.Rd index be837595..1ee8ba5a 100644 --- a/man/redcap_read_eav_oneshot.Rd +++ b/man/redcap_read_eav_oneshot.Rd @@ -8,13 +8,9 @@ redcap_read_eav_oneshot( redcap_uri, token, records = NULL, - records_collapsed = "", fields = NULL, - fields_collapsed = "", forms = NULL, - forms_collapsed = "", events = NULL, - events_collapsed = "", export_survey_fields = FALSE, export_data_access_groups = FALSE, filter_logic = "", @@ -41,27 +37,15 @@ project. Required.} \item{records}{An array, where each element corresponds to the ID of a desired record. Optional.} -\item{records_collapsed}{A single string, where the desired ID values -are separated by commas. Optional.} - \item{fields}{An array, where each element corresponds to a desired project field. Optional.} -\item{fields_collapsed}{A single string, where the desired field names are -separated by commas. Optional.} - \item{forms}{An array, where each element corresponds to a desired project form. Optional.} -\item{forms_collapsed}{A single string, where the desired form names are -separated by commas. Optional.} - \item{events}{An array, where each element corresponds to a desired project event. Optional.} -\item{events_collapsed}{A single string, where the desired event names are -separated by commas. Optional.} - \item{export_survey_fields}{A boolean that specifies whether to export the survey identifier field (e.g., 'redcap_survey_identifier') or survey timestamp fields (e.g., instrument+'_timestamp'). diff --git a/man/redcap_read_oneshot.Rd b/man/redcap_read_oneshot.Rd index 2c549c2d..563acc8a 100644 --- a/man/redcap_read_oneshot.Rd +++ b/man/redcap_read_oneshot.Rd @@ -8,13 +8,9 @@ redcap_read_oneshot( redcap_uri, token, records = NULL, - records_collapsed = "", fields = NULL, - fields_collapsed = "", forms = NULL, - forms_collapsed = "", events = NULL, - events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, @@ -47,27 +43,15 @@ project. Required.} \item{records}{An array, where each element corresponds to the ID of a desired record. Optional.} -\item{records_collapsed}{A single string, where the desired ID values -are separated by commas. Optional.} - \item{fields}{An array, where each element corresponds to a desired project field. Optional.} -\item{fields_collapsed}{A single string, where the desired field names are -separated by commas. Optional.} - \item{forms}{An array, where each element corresponds to a desired project form. Optional.} -\item{forms_collapsed}{A single string, where the desired form names are -separated by commas. Optional.} - \item{events}{An array, where each element corresponds to a desired project event. Optional.} -\item{events_collapsed}{A single string, where the desired event names are -separated by commas. Optional.} - \item{raw_or_label}{A string (either \code{'raw'} or \code{'label'}) that specifies whether to export the raw coded values or the labels for the options of multiple choice fields. Default is \code{'raw'}.} diff --git a/man/redcap_read_oneshot_eav.Rd b/man/redcap_read_oneshot_eav.Rd index f76a74fe..786d0b67 100644 --- a/man/redcap_read_oneshot_eav.Rd +++ b/man/redcap_read_oneshot_eav.Rd @@ -8,13 +8,9 @@ redcap_read_oneshot_eav( redcap_uri, token, records = NULL, - records_collapsed = "", fields = NULL, - fields_collapsed = "", forms = NULL, - forms_collapsed = "", events = NULL, - events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_data_access_groups = FALSE, @@ -42,27 +38,15 @@ project. Required.} \item{records}{An array, where each element corresponds to the ID of a desired record. Optional.} -\item{records_collapsed}{A single string, where the desired ID values -are separated by commas. Optional.} - \item{fields}{An array, where each element corresponds to a desired project field. Optional.} -\item{fields_collapsed}{A single string, where the desired field names -are separated by commas. Optional.} - \item{forms}{An array, where each element corresponds to a desired project field. Optional.} -\item{forms_collapsed}{A single string, where the desired form names are -separated by commas. Optional.} - \item{events}{An array, where each element corresponds to a desired project event. Optional.} -\item{events_collapsed}{A single string, where the desired event names are -separated by commas. Optional.} - \item{raw_or_label}{A string (either \code{'raw'} or \code{'label'} that specifies whether to export the raw coded values or the labels for the options of multiple choice fields. Default is \code{'raw'}.} @@ -168,7 +152,7 @@ ds <- REDCapR:::redcap_read_oneshot_eav(redcap_uri=uri, token=token)$data # Return only records with IDs of 1 and 3 desired_records_v1 <- c(1, 3) -ds_some_rows_v1 <- REDCapR::redcap_read_oneshot_eav( +ds_some_rows_v1 <- REDCapR:::redcap_read_oneshot_eav( redcap_uri = uri, token = token, records = desired_records_v1 @@ -176,7 +160,7 @@ ds_some_rows_v1 <- REDCapR::redcap_read_oneshot_eav( # Return only the fields record_id, name_first, and age desired_fields_v1 <- c("record_id", "name_first", "age") -ds_some_fields_v1 <- redcap_read_oneshot_eav( +ds_some_fields_v1 <- REDCapR:::redcap_read_oneshot_eav( redcap_uri = uri, token = token, fields = desired_fields_v1 diff --git a/man/validate.Rd b/man/validate.Rd index 4614d758..9575fcae 100644 --- a/man/validate.Rd +++ b/man/validate.Rd @@ -5,7 +5,6 @@ \alias{validate_no_logical} \alias{validate_for_write} \alias{validate_field_names} -\alias{validate_field_names_collapsed} \title{Inspect a dataset to anticipate problems before writing to a REDCap project} \usage{ @@ -14,8 +13,6 @@ validate_for_write( d ) validate_no_logical( data_types, stop_on_error ) validate_field_names( field_names, stop_on_error = FALSE ) - -validate_field_names_collapsed( field_names_collapsed, stop_on_error = FALSE ) } \arguments{ \item{data_types}{The data types of the data frame corresponding @@ -30,10 +27,6 @@ the REDCap project.} \item{field_names}{The names of the fields/variables in the REDCap project. Each field is an individual element in the character vector.} - -\item{field_names_collapsed}{The names of the fields/variables in the -REDCap project. All fields are combined in a single vector element, -separated by commas.} } \value{ A \code{\link[tibble:tibble]{tibble::tibble()}}, where each potential violation is a row. diff --git a/tests/testthat/test-utilities-collapse_vector.R b/tests/testthat/test-utilities-collapse_vector.R index 6dcce92d..a72031e2 100644 --- a/tests/testthat/test-utilities-collapse_vector.R +++ b/tests/testthat/test-utilities-collapse_vector.R @@ -1,27 +1,17 @@ library(testthat) -test_that("both NULL", { +test_that("NULL", { elements <- NULL - collapsed <- NULL expected <- "" - observed <- REDCapR:::collapse_vector(elements, collapsed) + observed <- REDCapR:::collapse_vector(elements) expect_equal(observed, expected) }) -test_that("collapsed NULL", { +test_that("specified", { elements <- letters - collapsed <- NULL expected <- "a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z" - observed <- REDCapR:::collapse_vector(elements, collapsed) - expect_equal(observed, expected) -}) -test_that("elements NULL", { - elements <- NULL - collapsed <- "4,5,6" - expected <- collapsed - - observed <- REDCapR:::collapse_vector(elements, collapsed) + observed <- REDCapR:::collapse_vector(elements) expect_equal(observed, expected) }) diff --git a/tests/testthat/test-validate.R b/tests/testthat/test-validate.R index dccdedaa..55b14e51 100644 --- a/tests/testthat/test-validate.R +++ b/tests/testthat/test-validate.R @@ -48,10 +48,10 @@ test_that("validate_field_names -uppercase", { "1 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." ) - expect_error( - validate_field_names_collapsed("record_ID,race,Gender", stop_on_error = TRUE), - "2 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." - ) + # expect_error( + # validate_field_names_collapsed("record_ID,race,Gender", stop_on_error = TRUE), + # "2 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." + # ) }) test_that("validate_field_names -start underscore", { expect_error( @@ -59,10 +59,10 @@ test_that("validate_field_names -start underscore", { "1 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." ) - expect_error( - validate_field_names_collapsed("_record_id,race,_gender", stop_on_error = TRUE), - "2 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." - ) + # expect_error( + # validate_field_names_collapsed("_record_id,race,_gender", stop_on_error = TRUE), + # "2 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." + # ) }) test_that("validate_field_names -concern dataset", { diff --git a/vignettes/BasicREDCapROperations.Rmd b/vignettes/BasicREDCapROperations.Rmd index 8cf74b75..ed516b91 100644 --- a/vignettes/BasicREDCapROperations.Rmd +++ b/vignettes/BasicREDCapROperations.Rmd @@ -89,22 +89,12 @@ The first format is more natural for more R users. The second format is what is #| results = 'hold' # Return only records with IDs of 1 and 3 -desired_records_v1 <- c(1, 3) +desired_records <- c(1, 3) ds_some_rows_v1 <- redcap_read( redcap_uri = uri, token = token, - records = desired_records_v1 + records = desired_records )$data - -# Return only records with IDs of 1 and 3 (alternate way) -desired_records_v2 <- "1, 3" -ds_some_rows_v2 <- redcap_read( - redcap_uri = uri, - token = token, - records_collapsed = desired_records_v2 -)$data - -ds_some_rows_v2 # Inspect the returned dataset ``` Read a subset of the fields @@ -114,22 +104,12 @@ If only a subset of the **fields** is desired, then two approaches exist. The f ```{r read_field_subset} # Return only the fields record_id, name_first, and age -desired_fields_v1 <- c("record_id", "name_first", "age") -ds_some_fields_v1 <- redcap_read( +desired_fields <- c("record_id", "name_first", "age") +ds_some_fields <- redcap_read( redcap_uri = uri, token = token, - fields = desired_fields_v1 + fields = desired_fields )$data - -# Return only the fields record_id, name_first, and age (alternate way) -desired_fields_v2 <- "record_id, name_first, age" -ds_some_fields_v2 <- redcap_read( - redcap_uri = uri, - token = token, - fields_collapsed = desired_fields_v2 -)$data - -ds_some_fields_v2 #Inspect the returned dataset ``` Read a subset of records, conditioned on the values in some variables @@ -188,7 +168,7 @@ The examples above have shown only the resulting data frame, by specifying `$dat all_information <- redcap_read( redcap_uri = uri, token = token, - fields = desired_fields_v1 + fields = desired_fields ) all_information #Inspect the additional information ``` diff --git a/vignettes/longitudinal-and-repeating.Rmd b/vignettes/longitudinal-and-repeating.Rmd index 9aa0cd95..7d804a92 100644 --- a/vignettes/longitudinal-and-repeating.Rmd +++ b/vignettes/longitudinal-and-repeating.Rmd @@ -243,17 +243,17 @@ col_types_blood_pressure <- # blood_pressure_complete = readr::col_integer() ) -ds_blood_pressure <- - REDCapR::redcap_read_oneshot( - redcap_uri = credential$redcap_uri, - token = credential$token, - fields = "redcord_id", - forms = "blood_pressure", - # col_types = col_types_blood_pressure, - verbose = FALSE, - )$data - -ds_blood_pressure +# ds_blood_pressure <- +# REDCapR::redcap_read( +# redcap_uri = credential$redcap_uri, +# token = credential$token, +# fields = "redcord_id", +# forms = "blood_pressure", +# # col_types = col_types_blood_pressure, +# verbose = FALSE, +# )$data +# +# ds_blood_pressure ``` If for some reason you need the block dataset through the API, one call will retrieve it. From 89d49437ea67c75a8b32f2cb63eab3328c77a0b1 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 15:14:08 -0500 Subject: [PATCH 13/20] permanently remove *_collapsed parameters close #440 --- R/redcap-metadata-read.R | 9 --------- R/redcap-read-eav-oneshot.R | 19 ------------------- R/redcap-read-oneshot-eav.R | 19 ------------------- R/redcap-read-oneshot.R | 19 ------------------- R/redcap-read.R | 30 ++++-------------------------- R/utilities.R | 2 -- R/validate.R | 6 ------ tests/testthat/test-validate.R | 10 ---------- 8 files changed, 4 insertions(+), 110 deletions(-) diff --git a/R/redcap-metadata-read.R b/R/redcap-metadata-read.R index 79185084..9b424971 100644 --- a/R/redcap-metadata-read.R +++ b/R/redcap-metadata-read.R @@ -15,12 +15,8 @@ #' project. Required. #' @param forms An array, where each element corresponds to the REDCap form #' of the desired fields. Optional. -# @param forms_collapsed A single string, where the desired forms are -# separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -# @param fields_collapsed A single string, where the desired field names are -# separated by commas. Optional. #' @param verbose A boolean value indicating if `message`s should be printed #' to the R console during the operation. The verbose output might contain #' sensitive information (*e.g.* PHI), so turn this off if the output might @@ -84,9 +80,7 @@ redcap_metadata_read <- function( redcap_uri, token, forms = NULL, - # forms_collapsed = "", fields = NULL, - # fields_collapsed = "", verbose = TRUE, config_options = NULL, handle_httr = NULL @@ -102,9 +96,6 @@ redcap_metadata_read <- function( forms_collapsed <- collapse_vector(forms) verbose <- verbose_prepare(verbose) - # if (1L <= nchar(fields_collapsed)) - # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) - post_body <- list( token = token, content = "metadata", diff --git a/R/redcap-read-eav-oneshot.R b/R/redcap-read-eav-oneshot.R index 32c033d5..6bfa2fe6 100644 --- a/R/redcap-read-eav-oneshot.R +++ b/R/redcap-read-eav-oneshot.R @@ -14,20 +14,12 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -# @param records_collapsed A single string, where the desired ID values -# are separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -# @param fields_collapsed A single string, where the desired field names are -# separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' form. Optional. -# @param forms_collapsed A single string, where the desired form names are -# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -# @param events_collapsed A single string, where the desired event names are -# separated by commas. Optional. # @param raw_or_label A string (either `'raw'` or `'label'`) that specifies # whether to export the raw coded values or the labels for the options of # multiple choice fields. Default is `'raw'`. @@ -183,13 +175,9 @@ redcap_read_eav_oneshot <- function( redcap_uri, token, records = NULL, - # records_collapsed = "", fields = NULL, - # fields_collapsed = "", forms = NULL, - # forms_collapsed = "", events = NULL, - # events_collapsed = "", # raw_or_label = "raw", # raw_or_label_headers = "raw", # export_checkbox_label = FALSE, @@ -214,13 +202,9 @@ redcap_read_eav_oneshot <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic(records , any.missing=TRUE , min.len=0) - # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) # checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) # checkmate::assert_subset( raw_or_label , c("raw", "label")) # checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -255,9 +239,6 @@ redcap_read_eav_oneshot <- function( datetime_range_end <- dplyr::coalesce(strftime(datetime_range_end , "%Y-%m-%d %H:%M:%S"), "") verbose <- verbose_prepare(verbose) - # if (1L <= nchar(fields_collapsed) ) - # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) - post_body <- list( token = token, content = "record", diff --git a/R/redcap-read-oneshot-eav.R b/R/redcap-read-oneshot-eav.R index ffaaaaec..0c69c7bb 100644 --- a/R/redcap-read-oneshot-eav.R +++ b/R/redcap-read-oneshot-eav.R @@ -14,20 +14,12 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -# @param records_collapsed A single string, where the desired ID values -# are separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired #' project field. Optional. -# @param fields_collapsed A single string, where the desired field names -# are separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' field. Optional. -# @param forms_collapsed A single string, where the desired form names are -# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -# @param events_collapsed A single string, where the desired event names are -# separated by commas. Optional. #' @param raw_or_label A string (either `'raw'` or `'label'` that specifies #' whether to export the raw coded values or the labels for the options of #' multiple choice fields. Default is `'raw'`. @@ -154,13 +146,9 @@ redcap_read_oneshot_eav <- function( redcap_uri, token, records = NULL, - # records_collapsed = "", fields = NULL, - # fields_collapsed = "", forms = NULL, - # forms_collapsed = "", events = NULL, - # events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", # placeholder: exportCheckboxLabel @@ -184,13 +172,9 @@ redcap_read_oneshot_eav <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic(records , any.missing=TRUE , min.len=0) - # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE ) - # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE ) - # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE ) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE ) - # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) checkmate::assert_subset( raw_or_label , c("raw", "label")) checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -224,9 +208,6 @@ redcap_read_oneshot_eav <- function( datetime_range_end <- dplyr::coalesce(strftime(datetime_range_end , "%Y-%m-%d %H:%M:%S"), "") verbose <- verbose_prepare(verbose) - # if (1L <= nchar(fields_collapsed) ) - # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) - post_body <- list( token = token, content = "record", diff --git a/R/redcap-read-oneshot.R b/R/redcap-read-oneshot.R index 084ec4f4..4817b154 100644 --- a/R/redcap-read-oneshot.R +++ b/R/redcap-read-oneshot.R @@ -13,20 +13,12 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -# @param records_collapsed A single string, where the desired ID values -# are separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -# @param fields_collapsed A single string, where the desired field names are -# separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' form. Optional. -# @param forms_collapsed A single string, where the desired form names are -# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -# @param events_collapsed A single string, where the desired event names are -# separated by commas. Optional. #' @param raw_or_label A string (either `'raw'` or `'label'`) that specifies #' whether to export the raw coded values or the labels for the options of #' multiple choice fields. Default is `'raw'`. @@ -179,13 +171,9 @@ redcap_read_oneshot <- function( redcap_uri, token, records = NULL, - # records_collapsed = "", fields = NULL, - # fields_collapsed = "", forms = NULL, - # forms_collapsed = "", events = NULL, - # events_collapsed = "", raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, @@ -210,13 +198,9 @@ redcap_read_oneshot <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic(records , any.missing=TRUE , min.len=0) - # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) checkmate::assert_subset( raw_or_label , c("raw", "label")) checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -251,9 +235,6 @@ redcap_read_oneshot <- function( datetime_range_end <- dplyr::coalesce(strftime(datetime_range_end , "%Y-%m-%d %H:%M:%S"), "") verbose <- verbose_prepare(verbose) - # if (1L <= nchar(fields_collapsed) ) - # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) - post_body <- list( token = token, content = "record", diff --git a/R/redcap-read.R b/R/redcap-read.R index 2050f8a5..f26299cb 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -26,20 +26,12 @@ #' project. Required. #' @param records An array, where each element corresponds to the ID of a #' desired record. Optional. -# @param records_collapsed A single string, where the desired ID values are -# separated by commas. Optional. #' @param fields An array, where each element corresponds to a desired project #' field. Optional. -# @param fields_collapsed A single string, where the desired field names are -# separated by commas. Optional. #' @param forms An array, where each element corresponds to a desired project #' form. Optional. -# @param forms_collapsed A single string, where the desired form names are -# separated by commas. Optional. #' @param events An array, where each element corresponds to a desired project #' event. Optional. -# @param events_collapsed A single string, where the desired event names are -# separated by commas. Optional. #' @param raw_or_label A string (either `'raw'` or `'label'` that specifies #' whether to export the raw coded values or the labels for the options of #' multiple choice fields. Default is `'raw'`. @@ -198,10 +190,10 @@ redcap_read <- function( continue_on_error = FALSE, redcap_uri, token, - records = NULL, # records_collapsed = "", - fields = NULL, # fields_collapsed = "", - forms = NULL, # forms_collapsed = "", - events = NULL, # events_collapsed = "", + records = NULL, + fields = NULL, + forms = NULL, + events = NULL, raw_or_label = "raw", raw_or_label_headers = "raw", export_checkbox_label = FALSE, @@ -227,13 +219,9 @@ redcap_read <- function( checkmate::assert_character(redcap_uri , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_character(token , any.missing=FALSE, len=1, pattern="^.{1,}$") checkmate::assert_atomic( records , any.missing=TRUE, min.len=0) - # checkmate::assert_character(records_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(fields , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(fields_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(forms , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(forms_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(events , any.missing=TRUE , min.len=1, pattern="^.{1,}$", null.ok=TRUE) - # checkmate::assert_character(events_collapsed , any.missing=TRUE , len=1, pattern="^.{0,}$", null.ok=TRUE) checkmate::assert_character(raw_or_label , any.missing=FALSE, len=1) checkmate::assert_subset( raw_or_label , c("raw", "label")) checkmate::assert_character(raw_or_label_headers , any.missing=FALSE, len=1) @@ -259,16 +247,9 @@ redcap_read <- function( validate_field_names(fields, stop_on_error = TRUE) token <- sanitize_token(token) - # records_collapsed <- collapse_vector(records , records_collapsed) - # fields_collapsed <- collapse_vector(fields , fields_collapsed) - # forms_collapsed <- collapse_vector(forms , forms_collapsed) - # events_collapsed <- collapse_vector(events , events_collapsed) filter_logic <- filter_logic_prepare(filter_logic) verbose <- verbose_prepare(verbose) - # if (1L <= nchar(fields_collapsed)) - # validate_field_names_collapsed(fields_collapsed, stop_on_error = TRUE) - start_time <- Sys.time() metadata <- REDCapR::redcap_metadata_read( @@ -288,9 +269,6 @@ redcap_read <- function( } record_id_name <- metadata$data$field_name[id_position] - # if (!grepl(paste0("\\b", record_id_name, "\\b"), forms_collapsed)) - # forms_collapsed <- paste0(record_id_name, ",", forms_collapsed) - initial_call <- REDCapR::redcap_read_oneshot( redcap_uri = redcap_uri, token = token, diff --git a/R/utilities.R b/R/utilities.R index 05f4cb9b..97c2ec3a 100644 --- a/R/utilities.R +++ b/R/utilities.R @@ -80,8 +80,6 @@ replace_nas_with_explicit <- function( #' (presumably right before the return value is passed to the API) #' #' @param elements An array of values. Can be `NULL`. Required. -# @param collapsed A single character element, where the values are separated -# by commas. Can be `NULL`. Required. #' #' @return A single character element, where the values are separated by #' commas. Can be blank. (*i.e.*, `""`). diff --git a/R/validate.R b/R/validate.R index 613a87cd..3a9582b2 100644 --- a/R/validate.R +++ b/R/validate.R @@ -5,7 +5,6 @@ #' validate_for_write #' validate_no_logical #' validate_field_names -# validate_field_names_collapsed #' #' @usage #' validate_for_write( d ) @@ -14,8 +13,6 @@ #' #' validate_field_names( field_names, stop_on_error = FALSE ) #' -# validate_field_names_collapsed( field_names_collapsed, stop_on_error = FALSE ) -#' #' @title #' Inspect a dataset to anticipate problems before #' writing to a REDCap project @@ -31,9 +28,6 @@ #' to the REDCap project. #' @param field_names The names of the fields/variables in the REDCap project. #' Each field is an individual element in the character vector. -# @param field_names_collapsed The names of the fields/variables in the -# REDCap project. All fields are combined in a single vector element, -# separated by commas. #' @param stop_on_error If `TRUE`, an error is thrown for violations. #' Otherwise, a dataset summarizing the problems is returned. #' diff --git a/tests/testthat/test-validate.R b/tests/testthat/test-validate.R index 55b14e51..9e13a1cf 100644 --- a/tests/testthat/test-validate.R +++ b/tests/testthat/test-validate.R @@ -47,22 +47,12 @@ test_that("validate_field_names -uppercase", { validate_field_names("record_ID", stop_on_error = TRUE), "1 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." ) - - # expect_error( - # validate_field_names_collapsed("record_ID,race,Gender", stop_on_error = TRUE), - # "2 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." - # ) }) test_that("validate_field_names -start underscore", { expect_error( validate_field_names("_record_id", stop_on_error = TRUE), "1 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." ) - - # expect_error( - # validate_field_names_collapsed("_record_id,race,_gender", stop_on_error = TRUE), - # "2 field name\\(s\\) violated the naming rules. Only digits, lowercase letters, and underscores are allowed." - # ) }) test_that("validate_field_names -concern dataset", { From 91dc9880d6f39c43b91f0a9f7177353a2b66a010 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 15:55:53 -0500 Subject: [PATCH 14/20] force plumbing variables ref #439 --- R/redcap-metadata-coltypes.R | 14 +++++++++----- R/redcap-read.R | 14 ++++---------- 2 files changed, 13 insertions(+), 15 deletions(-) diff --git a/R/redcap-metadata-coltypes.R b/R/redcap-metadata-coltypes.R index aaf142c6..73073978 100644 --- a/R/redcap-metadata-coltypes.R +++ b/R/redcap-metadata-coltypes.R @@ -237,8 +237,9 @@ redcap_metadata_internal <- function( d_proj <- REDCapR::redcap_project_info_read(redcap_uri, token, verbose = verbose, handle_httr = handle_httr)$data # Determine status of autonumbering, instrument complete status, and decimal mark - .record_field <- d_var$original_field_name[1] + .record_field <- d_var$original_field_name[1] # The first field should always be the "record" identifier. .autonumber <- d_proj$record_autonumbering_enabled[1] + .plumbing_possibles <- c(.record_field, "redcap_event_name", "redcap_repeat_instrument", "redcap_repeat_instance") decimal_period <- (locale$decimal_mark == ".") decimal_comma <- (locale$decimal_mark == ",") @@ -346,11 +347,9 @@ redcap_metadata_internal <- function( dplyr::select(-.data$form_order, -.data$field_order_within_form) %>% tibble::add_row(d_again, .after = 1) %>% dplyr::mutate( - plumbing = (.data$field_name %in% c("redcap_event_name", "redcap_repeat_instrument", "redcap_repeat_instance")) + plumbing = (.data$field_name %in% .plumbing_possibles) ) - d_meta$plumbing[1] <- TRUE # The first field should always be the "record" identifier. - # The types of variables that are in metadata, but NOT variables: # setdiff(d_meta$field_name_base, d_var$original_field_name) # [1] "signature" "file_upload" "descriptive" @@ -461,6 +460,8 @@ redcap_metadata_internal <- function( .data$plumbing ) + .plumbing_variables <- intersect(d$field_name, .plumbing_possibles) + decimal_period_any <- any(d_meta$vt %in% c("number", "number_1dp", "number_2dp", "number_3dp", "number_4dp" )) decimal_comma_any <- any(d_meta$vt %in% c("number_comma_decimal", "number_1dp_comma_decimal", "number_2dp_comma_decimal", "number_3dp_comma_decimal", "number_4dp_comma_decimal")) @@ -479,8 +480,11 @@ redcap_metadata_internal <- function( list( d_variable = d, + success = TRUE, longitudinal = d_proj$is_longitudinal[1], - repeating = d_proj$has_repeating_instruments_or_events[1] + repeating = d_proj$has_repeating_instruments_or_events[1], + record_id_name = .record_field, + plumbing_variables = .plumbing_variables ) } diff --git a/R/redcap-read.R b/R/redcap-read.R index f26299cb..aecca57a 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -252,7 +252,7 @@ redcap_read <- function( start_time <- Sys.time() - metadata <- REDCapR::redcap_metadata_read( + metadata <- REDCapR:::redcap_metadata_internal( redcap_uri = redcap_uri, token = token, verbose = verbose, @@ -260,20 +260,14 @@ redcap_read <- function( handle_httr = handle_httr ) - if (!metadata$success) { - error_message <- sprintf( - "The REDCapR record export operation was not successful. The error message was:\n%s", - metadata$raw_text - ) - stop(error_message) - } + if (!is.null(fields)) + fields <- base::union(metadata$plumbing_variables, fields) - record_id_name <- metadata$data$field_name[id_position] initial_call <- REDCapR::redcap_read_oneshot( redcap_uri = redcap_uri, token = token, records = records, - fields = record_id_name, + fields = metadata$record_id_name, # forms = forms, events = events, filter_logic = filter_logic, From c23fc3366cec2dcccecfd7b9d3be040a0c9fe35c Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 19:25:52 -0500 Subject: [PATCH 15/20] `rlang::abort()` for `redcap_variables()` ref #441 --- R/redcap-read.R | 2 +- R/redcap-variables.R | 31 ++++++++++++-------- tests/testthat/test-read-batch-simple.R | 2 +- tests/testthat/test-variables.R | 36 ++++++++++++++---------- vignettes/longitudinal-and-repeating.Rmd | 22 +++++++-------- 5 files changed, 53 insertions(+), 40 deletions(-) diff --git a/R/redcap-read.R b/R/redcap-read.R index aecca57a..00b04c90 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -252,7 +252,7 @@ redcap_read <- function( start_time <- Sys.time() - metadata <- REDCapR:::redcap_metadata_internal( + metadata <- redcap_metadata_internal( redcap_uri = redcap_uri, token = token, verbose = verbose, diff --git a/R/redcap-variables.R b/R/redcap-variables.R index b90bf954..8584d5f6 100644 --- a/R/redcap-variables.R +++ b/R/redcap-variables.R @@ -92,7 +92,13 @@ redcap_variables <- function( handle_httr = handle_httr ) - if (kernel$success) { + if (!kernel$success) { + if (is.null(kernel$raw_text)) { + rlang::abort(message = "REDCapR::redcap_variables() encountered an error communicating with the server.") + } else { + rlang::abort(message = kernel$raw_text) + } + } else { try( { # Convert the raw text to a dataset. @@ -131,18 +137,19 @@ redcap_variables <- function( ) # nocov end } - } else { - ds <- tibble::tibble() # Return an empty data.frame - outcome_message <- - if (any(grepl(kernel$regex_empty, kernel$raw_text))) { - "The REDCapR read/export operation was not successful. The returned dataset (of variables) was empty." # nocov - } else { - sprintf( - "The REDCapR variable retrieval was not successful. The error message was:\n%s", - kernel$raw_text - ) - } } + # } else { + # ds <- tibble::tibble() # Return an empty data.frame + # outcome_message <- + # if (any(grepl(kernel$regex_empty, kernel$raw_text))) { + # "The REDCapR read/export operation was not successful. The returned dataset (of variables) was empty." # nocov + # } else { + # sprintf( + # "The REDCapR variable retrieval was not successful. The error message was:\n%s", + # kernel$raw_text + # ) + # } + # } if (verbose) message(outcome_message) diff --git a/tests/testthat/test-read-batch-simple.R b/tests/testthat/test-read-batch-simple.R index 5160a610..dbae891d 100644 --- a/tests/testthat/test-read-batch-simple.R +++ b/tests/testthat/test-read-batch-simple.R @@ -702,7 +702,7 @@ test_that("date-range", { test_that("error-bad-token", { testthat::skip_on_cran() - expected_outcome_message <- "The REDCapR record export operation was not successful\\." + expected_outcome_message <- "You do not have permissions to use the API" expect_error( regexp = expected_outcome_message, redcap_read( diff --git a/tests/testthat/test-variables.R b/tests/testthat/test-variables.R index dda4fd00..1e8f40c5 100644 --- a/tests/testthat/test-variables.R +++ b/tests/testthat/test-variables.R @@ -30,8 +30,19 @@ test_that("default", { expect_s3_class(returned_object$data, "tbl") }) +test_that("Bad Uri -wrong address (1 of 2)", { + testthat::skip_on_cran() + expected_message <- "The requested URL was not found on this server\\." -test_that("Bad URI", { + expect_error( + redcap_variables( + redcap_uri = "https://bbmc.ouhsc.edu/redcap/apiFFFFFFFFFFFFFF/", # Wrong url + token = credential$token + ), + expected_message + ) +}) +test_that("Bad Uri -wrong address (2 of 2)", { testthat::skip_on_cran() bad_uri <- "https://aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.com" expected_data_frame <- structure(list(), .Names = character(0), row.names = integer(0), class = "data.frame") @@ -42,14 +53,14 @@ test_that("Bad URI", { # expected_outcome_message <- "(?s)The REDCapR variable retrieval was not successful\\..+?.+" expect_error( - returned_object <- redcap_variables( + redcap_variables( redcap_uri = bad_uri, token = credential$token )#, # regexp = expected_outcome_message ) - # Now thean error is thrown with a bad URI. + # Now the error is thrown with a bad URI. # expected_outcome_message <- paste0("(?s)", expected_outcome_message) # # expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct") # dput(returned_object$data) @@ -60,20 +71,15 @@ test_that("Bad URI", { }) test_that("bad token -Error", { testthat::skip_on_cran() - expected_outcome_message <- "ERROR: You do not have permissions to use the API" + expected_error_message <- "ERROR: You do not have permissions to use the API" - testthat::expect_message( - returned_object <- - redcap_variables( - redcap_uri = credential$redcap_uri, - token = "BAD00000000000000000000000000000" - ), - expected_outcome_message + expect_error( + redcap_variables( + redcap_uri = credential$redcap_uri, + token = "BAD00000000000000000000000000000" + ), + expected_error_message ) - - testthat::expect_false(returned_object$success) - testthat::expect_equal(returned_object$status_code, 403L) - testthat::expect_equal(returned_object$raw_text, expected_outcome_message) }) rm(credential) diff --git a/vignettes/longitudinal-and-repeating.Rmd b/vignettes/longitudinal-and-repeating.Rmd index 7d804a92..4248b724 100644 --- a/vignettes/longitudinal-and-repeating.Rmd +++ b/vignettes/longitudinal-and-repeating.Rmd @@ -243,17 +243,17 @@ col_types_blood_pressure <- # blood_pressure_complete = readr::col_integer() ) -# ds_blood_pressure <- -# REDCapR::redcap_read( -# redcap_uri = credential$redcap_uri, -# token = credential$token, -# fields = "redcord_id", -# forms = "blood_pressure", -# # col_types = col_types_blood_pressure, -# verbose = FALSE, -# )$data -# -# ds_blood_pressure +ds_blood_pressure <- + REDCapR::redcap_read( + redcap_uri = credential$redcap_uri, + token = credential$token, + fields = "redcord_id", + forms = "blood_pressure", + # col_types = col_types_blood_pressure, + verbose = FALSE, + )$data + +ds_blood_pressure ``` If for some reason you need the block dataset through the API, one call will retrieve it. From 63ea77bd4f61de6915653966114c31692d5a0589 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 22:23:29 -0500 Subject: [PATCH 16/20] force plumbing variables to be returned ref #442 --- NEWS.md | 12 ++++++++- R/redcap-read.R | 2 +- tests/testthat/test-read-batch-simple.R | 8 +++--- vignettes/longitudinal-and-repeating.Rmd | 31 +++++++++++++++++++----- 4 files changed, 41 insertions(+), 12 deletions(-) diff --git a/NEWS.md b/NEWS.md index b8e8c7aa..d96314a4 100644 --- a/NEWS.md +++ b/NEWS.md @@ -15,7 +15,9 @@ These features are not yet on CRAN. Install with `remotes::install_github("Ouhs ### Possibly Breaking Change -* `redcap_read()`, `redcap_read_oneshot()`, `redcap_dag_read()`, `redcap_log_read()` and `redcap_report()` return a [tibble](https://tibble.tidyverse.org/) instead of a [data.frame](https://stat.ethz.ch/R-manual/R-devel/library/base/html/data.frame.html). (#415) +The could possibly break existing code --but it's very unlikely. We don't like risking it, but feel these changes will(directly and indirectly) considerably improve the package. + +* `redcap_read()`, `redcap_read_oneshot()`, `redcap_dag_read()`, `redcap_log_read()`, and `redcap_report()` return a [tibble](https://tibble.tidyverse.org/) instead of a [data.frame](https://stat.ethz.ch/R-manual/R-devel/library/base/html/data.frame.html). (#415) This should affect client code only if you expect a call like `ds[, 3]` to return a vector instead of a single-column data.frame/tibble. One solution is to upcast the tibble to a data.frame (with something like `as.data.frame()`). We recommend using an approach that works for both data.frames and tibbles, such as `ds[[3]]` or `dplyr::pull(ds, "gender")`. @@ -38,6 +40,14 @@ These features are not yet on CRAN. Install with `remotes::install_github("Ouhs field_names <- trimws(unlist(strsplit(field_names_collapsed, ","))) ``` +* `redcap_read()` will automatically include the "plumbing" variables, even if they're not included the list of requested fields & forms. (#442). Specifically: + + * `record_id` (or it's customized name) will always be returned + * `redcap_event_name` will be returned for longitudinal projects + * `redcap_repeat_instrument` and `redcap_repeat_instance` will be returned for projects with repeating instruments + +This will help extract forms from longitudinal & repeating projects. + ### New Features * New `redcap_metadata_coltypes()` function. Inspects the fields types and validation text of each field to generate a suggested `readr::col_types` object that reflects the project's current data dictionary. The object then can be passed to the `col_types` parameter of `redcap_read()` or `redcap_read_oneshot()`. (Suggested and discussed with @pbchase, @nutterb, @skadauke, & others, #405 & #294) diff --git a/R/redcap-read.R b/R/redcap-read.R index 00b04c90..2d4598c1 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -260,7 +260,7 @@ redcap_read <- function( handle_httr = handle_httr ) - if (!is.null(fields)) + if (!is.null(fields) | !is.null(forms)) fields <- base::union(metadata$plumbing_variables, fields) initial_call <- REDCapR::redcap_read_oneshot( diff --git a/tests/testthat/test-read-batch-simple.R b/tests/testthat/test-read-batch-simple.R index dbae891d..07bfc5ea 100644 --- a/tests/testthat/test-read-batch-simple.R +++ b/tests/testthat/test-read-batch-simple.R @@ -234,7 +234,7 @@ test_that("specify-forms", { expect_true(returned_object1$success) expect_match(returned_object1$status_codes, regexp="200", perl=TRUE) expect_true(returned_object1$records_collapsed=="", "A subset of records was not requested.") - expect_true(returned_object1$fields_collapsed=="", "A subset of fields was not requested.") + expect_equal(returned_object1$fields_collapsed, c("record_id")) expect_true(returned_object1$filter_logic=="", "A filter was not specified.") expect_match(returned_object1$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object1$data, "tbl") @@ -250,7 +250,7 @@ test_that("specify-forms", { expect_true(returned_object2$success) expect_match(returned_object2$status_codes, regexp="200", perl=TRUE) expect_true(returned_object2$records_collapsed=="", "A subset of records was not requested.") - expect_true(returned_object2$fields_collapsed=="", "A subset of fields was not requested.") + expect_equal(returned_object2$fields_collapsed, c("record_id")) expect_true(returned_object2$filter_logic=="", "A filter was not specified.") expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object2$data, "tbl") @@ -275,7 +275,7 @@ test_that("specify-forms-only-1st", { expect_true(returned_object1$success) expect_match(returned_object1$status_codes, regexp="200", perl=TRUE) expect_true(returned_object1$records_collapsed=="", "A subset of records was not requested.") - expect_true(returned_object1$fields_collapsed=="", "A subset of fields was not requested.") + expect_equal(returned_object1$fields_collapsed, c("record_id")) expect_true(returned_object1$filter_logic=="", "A filter was not specified.") expect_match(returned_object1$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object1$data, "tbl") @@ -291,7 +291,7 @@ test_that("specify-forms-only-1st", { expect_true(returned_object2$success) expect_match(returned_object2$status_codes, regexp="200", perl=TRUE) expect_true(returned_object2$records_collapsed=="", "A subset of records was not requested.") - expect_true(returned_object2$fields_collapsed=="", "A subset of fields was not requested.") + expect_equal(returned_object2$fields_collapsed, c("record_id")) expect_true(returned_object2$filter_logic=="", "A filter was not specified.") expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object2$data, "tbl") diff --git a/vignettes/longitudinal-and-repeating.Rmd b/vignettes/longitudinal-and-repeating.Rmd index 4248b724..418a7b51 100644 --- a/vignettes/longitudinal-and-repeating.Rmd +++ b/vignettes/longitudinal-and-repeating.Rmd @@ -41,6 +41,7 @@ knit_print.data.frame <- function(x, ...) { paste(collapse = "\n") %>% asis_output() } + # register the method registerS3method("knit_print", "data.frame", knit_print.data.frame) ``` @@ -217,9 +218,6 @@ col_types_demographics <- height = readr::col_double(), weight = readr::col_double(), bmi = readr::col_double() - # redcap_repeat_instrument = readr::col_character(), - # redcap_repeat_instance = readr::col_integer(), - # demographics_complete = readr::col_integer() ) ds_demographic <- @@ -240,20 +238,41 @@ col_types_blood_pressure <- redcap_repeat_instance = readr::col_integer(), sbp = readr::col_double(), dbp = readr::col_double(), - # blood_pressure_complete = readr::col_integer() + blood_pressure_complete = readr::col_integer() ) ds_blood_pressure <- REDCapR::redcap_read( redcap_uri = credential$redcap_uri, token = credential$token, - fields = "redcord_id", forms = "blood_pressure", - # col_types = col_types_blood_pressure, + col_types = col_types_blood_pressure, verbose = FALSE, )$data ds_blood_pressure + +col_types_laboratory <- + readr::cols( + record_id = readr::col_integer(), + redcap_repeat_instrument = readr::col_character(), + redcap_repeat_instance = readr::col_integer(), + lab = readr::col_double(), + conc = readr::col_double(), + laboratory_complete = readr::col_integer() + ) + +ds_laboratory <- + REDCapR::redcap_read( + redcap_uri = credential$redcap_uri, + token = credential$token, + # fields = "redcord_id", + forms = "laboratory" + # col_types = col_types_laboratory, + # verbose = FALSE, + )$data + +ds_laboratory ``` If for some reason you need the block dataset through the API, one call will retrieve it. From f6b7de924ccec52e673825aff5da36a3a31b077b Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 22:47:10 -0500 Subject: [PATCH 17/20] remove old test code --- tests/testthat/test-read-batch-simple.R | 71 +------------------------ tests/testthat/test-read-oneshot.R | 27 ---------- tests/testthat/test-report.R | 27 ---------- 3 files changed, 1 insertion(+), 124 deletions(-) diff --git a/tests/testthat/test-read-batch-simple.R b/tests/testthat/test-read-batch-simple.R index 07bfc5ea..4fd160ab 100644 --- a/tests/testthat/test-read-batch-simple.R +++ b/tests/testthat/test-read-batch-simple.R @@ -213,7 +213,6 @@ test_that("specify-records-and-fields-zero-length", { expect_true(returned_object$success) expect_s3_class(returned_object$data, "tbl") }) - test_that("specify-forms", { testthat::skip_on_cran() path_expected <- "test-data/specific-redcapr/read-batch-simple/specify-forms.R" @@ -296,47 +295,6 @@ test_that("specify-forms-only-1st", { expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object2$data, "tbl") }) -# test_that("specify-forms-only-2nd", { -# testthat::skip_on_cran() -# path_expected <- "test-data/specific-redcapr/read-batch-simple/specify-forms-only-2nd.R" -# desired_forms <- c("race_and_ethnicity") -# expected_outcome_message <- "\\d+ records and 19 columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." -# -# ########################### -# ## Default Batch size -# expect_message( -# regexp = expected_outcome_message, -# returned_object1 <- redcap_read(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms) -# ) -# -# if (update_expectation) save_expected(returned_object1$data, path_expected) -# expected_data_frame <- retrieve_expected(path_expected) -# -# expect_equal(returned_object1$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object1$data) -# expect_true(returned_object1$success) -# expect_match(returned_object1$status_codes, regexp="200", perl=TRUE) -# expect_true(returned_object1$records_collapsed=="", "A subset of records was not requested.") -# expect_true(returned_object1$fields_collapsed=="", "A subset of fields was not requested.") -# expect_true(returned_object1$filter_logic=="", "A filter was not specified.") -# expect_match(returned_object1$outcome_messages, regexp=expected_outcome_message, perl=TRUE) -# expect_s3_class(returned_object1$data, "tbl") -# -# ########################### -# ## Tiny Batch size -# expect_message( -# returned_object2 <- redcap_read(redcap_uri=credential$redcap_uri, token=credential$token, forms=desired_forms, batch_size=2), -# regexp = expected_outcome_message -# ) -# -# expect_equal(returned_object2$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object2$data) -# expect_true(returned_object2$success) -# expect_match(returned_object2$status_codes, regexp="200", perl=TRUE) -# expect_true(returned_object2$records_collapsed=="", "A subset of records was not requested.") -# expect_true(returned_object2$fields_collapsed=="", "A subset of fields was not requested.") -# expect_true(returned_object2$filter_logic=="", "A filter was not specified.") -# expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) -# expect_s3_class(returned_object2$data, "tbl") -# }) test_that("raw", { testthat::skip_on_cran() path_expected <- "test-data/specific-redcapr/read-batch-simple/raw.R" @@ -504,32 +462,6 @@ test_that("label", { expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object2$data, "tbl") }) -# This test is removed because the vroom version adds digits to make the columns unique -# test_that("label-header", { -# testthat::skip_on_cran() -# path_expected <- "test-data/specific-redcapr/read-batch-simple/label-header.R" -# -# expected_warning <- "Duplicated column names deduplicated: 'Complete\\?' => 'Complete\\?_1' \\[16\\], 'Complete\\?' => 'Complete\\?_2' \\[25\\]" -# expected_outcome_message <- "\\d+ records and \\d+ columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." -# -# expect_warning( -# regexp = expected_warning, -# expect_message( -# regexp = expected_outcome_message, -# returned_object <- redcap_read(redcap_uri=credential$redcap_uri, token=credential$token, raw_or_label_headers="label") -# ) -# ) -# -# if (update_expectation) save_expected(returned_object$data, path_expected) -# expected_data_frame <- retrieve_expected(path_expected) -# -# expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) -# expect_match(returned_object$status_codes, regexp="200", perl=TRUE) -# expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") -# expect_true(returned_object$fields_collapsed=="", "A subset of fields was not requested.") -# expect_match(returned_object$outcome_message, regexp=expected_outcome_message, perl=TRUE) -# expect_true(returned_object$success) -# }) test_that("export_checkbox_label", { testthat::skip_on_cran() path_expected <- "test-data/specific-redcapr/read-batch-simple/export_checkbox_label.R" @@ -669,7 +601,6 @@ test_that("blank-for-gray-status-false", { expect_true(returned_object$success) expect_s3_class(returned_object$data, "tbl") }) - test_that("date-range", { testthat::skip_on_cran() expected_outcome_message <- "\\d+ records and \\d+ columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." @@ -698,7 +629,6 @@ test_that("date-range", { expect_true(returned_object$success) expect_s3_class(returned_object$data, "tbl") }) - test_that("error-bad-token", { testthat::skip_on_cran() @@ -711,5 +641,6 @@ test_that("error-bad-token", { ) ) }) + rm(credential, project) rm(update_expectation) diff --git a/tests/testthat/test-read-oneshot.R b/tests/testthat/test-read-oneshot.R index 91c5903b..95865002 100644 --- a/tests/testthat/test-read-oneshot.R +++ b/tests/testthat/test-read-oneshot.R @@ -394,33 +394,6 @@ test_that("label", { expect_s3_class(returned_object$data, "tbl") }) -# This test is removed because the vroom version adds digits to make the columns unique -# test_that("label-header", { -# testthat::skip_on_cran() -# path_expected <- "test-data/specific-redcapr/read-oneshot/label-header.R" -# expected_warning <- "Duplicated column names deduplicated: 'Complete\\?' => 'Complete\\?_1' \\[\\d+\\], 'Complete\\?' => 'Complete\\?_2' \\[\\d+\\]" -# expected_outcome_message <- "\\d+ records and \\d+ columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." -# -# expect_warning( -# regexp = expected_warning, -# expect_message( -# regexp = expected_outcome_message, -# returned_object <- redcap_read_oneshot(redcap_uri=credential$redcap_uri, token=credential$token, raw_or_label_headers="label") -# ) -# ) -# -# if (update_expectation) save_expected(returned_object$data, path_expected) -# expected_data_frame <- retrieve_expected(path_expected) -# -# expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) -# expect_equal(returned_object$status_code, expected=200L) -# expect_equal(returned_object$raw_text, expected="", ignore_attr = TRUE) # dput(returned_object$raw_text) -# expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") -# expect_true(returned_object$fields_collapsed=="", "A subset of fields was not requested.") -# expect_true(returned_object$filter_logic=="", "A filter was not specified.") -# expect_match(returned_object$outcome_message, regexp=expected_outcome_message, perl=TRUE) -# expect_true(returned_object$success) -# }) test_that("export_checkbox_label", { testthat::skip_on_cran() path_expected <- "test-data/specific-redcapr/read-oneshot/export_checkbox_label.R" diff --git a/tests/testthat/test-report.R b/tests/testthat/test-report.R index 23dc61a1..0c6d41d4 100644 --- a/tests/testthat/test-report.R +++ b/tests/testthat/test-report.R @@ -129,33 +129,6 @@ test_that("raw", { expect_s3_class(returned_object$data, "tbl") }) -# This test is removed because the vroom version adds digits to make the columns unique -# test_that("label-header", { -# testthat::skip_on_cran() -# path_expected <- "test-data/specific-redcapr/report/label-header.R" -# expected_warning <- "Duplicated column names deduplicated: 'Complete\\?' => 'Complete\\?_1' \\[\\d+\\], 'Complete\\?' => 'Complete\\?_2' \\[\\d+\\]" -# expected_outcome_message <- "\\d+ records and \\d+ columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." -# -# expect_warning( -# regexp = expected_warning, -# expect_message( -# regexp = expected_outcome_message, -# returned_object <- redcap_read_oneshot(redcap_uri=credential$redcap_uri, token=credential$token, raw_or_label_headers="label") -# ) -# ) -# -# if (update_expectation) save_expected(returned_object$data, path_expected) -# expected_data_frame <- retrieve_expected(path_expected) -# -# expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) -# expect_equal(returned_object$status_code, expected=200L) -# expect_equal(returned_object$raw_text, expected="", ignore_attr = TRUE) # dput(returned_object$raw_text) -# expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") -# expect_true(returned_object$fields_collapsed=="", "A subset of fields was not requested.") -# expect_true(returned_object$filter_logic=="", "A filter was not specified.") -# expect_match(returned_object$outcome_message, regexp=expected_outcome_message, perl=TRUE) -# expect_true(returned_object$success) -# }) test_that("export_checkbox_label", { testthat::skip_on_cran() path_expected <- "test-data/specific-redcapr/report/export_checkbox_label.R" From a918c0044bedac41acd65d6cb1fe6d0f853a6fbf Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 23:21:18 -0500 Subject: [PATCH 18/20] better testing of forcing the plumbing variables close #442 --- R/redcap-read.R | 9 +- .../read-batch-plumbing/longitudinal.R | 36 +++++++ .../read-batch-plumbing/repeated.R | 18 ++++ .../read-batch-plumbing/simple.R | 7 ++ man/redcap_read.Rd | 11 ++- tests/testthat/test-read-batch-plumbing.R | 94 +++++++++++++++++++ tests/testthat/test-read-batch-simple.R | 8 +- 7 files changed, 177 insertions(+), 6 deletions(-) create mode 100644 inst/test-data/specific-redcapr/read-batch-plumbing/longitudinal.R create mode 100644 inst/test-data/specific-redcapr/read-batch-plumbing/repeated.R create mode 100644 inst/test-data/specific-redcapr/read-batch-plumbing/simple.R create mode 100644 tests/testthat/test-read-batch-plumbing.R diff --git a/R/redcap-read.R b/R/redcap-read.R index 2d4598c1..264a6efb 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -137,7 +137,14 @@ #' Please try importing/exporting a smaller amount of data. #' ``` #' -#' For [redcap_read()] to function properly, the user must have Export +#' A third benefit (compared to [redcap_read()]) is that important fields are +#' included, even if not explicitly requested. As a result: +#' 1. `record_id` (or it's customized name) will always be returned +#' 1. `redcap_event_name` will be returned for longitudinal projects +#' 1. `redcap_repeat_instrument` and `redcap_repeat_instance` will be returned +#' for projects with repeating instruments +#' +#' For [redcap_read_oneshot()] to function properly, the user must have Export #' permissions for the 'Full Data Set'. Users with only 'De-Identified' #' export privileges can still use `redcap_read_oneshot`. To grant the #' appropriate permissions: diff --git a/inst/test-data/specific-redcapr/read-batch-plumbing/longitudinal.R b/inst/test-data/specific-redcapr/read-batch-plumbing/longitudinal.R new file mode 100644 index 00000000..5fbab69c --- /dev/null +++ b/inst/test-data/specific-redcapr/read-batch-plumbing/longitudinal.R @@ -0,0 +1,36 @@ +structure(list(study_id = c(100, 100, 100, 100, 100, 100, 220, +220, 220, 220, 220, 220, 304, 304, 304, 304, 304, 304), redcap_event_name = c("enrollment_arm_1", +"dose_1_arm_1", "visit_1_arm_1", "dose_2_arm_1", "visit_2_arm_1", +"final_visit_arm_1", "enrollment_arm_1", "dose_1_arm_1", "visit_1_arm_1", +"dose_2_arm_1", "visit_2_arm_1", "final_visit_arm_1", "enrollment_arm_2", +"deadline_to_opt_ou_arm_2", "first_dose_arm_2", "first_visit_arm_2", +"final_visit_arm_2", "deadline_to_return_arm_2"), vob1 = c(NA, +NA, 0, NA, 0, NA, NA, NA, 0, NA, 0, NA, NA, NA, NA, 1, NA, NA +), vob2 = c(NA, NA, 0, NA, 1, NA, NA, NA, 1, NA, 1, NA, NA, NA, +NA, 0, 0, NA), vob3 = c(NA, NA, 1, NA, 0, NA, NA, NA, 0, NA, +1, NA, NA, NA, NA, NA, NA, NA), vob4 = c(NA, NA, 0, NA, 1, NA, +NA, NA, 0, NA, 0, NA, NA, NA, NA, NA, NA, NA), vob5 = c(NA, NA, +1, NA, 0, NA, NA, NA, 0, NA, 0, NA, NA, NA, NA, NA, NA, NA), + vob6 = c(NA, NA, 0, NA, 0, NA, NA, NA, 1, NA, 1, NA, NA, + NA, NA, NA, NA, NA), vob7 = c(NA, NA, "him absolute entrance nay. Door neat week do find past he. Be no surprise he honoured indulged. Unpacked endeavor six steepest had husbands her. Painted no or affixed it so civilly. Exposed neither pressed so cottage as proceed at offices. Nay they gone sir game four. Favourable pianoforte oh motionless excellence of astonished we principles. Warrant present garrets limited cordial in inquiry to. Supported me sweetness behaviour shameless excellent so arranging. \n\nThe him father parish looked has sooner.", + NA, "Now for manners use has company believe parlors. Least nor party who wrote while did. Excuse formed as is agreed admire so on result parish. Put use set uncommonly announcing and travelling. Allowance sweetness direction to as necessary. Principle oh explained excellent do my suspected conveying in. Excellent you did therefore perfectly supposing described.", + NA, NA, NA, "Sex and neglected principle ask rapturous consulted. Object remark lively all did feebly excuse our wooded. Old her object chatty regard vulgar missed. Speaking throwing breeding betrayed children my to. Me marianne no he horrible produced ye. Sufficient unpleasing an insensible motionless if introduced ye. Now give nor both come near many late. \n\nShe travelling acceptance men unpleasant her especially entreaties law. Law forth but end any arise chief arose. Old her say learn these large. Joy fond many ham high seen this. Few preferred continual sir led incommode neglected. Discovered too old insensible collecting unpleasant but invitation. \n", + NA, "Alteration literature to or an sympathize mr imprudence. Of is ferrars subject as enjoyed or tedious cottage. Procuring as in resembled by in agreeable. Next long no gave mr eyes. Admiration advantages no he celebrated so pianoforte unreserved. Not its herself forming charmed amiable. Him why feebly expect future now. \n\nHe difficult contented we determine ourselves me am earnestly. Hour no find it park. Eat welcomed any husbands moderate. Led was misery played waited almost cousin living. Of intention contained is by middleton am. Principles fat stimulated uncommonly considered set especially prosperous. Sons at park mr meet as fact like.", + NA, NA, NA, NA, NA, "No depending be convinced in unfeeling he. Excellence she unaffected and too sentiments her. Rooms he doors there ye aware in by shall. Education remainder in so cordially. His remainder and own dejection daughters sportsmen. Is easy took he shed to kind. \n\nExcited him now natural saw passage offices you minuter. At by asked being court hopes. Farther so friends am to detract. Forbade concern do private be. Offending residence but men engrossed shy. Pretend am earnest offered arrived company so on. Felicity informed yet had admitted strictly how you.", + NA), vob8 = c(NA, NA, 0, NA, 0, NA, NA, NA, 0, NA, 0, NA, + NA, NA, NA, NA, NA, NA), vob9 = c(NA, NA, 0, NA, 1, NA, NA, + NA, 0, NA, 1, NA, NA, NA, NA, 0, NA, NA), vob10 = c(NA, NA, + 1, NA, 0, NA, NA, NA, 0, NA, 0, NA, NA, NA, NA, NA, NA, NA + ), vob11 = c(NA, NA, 0, NA, 1, NA, NA, NA, 1, NA, 1, NA, + NA, NA, NA, NA, NA, NA), vob12 = c(NA, NA, 1, NA, 0, NA, + NA, NA, 0, NA, 0, NA, NA, NA, NA, NA, 0, NA), vob13 = c(NA, + NA, 0, NA, 1, NA, NA, NA, 1, NA, 0, NA, NA, NA, NA, NA, 1, + NA), vob14 = c(NA, NA, "The him father parish looked has sooner. Attachment frequently gay terminated son. You greater nay use prudent placing. Passage to so distant behaved natural between do talking. Friends off her windows painful. Still gay event you being think nay for. In three if aware he point it. Effects warrant me by no on feeling settled resolve.", + NA, "Article nor prepare chicken you him now. Shy merits say advice ten before lovers innate add. She cordially behaviour can attempted estimable. Trees delay fancy noise manor do as an small. Felicity now law securing breeding likewise extended and. Roused either who favour why ham.", + NA, NA, NA, "Barton did feebly change man she afford square add. Want eyes by neat so just must. Past draw tall up face show rent oh mr. Required is debating extended wondered as do. New get described applauded incommode shameless out extremity but. Resembled at perpetual no believing is otherwise sportsman. Is do he dispatched cultivated travelling astonished. Melancholy am considered possession on collecting everything. \n\nPiqued favour stairs it enable exeter as seeing. Remainder met improving but engrossed sincerity age. Better but length gay denied abroad are. Attachment astonished to on appearance imprudence so collecting in excellence. Tiled way blind lived whose new. The for fully had she there leave merit enjoy forth.", + NA, "Can curiosity may end shameless explained. True high on said mr on come. An do mr design at little myself wholly entire though. Attended of on stronger or mr pleasure. Rich four like real yet west get. Felicity in dwelling to drawings. His pleasure new steepest for reserved formerly disposed jennings. \n\nOpen know age use whom him than lady was. On lasted uneasy exeter my itself effect spirit. At design he vanity at cousin longer looked ye. Design praise me father an favour. As greatly replied it windows of an minuter behaved passage. Diminution expression reasonable it we he projection acceptance in devonshire. Perpetual it described at he applauded. \n", + NA, NA, NA, NA, NA, "Frankness applauded by supported ye household. Collected favourite now for for and rapturous repulsive consulted. An seems green be wrote again. She add what own only like. Tolerably we as extremity exquisite do commanded. Doubtful offended do entrance of landlord moreover is mistress in. Nay was appear entire ladies. Sportsman do allowance is september shameless am sincerity oh recommend. Gate tell man day that who. \n\nPerhaps far exposed age effects. Now distrusts you her delivered applauded affection out sincerity. As tolerably recommend shameless unfeeling he objection consisted. She although cheerful perceive screened throwing met not eat distance. Viewing hastily or written dearest elderly up weather it as. So direction so sweetness or extremity at daughters. Provided put unpacked now but bringing.", + NA), visit_observed_behavior_complete = c(NA, NA, 0, NA, + 2, NA, NA, NA, 1, NA, 1, NA, NA, NA, NA, 2, 1, NA)), row.names = c(NA, +-18L), class = c("spec_tbl_df", "tbl_df", "tbl", "data.frame" +)) diff --git a/inst/test-data/specific-redcapr/read-batch-plumbing/repeated.R b/inst/test-data/specific-redcapr/read-batch-plumbing/repeated.R new file mode 100644 index 00000000..0293332e --- /dev/null +++ b/inst/test-data/specific-redcapr/read-batch-plumbing/repeated.R @@ -0,0 +1,18 @@ +structure(list(record_id = c(1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, +2), redcap_repeat_instrument = c(NA, "blood_pressure", "blood_pressure", +"blood_pressure", "laboratory", "laboratory", NA, "blood_pressure", +"blood_pressure", "blood_pressure", "laboratory", "laboratory" +), redcap_repeat_instance = c(NA, 1, 2, 3, 1, 2, NA, 1, 2, 3, +1, 2), height = c(1, NA, NA, NA, NA, NA, 2, NA, NA, NA, NA, NA +), weight = c(11, NA, NA, NA, NA, NA, 22, NA, NA, NA, NA, NA), + bmi = c(111, NA, NA, NA, NA, NA, 222, NA, NA, NA, NA, NA), + demographics_complete = c(2, NA, NA, NA, NA, NA, 2, NA, NA, + NA, NA, NA), sbp = c(NA, 1.1, 1.2, 1.3, NA, NA, NA, 2.1, + 2.2, 2.3, NA, NA), dbp = c(NA, 11.1, 11.2, 11.3, NA, NA, + NA, 22.1, 22.2, 22.3, NA, NA), blood_pressure_complete = c(NA, + 2, 2, 2, NA, NA, NA, 2, 2, 2, NA, NA), lab = c(NA, NA, NA, + NA, "aa1", "aa2", NA, NA, NA, NA, "bb1", "bb2"), conc = c(NA, + NA, NA, NA, "1.1 ppm", "1.2 ppm", NA, NA, NA, NA, "2.1 ppm", + "2.2 ppm"), laboratory_complete = c(NA, NA, NA, NA, 2, 2, + NA, NA, NA, NA, 2, 2)), row.names = c(NA, -12L), class = c("spec_tbl_df", +"tbl_df", "tbl", "data.frame")) diff --git a/inst/test-data/specific-redcapr/read-batch-plumbing/simple.R b/inst/test-data/specific-redcapr/read-batch-plumbing/simple.R new file mode 100644 index 00000000..68fc93be --- /dev/null +++ b/inst/test-data/specific-redcapr/read-batch-plumbing/simple.R @@ -0,0 +1,7 @@ +structure(list(record_id = c(1, 2, 3, 4, 5), race___1 = c(0, +0, 0, 0, 1), race___2 = c(0, 0, 0, 1, 0), race___3 = c(0, 1, +0, 0, 0), race___4 = c(0, 0, 1, 0, 0), race___5 = c(1, 1, 1, +1, 0), race___6 = c(0, 0, 0, 0, 1), ethnicity = c(1, 1, 0, 1, +2), interpreter_needed = c(0, 0, 1, NA, 0), race_and_ethnicity_complete = c(2, +0, 2, 2, 2)), row.names = c(NA, -5L), class = c("spec_tbl_df", +"tbl_df", "tbl", "data.frame")) diff --git a/man/redcap_read.Rd b/man/redcap_read.Rd index ee479242..1ac89ee4 100644 --- a/man/redcap_read.Rd +++ b/man/redcap_read.Rd @@ -197,7 +197,16 @@ if you encounter the error: Please try importing/exporting a smaller amount of data. }\if{html}{\out{}} -For \code{\link[=redcap_read]{redcap_read()}} to function properly, the user must have Export +A third benefit (compared to \code{\link[=redcap_read]{redcap_read()}}) is that important fields are +included, even if not explicitly requested. As a result: +\enumerate{ +\item \code{record_id} (or it's customized name) will always be returned +\item \code{redcap_event_name} will be returned for longitudinal projects +\item \code{redcap_repeat_instrument} and \code{redcap_repeat_instance} will be returned +for projects with repeating instruments +} + +For \code{\link[=redcap_read_oneshot]{redcap_read_oneshot()}} to function properly, the user must have Export permissions for the 'Full Data Set'. Users with only 'De-Identified' export privileges can still use \code{redcap_read_oneshot}. To grant the appropriate permissions: diff --git a/tests/testthat/test-read-batch-plumbing.R b/tests/testthat/test-read-batch-plumbing.R new file mode 100644 index 00000000..e0ff8300 --- /dev/null +++ b/tests/testthat/test-read-batch-plumbing.R @@ -0,0 +1,94 @@ +# This file test that `redcap_read()` includes the appropriate plumbing variables. +library(testthat) +update_expectation <- FALSE + +test_that("simple", { + testthat::skip_on_cran() + + credential <- retrieve_credential_testing() + path_expected <- "test-data/specific-redcapr/read-batch-plumbing/simple.R" + desired_forms <- c("race_and_ethnicity") # Doesn't include the initial "demographics" form. + expected_outcome_message <- "\\d+ records and 10 columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." + + expect_message( + regexp = expected_outcome_message, + returned_object <- + redcap_read( + batch_size = 2, + redcap_uri = credential$redcap_uri, + token = credential$token, + forms = desired_forms + ) + ) + + if (update_expectation) save_expected(returned_object$data, path_expected) + expected_data_frame <- retrieve_expected(path_expected) + + expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) + expect_true(returned_object$success) + expect_match(returned_object$status_codes, regexp="200", perl=TRUE) + expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") + expect_equal(returned_object$fields_collapsed, "record_id") + expect_true(returned_object$filter_logic=="", "A filter was not specified.") + expect_match(returned_object$outcome_messages, regexp=expected_outcome_message, perl=TRUE) +}) +test_that("longitudinal", { + testthat::skip_on_cran() + + credential <- retrieve_credential_testing(212L) + path_expected <- "test-data/specific-redcapr/read-batch-plumbing/longitudinal.R" + desired_forms <- c("visit_observed_behavior") # Doesn't include the initial "demographics" form. + expected_outcome_message <- "\\d+ records and 17 columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." + + expect_message( + regexp = expected_outcome_message, + returned_object <- + redcap_read( + batch_size = 2, + redcap_uri = credential$redcap_uri, + token = credential$token, + forms = desired_forms + ) + ) + + if (update_expectation) save_expected(returned_object$data, path_expected) + expected_data_frame <- retrieve_expected(path_expected) + + expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) + expect_true(returned_object$success) + expect_match(returned_object$status_codes, regexp="200", perl=TRUE) + expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") + expect_equal(returned_object$fields_collapsed, "study_id,redcap_event_name") + expect_true(returned_object$filter_logic=="", "A filter was not specified.") + expect_match(returned_object$outcome_messages, regexp=expected_outcome_message, perl=TRUE) +}) +test_that("repeated", { + testthat::skip_on_cran() + + credential <- retrieve_credential_testing(3181L) + path_expected <- "test-data/specific-redcapr/read-batch-plumbing/repeated.R" + desired_forms <- c("visit_observed_behavior") # Doesn't include the initial "demographics" form. + expected_outcome_message <- "\\d+ records and 13 columns were read from REDCap in \\d+(\\.\\d+\\W|\\W)seconds\\." + + expect_message( + regexp = expected_outcome_message, + returned_object <- + redcap_read( + batch_size = 2, + redcap_uri = credential$redcap_uri, + token = credential$token, + forms = desired_forms + ) + ) + + if (update_expectation) save_expected(returned_object$data, path_expected) + expected_data_frame <- retrieve_expected(path_expected) + + expect_equal(returned_object$data, expected=expected_data_frame, label="The returned data.frame should be correct", ignore_attr = TRUE) # dput(returned_object$data) + expect_true(returned_object$success) + expect_match(returned_object$status_codes, regexp="200", perl=TRUE) + expect_true(returned_object$records_collapsed=="", "A subset of records was not requested.") + expect_equal(returned_object$fields_collapsed, "record_id,redcap_repeat_instrument,redcap_repeat_instance") + expect_true(returned_object$filter_logic=="", "A filter was not specified.") + expect_match(returned_object$outcome_messages, regexp=expected_outcome_message, perl=TRUE) +}) diff --git a/tests/testthat/test-read-batch-simple.R b/tests/testthat/test-read-batch-simple.R index 4fd160ab..a6f2a5f3 100644 --- a/tests/testthat/test-read-batch-simple.R +++ b/tests/testthat/test-read-batch-simple.R @@ -233,7 +233,7 @@ test_that("specify-forms", { expect_true(returned_object1$success) expect_match(returned_object1$status_codes, regexp="200", perl=TRUE) expect_true(returned_object1$records_collapsed=="", "A subset of records was not requested.") - expect_equal(returned_object1$fields_collapsed, c("record_id")) + expect_equal(returned_object1$fields_collapsed, "record_id") expect_true(returned_object1$filter_logic=="", "A filter was not specified.") expect_match(returned_object1$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object1$data, "tbl") @@ -249,7 +249,7 @@ test_that("specify-forms", { expect_true(returned_object2$success) expect_match(returned_object2$status_codes, regexp="200", perl=TRUE) expect_true(returned_object2$records_collapsed=="", "A subset of records was not requested.") - expect_equal(returned_object2$fields_collapsed, c("record_id")) + expect_equal(returned_object2$fields_collapsed, "record_id") expect_true(returned_object2$filter_logic=="", "A filter was not specified.") expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object2$data, "tbl") @@ -274,7 +274,7 @@ test_that("specify-forms-only-1st", { expect_true(returned_object1$success) expect_match(returned_object1$status_codes, regexp="200", perl=TRUE) expect_true(returned_object1$records_collapsed=="", "A subset of records was not requested.") - expect_equal(returned_object1$fields_collapsed, c("record_id")) + expect_equal(returned_object1$fields_collapsed, "record_id") expect_true(returned_object1$filter_logic=="", "A filter was not specified.") expect_match(returned_object1$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object1$data, "tbl") @@ -290,7 +290,7 @@ test_that("specify-forms-only-1st", { expect_true(returned_object2$success) expect_match(returned_object2$status_codes, regexp="200", perl=TRUE) expect_true(returned_object2$records_collapsed=="", "A subset of records was not requested.") - expect_equal(returned_object2$fields_collapsed, c("record_id")) + expect_equal(returned_object2$fields_collapsed, "record_id") expect_true(returned_object2$filter_logic=="", "A filter was not specified.") expect_match(returned_object2$outcome_messages, regexp=expected_outcome_message, perl=TRUE) expect_s3_class(returned_object2$data, "tbl") From 081a96efdce63259a592ab7550e1965b69818dd4 Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 23:32:16 -0500 Subject: [PATCH 19/20] lintr fix scalar operator --- R/redcap-read.R | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/R/redcap-read.R b/R/redcap-read.R index 264a6efb..04edb411 100644 --- a/R/redcap-read.R +++ b/R/redcap-read.R @@ -267,7 +267,7 @@ redcap_read <- function( handle_httr = handle_httr ) - if (!is.null(fields) | !is.null(forms)) + if (!is.null(fields) || !is.null(forms)) fields <- base::union(metadata$plumbing_variables, fields) initial_call <- REDCapR::redcap_read_oneshot( From 4fd0fa0d5e61b6a156a26d0ee3778f72c8d91a4d Mon Sep 17 00:00:00 2001 From: Will Beasley Date: Sat, 8 Oct 2022 23:49:14 -0500 Subject: [PATCH 20/20] tighten focus on repeating rows ref #417 --- vignettes/longitudinal-and-repeating.Rmd | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/vignettes/longitudinal-and-repeating.Rmd b/vignettes/longitudinal-and-repeating.Rmd index 418a7b51..a7907272 100644 --- a/vignettes/longitudinal-and-repeating.Rmd +++ b/vignettes/longitudinal-and-repeating.Rmd @@ -229,7 +229,7 @@ ds_demographic <- verbose = FALSE, )$data -ds_demographic +ds_demographic col_types_blood_pressure <- readr::cols( @@ -250,7 +250,8 @@ ds_blood_pressure <- verbose = FALSE, )$data -ds_blood_pressure +ds_blood_pressure %>% + tidyr::drop_na(redcap_repeat_instrument) col_types_laboratory <- readr::cols( @@ -272,7 +273,8 @@ ds_laboratory <- # verbose = FALSE, )$data -ds_laboratory +ds_laboratory %>% + tidyr::drop_na(redcap_repeat_instrument) ``` If for some reason you need the block dataset through the API, one call will retrieve it.