added tests and small aspects to articles

R/import_LL.R

@@ -66,28 +66,56 @@
 #' ```
 #' 
 #'   ## ActLumus 
+#'   Manufacturer: Condor Instruments
+#'   Model: ActLumus
+#'   Implemented: 2023
 #'   A sample file is provided with the package, it can be accessed through
 #'   `system.file("extdata/205_actlumus_Log_1020_20230904101707532.txt.zip",
 #'   package = "LightLogR")`. It does not need to be unzipped to be imported.
 #'   This sample file is a good example for a regular dataset without gaps
-#'   ## LYS 
+#'   ## LYS
+#'   Manufacturer: LYS Technologies
+#'   Model: LYS Button
+#'   Implemented: 2023
 #'   A sample file is provided with the package, it can be accessed
 #'   through `system.file("extdata/sample_data_LYS.csv", package =
 #'   "LightLogR")`. This sample file is a good example for an irregular dataset.
 #'   ## Actiwatch_Spectrum
+#'   Manufacturer: Philips Respironics
+#'   Model: Actiwatch Spectrum
+#'   Implemented: 2023
 #'   **Required Argument: `column_names`** A character vector containing column 
 #'   names in the order in which they appear in the file. This is necessary to 
 #'   find the starting point of actual data.
 #'   ## ActTrust
+#'   Manufacturer: Condor Instruments
+#'   Model: ActTrust1, ActTrust2
+#'   Implemented: 2024
 #'   This function works for both ActTrust 1 and 2 devices
 #'   ## Speccy
-#'   .  
+#'   Manufacturer: Monash University
+#'   Model: Speccy
+#'   Implemented: 2024
 #'   ## DeLux
-#'   . 
+#'   Manufacturer: Intelligent Automation Inc
+#'   Model: DeLux
+#'   Implemented: 2023
 #'   ## LiDo
-#'   .  
+#'   Manufacturer: University of Lucerne
+#'   Model: LiDo
+#'   Implemented: 2023
 #'   ## SpectraWear
-#'   .
+#'   Manufacturer:
+#'   Model: SpectraWear
+#'   Implemented: 2024
+#'   ## NanoLambda
+#'   Manufacturer: NanoLambda
+#'   Model: XL-500 BLE
+#'   Implemented: 2024
+#'   ## LightWatcher
+#'   Manufacturer: Object-Tracker
+#'   Model: LightWatcher
+#'   Implemented: 2024
 #'
 #' @section Examples:
 #'

tests/testthat/test-aggregate_Date.R

@@ -0,0 +1,29 @@
+test_that("aggregate_Date works", {
+  dates <- sample.data.environment
+  #aggregate_Date works
+  expect_no_error(aggregate_Date(dates))
+
+  #only one day left
+  dates_agg <- aggregate_Date(dates)
+  expect_equal(lubridate::date(dates_agg$Datetime) %>% unique() %>% length(), 1)
+  #leaving unit = "none" (default) yields different results depending on Id
+  expect_equal(dates_agg %>% dominant_epoch() %>% 
+                 dplyr::pull(dominant.epoch) %>% 
+                 as.numeric, 
+               c(30, 10)
+               )
+  #changing the date.handler to max works
+  dates_agg2 <- aggregate_Date(dates, date.handler = max)
+  expect_equal(lubridate::date(dates_agg2$Datetime) %>% 
+                 unique() %>% 
+                 as.character(), 
+               "2023-08-20")
+  #changing the numeric unit works
+  dates_agg3 <- aggregate_Date(dates, unit = "15 mins")
+  expect_equal(dates_agg3 %>% 
+                 count_difftime() %>% 
+                 dplyr::pull(difftime) %>% 
+                 as.numeric, 
+               c(15*60, 15*60)
+               )
+  })

tests/testthat/test-double_date.R

@@ -0,0 +1,58 @@
+test_that("repeat_date and next date works", {
+  #some example datetimes
+  dates <- tibble::tibble(
+    Datetime = as.POSIXct(
+      c("2023-08-20 10:00:00", 
+        "2023-08-20 10:15:00", 
+        "2023-08-20 10:30:00", 
+        "2023-08-20 10:45:00"
+        )
+      )
+  )
+
+  #repeat_date works
+  expect_equal(nrow(repeat_date(dates)), 2*nrow(dates))
+  expect_equal(repeat_date(dates) %>% 
+                 dplyr::pull(Datetime) %>% 
+                 lubridate::date() %>% 
+                 unique() %>% as.character(), 
+               c("2023-08-20", "2023-08-21"))
+  expect_equal(repeat_date(dates) %>% length(), 2)
+  expect_equal(repeat_date(dates) %>% dplyr::group_vars(), "Date.data")
+  expect_true(repeat_date(dates) %>% 
+                 create_Timedata() %>% 
+                 dplyr::pull(Time.data) %>% {all(.[1:4] == .[5:8])})
+
+  #next_date works
+  next_dates <- next_date(repeat_date(dates) %>% dplyr::ungroup())
+  expect_equal(nrow(next_dates), 3*nrow(dates))
+  expect_equal(next_dates %>% 
+                 dplyr::pull(Datetime) %>% 
+                 lubridate::date() %>% unique() %>% as.character(), 
+               c("2023-08-20", "2023-08-21"))
+  expect_equal(next_dates %>% 
+                 dplyr::pull(Datetime) %>% 
+                 lubridate::date() %>% table() %>% unname() %>% as.numeric(), 
+               c(4, 8))
+  expect_equal(next_dates %>% length(), 2)
+  expect_equal(next_dates %>% dplyr::group_vars(), "Date.data")
+})
+
+test_that("double_date works", {
+  #some example datetimes
+  dates <- tibble::tibble(
+    Datetime = as.POSIXct(
+      c("2023-08-20 10:00:00", 
+        "2023-08-20 10:15:00", 
+        "2023-08-20 10:30:00", 
+        "2023-08-20 10:45:00"
+      )
+    )
+  )
+  next_dates <- next_date(repeat_date(dates) %>% dplyr::ungroup())
+
+  #double_date correctly identifies the correct type of date
+  expect_equal(double_date(dates), repeat_date(dates))
+  expect_equal(double_date(repeat_date(dates) %>% dplyr::ungroup()), 
+               next_dates)
+})

vignettes/articles/Import.Rmd

@@ -9,15 +9,41 @@ knitr::opts_chunk$set(
 )
 ```
 
-This article focuses on the import from multiple files and participants, as well as the cleaning of the data. We need these packages
+This article focuses on the import from multiple files and participants, as well as the cleaning of the data. We need these packages:
 
 ```{r setup, message = FALSE}
 library(LightLogR)
 library(tidyverse)
 library(gghighlight)
 ```
 
-# Importing Data
+# From which devices can I import data?
+
+LightLogR aims to provide standard import routines for all devices used in research on personal light exposure. Currently, the following devices are supported:
+
+```{r}
+supported.devices
+```
+More Information on these devices can be found in the reference for 
+`import_Dataset()`.
+
+## What if my device is not listed?
+
+If you are using a device that is currently not supported, please contact the developers. We are always looking to expand the range of supported devices. The easiest and most trackable way to get in contact is by opening a new issue on our [Github repository](https://github.com/tscnlab/LightLogR/issues). Please also provide a sample file of your data, so we can test the import function.
+
+## What if my device is listed but the import does not work as expected?
+
+We regularly find that files exported from the same device model can differ in structure. This may be due to different settings, software or hardware updates. If you encounter problems with the import, please get in contact with us, e.g. by opening an issue on our [Github repository](https://github.com/tscnlab/LightLogR/issues). Please provide a sample file of your data.
+
+## Are there other ways to import data?
+
+Yes. `LightLogR` simply requires a `data.frame` with a column containing `datetime` formatted data. Even a `light` data column is not strictly necessary, as `LightLogR` is optimized for, but not restricted to, light data. Further, an `Id` column is used in some functions to distinguish between different participants. 
+
+To make life easier when using functions in `LightLogR`, the datetime column should be named `Datetime`, the id column `Id`, and, if present, the melanopic EDI light information `MEDI`.
+
+Lastly, you can modify or add import functions that build upon `LightLogR`s import functionality. See the last chapter in this article for more information on that.
+
+# Importing data
 
 The first step in every analysis is data import. We will work with data collected as part of the Master Thesis *Insights into real-world human light exposure: relating self-report with eye-level light logging* by Carolina Guidolin (2023). The data is stored in 17 text files in the *data/* folder. You can access the data yourself through the [LightLogR GitHub repository](https://github.com/tscnlab/LightLogR/tree/main/vignettes/articles/data).
 
@@ -50,6 +76,18 @@ Now we can import the data. Data were collected with the ActLumus device by Cond
 data <- import$ActLumus(files, tz = tz, auto.id = pattern, print_n=33)
 ```
 
+## My import is slow. Why is that and can I speed it up?
+
+There are several possibilities, why the import is slow. The most common reasons are:
+
+-   The data files are simply large. With short measurement intervals, many participants, and long measurement periods, files for a single study can easily reach several gigabytes. This takes some time to import and is ok.
+-   The data files contain many gaps. During import, `LightLogR` checks for and visualizes gaps in the data. Especially large datasets with small intervals contain many gaps, which can slow down the import process.
+-   The device model you are importing from has non-consistent data structures. Some devices have a varying number of rows before the actual data starts. This means a small portion of every file has to be read in and the correct starting row has to be found. This can slow down the import process if you have many files.
+
+If you are experiencing slow imports, you can try the following:
+-   Only read in part of your datasets, or split your dataset into several peaces, that each gets loaded in separately. You can combine them afterwards with `join_datasets()`.
+-   If you have many gaps in your data, you can set `auto.plot = FALSE` in the import function. This will eliminate the call to `gg_overview()`, which calculates and visualizes the gaps in the data.
+
 # Data cleaning #1
 
 Before we can dive into the analysis part, we need to make sure we have a clean dataset. The import summary shows us two problems with the data:
@@ -140,3 +178,68 @@ The data is now clean and we can proceed with the analysis. This dataset will be
 ```{r}
 # saveRDS(data, "cleaned_data/ll_data.rds")
 ```
+
+# Importing data: Miscellaneous
+
+## Other import arguments
+
+Other potentially important arguments are the `locale` argument - this is useful if you have special characters in your data (e.g. German ü or ä) that are not recognized by the default locale. Look at `readr::default_locale()` for more information.
+
+The `...` argument is passed through to whichever import function is used for the data. For some devices, it is also used to provide additional information, such as `column_names` for the Actiwatch devices, that differ depending on the language setting of the device and software. Whether a device requires additional information can be found in the import documentation (see `import_Dataset()`).
+
+## Other ways to call import
+
+Instead of using the import function as described above (`import$device()`), you can also use the function `import_Dataset()` and specify the device as a character string in the first argument. This might be useful if you want to import data programmatically from different devices, e.g., through a `purrr::map()` function. Only `supported.devices` will be accepted by the function.
+
+Here is an example:
+
+```
+devices <- c("ActLumus", "Speccy")
+files_AL <- c("path/to/ActLumus/file1.csv", "path/to/ActLumus/file2.csv")
+files_Sy <- c("path/to/Speccy/file1.csv", "path/to/Speccy/file2.csv")
+tz <- "Europe/Berlin"
+
+data <- purrr::map2(devices, list(files_AL, files_Sy), import_Dataset, tz = tz)
+```
+
+This way, you will end with a list of two dataframes, one with the ActLumus data and one with the Speccy data.
+
+## Creating your own import function
+
+**Note: This section is for advanced users only. You should be familiar with expressions in R and how to manipulate them.**
+
+`LightLogR` comes with a number of custom import routines for different devices, that are then implemented into the main import function, which covers some general aspects and also creates the summary overview.
+
+If you would like to write your own custom import function, `LightLogR` has you covered. First, you can see what makes all the import functions tick by looking at the included data set `ll_import_expr`. This is a list of all the individual routines. Let's have a look at the ActLumus routine
+
+```{r}
+ll_import_expr$ActLumus
+```
+We can see, it is rather simple, just a few lines of code. You can write your own expression and create an import function with it. The expression should create a `data` variable that contains the import script for the files. At the end of the expression, the `data` variable should contain the imported dataset and include a correctly formatted `Datetime` column, complete with the correct timezone.
+
+Here we will create a variation of the old routine, that just adds a short message:
+
+```{r}
+ll_import_expr$ActLumus_new <- ll_import_expr$ActLumus
+ll_import_expr$ActLumus_new[[4]] <- 
+  rlang::expr({ cat("**Congratulation, you made a new import function**\n")
+  data
+  })
+ll_import_expr$ActLumus_new
+
+```
+
+We can now create a new import function with this expression. The function will be called `import$ActLumus_new()`.
+
+```{r}
+import <- import_adjustment(ll_import_expr)
+```
+
+Let us now import a file of the previous dataset, setting the main summary and plotting function silent
+
+```{r}
+data <- import$ActLumus_new(files[1], tz = tz, auto.id = pattern, 
+                         auto.plot = FALSE, silent = TRUE)
+
+```
+

vignettes/articles/Metrics.Rmd

@@ -44,6 +44,8 @@ There are a lot of metrics associated with personal light exposure. You can find
 
 We will cover the practical considerations following from these aspects in the following sections. Further, every function documentation explicitly states whether different metrics are accessible through parameters, and which metrics are calculated by default.
 
+**Note: Most metrics require complete and regular data for a sensible output. While some metrics can handle missing data, it is generally advisable to clean the data before calculating metrics. LightLogR helps to identify gaps and irregularities and can also aggregate data to larger intervals, which can be acceptable for small gaps. In cases of larger gaps, dates or participants might have to be removed from analysis.**
+
 # Metric calculation: basics
 
 All metric functions are by default agnostic to the type of data. They require vectors of numeric data (e.g., light data) and commonly also of datetimes. This means that the functions can be used outside of the LightLogR framework, if applied correctly. Let us try this with a simple example for a days worth of light data for one participant across two functions.