Unexpected digest::digest() results within a test_that() block

[louisaslett]

We have encountered a bit of a head scratcher in tests that were failing in a fairly complex package currently under development. I've tracked it down to the following minimal reprexs and we would appreciate any assistance with the root cause. I've a gut instinct this might not be a bug, but rather something subtle in how test_that() is manipulating the runtime environment ... it would be nice to get to the bottom of understanding it either way, so that we can design the tests to work as expected.

One line problem description: Digests of lists not containing functions have the same digest immediately before and after returning from a function call, both interactively and in test_that() blocks, as expected. But, the digest of a list containing a function changes immediately before and after returning from a function call when run within a test_that() block.

OK scenario: No function in the returned list (correct behaviour both interactively and in test_that())

Running interactively at console:

f <- function(n) {
  x <- list(a = rnorm(n), b = "Hello!", c = n)
  print(digest::digest(x))
  return(x)
}

a <- f(10)
## [1] "c223717866fd9ae1b0d4f132a4fac580"
print(digest::digest(a))
## [1] "c223717866fd9ae1b0d4f132a4fac580"

Within a test_digest.R file (I realise there are no actual tests here, I'm trying to keep the example very minimal):

test_that("no function in list",{
  f <- function(n) {
    x <- list(a = rnorm(n), b = "Hello!", c = n)
    print(digest::digest(x))
    return(x)
  }
  
  a <- f(10)
  print(digest::digest(a))
})

Then run through testthat:

devtools::test()
## ℹ Testing reprex
## ⠏ |          0 | digest
## [1] "adb9e4d560424065ffcb0414a32fe9b2"
## [1] "adb9e4d560424065ffcb0414a32fe9b2"

Not OK scenario: Function in the returned list (correct behaviour interactively and unexpected output in test_that())

Running interactively at console:

f <- function(n) {
  x <- list(a = rnorm(n), b = "Hello!", c = n, function(z) { z^2 })
  print(digest::digest(x))
  return(x)
}

a <- f(10)
## [1] "b126ff7bbe0b2c48beb7ace5e1898993"
print(digest::digest(a))
## [1] "b126ff7bbe0b2c48beb7ace5e1898993"

Within a test_digest.R file:

test_that("with function in list",{
  f <- function(n) {
    x <- list(a = rnorm(n), b = "Hello!", c = n, function(z) { z^2 })
    print(digest::digest(x))
    return(x)
  }
  
  a <- f(10)
  print(digest::digest(a))
})

Then run through testthat:

devtools::test()
## ℹ Testing reprex
## ⠏ |          0 | digest
## [1] "1ecd7ae8a2c7f930cb5ded1faa911237"
## [1] "10fe76cf020f9faab8a69949b7647154"

The unexpected behaviour here is that under test_that() we get different digests of the same object immediately before and after the return from f().

Any help understanding this, even if it is not a bug, would be hugely appreciated.

Thanks!