Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 4 vignettes introducing fixed/group sequential design simulations #306

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

LittleBeannie
Copy link
Collaborator

@LittleBeannie LittleBeannie commented Jan 15, 2025

This issue adds 4 vignettes introducing fixed/group sequential design simulations.

  • Simulate Fixed Designs with Ease via sim_fixed_n
  • Simulate Group Sequential Designs with Ease via sim_gs_n
  • Custom Fixed Design Simulations: A Tutorial on Writing Code from the Ground Up
  • Custom Group Sequential Design Simulations: Crafting from Scratch

@LittleBeannie LittleBeannie added the documentation Improvements or additions to documentation label Jan 15, 2025
@LittleBeannie LittleBeannie self-assigned this Jan 15, 2025
@LittleBeannie LittleBeannie linked an issue Jan 15, 2025 that may be closed by this pull request
4 tasks
@LittleBeannie LittleBeannie marked this pull request as draft January 15, 2025 18:59
@LittleBeannie LittleBeannie marked this pull request as ready for review January 15, 2025 19:50
@LittleBeannie LittleBeannie requested a review from keaven January 15, 2025 19:50
@LittleBeannie
Copy link
Collaborator Author

Hi @jdblischak, I have a question regarding parallel computing. My current implementation is as follows, and I’m wondering if both the library(doFuture) and library(doRNG) are both necessary.

tic()
registerDoFuture()
registerDoRNG()
plan("multisession", workers = 4)
set.seed(2025)

ans <- foreach(
  sim_id = seq_len(n_sim),
  .combine = "rbind",
  .errorhandling = "stop"
  ) %dorng% {
    ans_new <- one_sim(...) # for one single simulation
    ans_new
  }

plan("sequential")
toc()

@LittleBeannie
Copy link
Collaborator Author

LittleBeannie commented Jan 17, 2025

Reference 1: https://github.com/Merck/simtrial/blob/main/R/sim_fixed_n.R#L253

Also reduce the number of workers to 2, per CRAN requirements.

@jdblischak
Copy link
Collaborator

I have a question regarding parallel computing. My current implementation is as follows, and I’m wondering if both the library(doFuture) and library(doRNG) are both necessary.

Following up on our offline discussion:

Your approach using {doFuture} and {doRNG} is valid. It produces reproducible results:

library("foreach")
library("doFuture")
library("doRNG")

registerDoFuture()
registerDoRNG()
plan("multisession", workers = 4)

set.seed(2025)
ans1 <- foreach(
  i = 1:10,
  .combine = "c",
  .errorhandling = "stop"
) %dorng% {
  rnorm(1)
}
head(ans1)
## [1] -0.46387557 -0.05150034 -0.22384683 -0.60535060  1.52634253  0.31122695

set.seed(2025)
ans2 <- foreach(
  i = 1:10,
  .combine = "c",
  .errorhandling = "stop"
) %dorng% {
  rnorm(1)
}
head(ans2)
## [1] -0.46387557 -0.05150034 -0.22384683 -0.60535060  1.52634253  0.31122695

stopifnot(identical(ans1, ans2))

However, in {simtrial}, we've standardized on using {future} itself to handle the parallel random number generation. Using {doRNG} is an alternative to {doParallel}, but isn't required when using {doFuture} as the backend.

library("foreach")
library("doFuture")

registerDoFuture()
plan("multisession", workers = 4)

set.seed(2025)
ans1 <- foreach(
  i = 1:10,
  .combine = "c",
  .errorhandling = "stop",
  .options.future = list(seed = TRUE)
) %dofuture% {
  rnorm(1)
}
head(ans1)
## [1] -1.0860108  1.3469334  0.1314693 -1.5942347  1.3797186  0.2381956

set.seed(2025)
ans2 <- foreach(
  i = 1:10,
  .combine = "c",
  .errorhandling = "stop",
  .options.future = list(seed = TRUE)
) %dofuture% {
  rnorm(1)
}
head(ans2)
## [1] -1.0860108  1.3469334  0.1314693 -1.5942347  1.3797186  0.2381956

stopifnot(identical(ans1, ans2))

Copy link
Collaborator

@jdblischak jdblischak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this code takes a while to run, you can consider only publishing it to the pkgdown site but not including it in the distributed package. See the docs for info at https://r-pkgs.org/vignettes.html#sec-vignettes-article

tidyr
tidyr,
doRNG,
tictoc
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How informative are the timings in the vignettes? Is it worth adding another dependency to be installed during each CI run?

If the simulations are small, and they are running on minimal hardware like a CRAN or GitHub machine, it's not clear to me if this will be that informative to an end user that will want to run many simulations on more robust hardware.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add 4 vignettes introducing fixed/group sequential design simulations
2 participants