Docker build fails #63

shankari · 2021-08-05T15:58:55Z

The current docker build on master fails.

It gets past all of the initial steps correctly,

 => [ 2/10] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic  72.9s
 => [ 3/10] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li  0.3s
 => [ 4/10] RUN R -e 'install.packages("remotes")'                                                       5.6s
 => [ 5/10] RUN R -e 'install.packages("devtools")'                                                    561.0s
 => [ 6/10] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                         6.3s
 => [ 7/10] RUN mkdir /build_zone                                                                        0.3s
 => [ 8/10] ADD . /build_zone                                                                            0.5s
 => [ 9/10] WORKDIR /build_zone                                                                          0.0s

but fails on the last step due to some download errors?

 => ERROR [10/10] RUN R -e 'remotes::install_local(upgrade="never")'                                    71.7s

#14 71.61 Error in download.file(url, destfile, method, mode = "wb", ...) :
#14 71.61   (converted from warning) downloaded length 288991 != reported length 581347
#14 71.61 Error in download.packages(pkgs, destdir = tmpd, available = available,  :
#14 71.61   (converted from warning) download of package ‘e1071’ failed
#14 71.61 Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
#14 71.66 Execution halted

Here is a more detailed log of that step

 => ERROR [10/10] RUN R -e 'remotes::install_local(upgrade="never")'                                    71.7s
...
#14 0.436 > remotes::install_local(upgrade="never")
#14 3.671 Installing 109 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, curl, data.table, datamods, DBI, diffobj, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, Rcpp, readr, readxl, rematch, rio, rmarkdown, roxygen2, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, waldo, webshot, wk, xtable
#14 3.673 Installing packages into ‘/usr/local/lib/R/site-library’
#14 3.673 (as ‘lib’ is unspecified)
#14 4.464 trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz'
#14 4.871 Content type 'application/x-gzip' length 79261 bytes (77 KB)
#14 4.871 ==================================================
#14 4.897 downloaded 77 KB
#14 4.897
#14 4.899 trying URL 'https://cran.rstudio.com/src/contrib/backports_1.2.1.tar.gz'
#14 5.034 Content type 'application/x-gzip' length 21184 bytes (20 KB)
#14 5.034 ==================================================
#14 5.036 downloaded 20 KB
...
#14 10.60 ==================================================
#14 10.60 downloaded 10 KB
#14 10.60
#14 10.60 trying URL 'https://cran.rstudio.com/src/contrib/dplyr_1.0.7.tar.gz'
#14 10.91 Content type 'application/x-gzip' length 852103 bytes (832 KB)
#14 10.92 ==================================================
#14 11.06 downloaded 832 KB
#14 11.06
#14 11.06 trying URL 'https://cran.rstudio.com/src/contrib/DT_0.18.tar.gz'
#14 11.36 Content type 'application/x-gzip' length 1578749 bytes (1.5 MB)
#14 11.36 ==================================================
#14 11.56 downloaded 1.5 MB
#14 11.56
#14 11.57 trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz'
#14 11.78 Content type 'application/x-gzip' length 581347 bytes (567 KB)
#14 11.78 ========================
#14 71.61 downloaded 282 KB
#14 71.61
#14 71.61 Error in download.file(url, destfile, method, mode = "wb", ...) :
#14 71.61   (converted from warning) downloaded length 288991 != reported length 581347
#14 71.61 Error in download.packages(pkgs, destdir = tmpd, available = available,  :
#14 71.61   (converted from warning) download of package ‘e1071’ failed
#14 71.61 Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
#14 71.66 Execution halted

The text was updated successfully, but these errors were encountered:

shankari · 2021-08-05T17:14:25Z

@allenmichael099 (in case you want to follow along)

Trying to downgrade since this is currently broken.

Downgrading

$ git checkout 72340fb34454e011ac53069b399463925c200d6c
M	docker_build_and_run.sh
Note: switching to '72340fb34454e011ac53069b399463925c200d6c'.
HEAD is now at 72340fb Use install_github for DTedit

Rebuilding, failed while trying to install package remotes

$ docker image rm emdash
Error: No such image: emdash
kshankar-35069s:emdash kshankar$ docker build -t emdash .
[+] Building 327.7s (9/37)
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 3.07kB                                                                   0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.0.2                                            0.0s
 => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2                                                            0.0s
 => [internal] load build context                                                                        0.2s
 => => transferring context: 1.70MB                                                                      0.2s
 => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag  0.0s
 => CACHED [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho  0.0s
 => CACHED [ 4/33] RUN R -e 'install.packages("remotes")'                                                0.0s
 => CANCELED [ 5/33] RUN R -e 'install.packages("devtools")'                                           327.6s
context canceled
kshankar-35069s:emdash kshankar$ docker build -t emdash --no-cache .
[+] Building 80.6s (10/37)
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 43B                                                                      0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.0.2                                            0.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 79.23kB                                                                     0.1s
 => CACHED [ 1/33] FROM docker.io/rocker/r-ver:4.0.2                                                     0.0s
 => [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic  76.0s
 => [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li  0.4s
 => [ 4/33] RUN R -e 'install.packages("remotes")'                                                       1.9s
 => [ 5/33] RUN R -e 'install.packages("devtools")'                                                      1.7s
 => ERROR [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                   0.6s
------
 > [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")':
#9 0.384
#9 0.384 R version 4.0.2 (2020-06-22) -- "Taking Off Again"
#9 0.384 Copyright (C) 2020 The R Foundation for Statistical Computing
#9 0.384 Platform: x86_64-pc-linux-gnu (64-bit)
#9 0.384
#9 0.384 R is free software and comes with ABSOLUTELY NO WARRANTY.
#9 0.384 You are welcome to redistribute it under certain conditions.
#9 0.384 Type 'license()' or 'licence()' for distribution details.
#9 0.384
#9 0.384 R is a collaborative project with many contributors.
#9 0.384 Type 'contributors()' for more information and
#9 0.384 'citation()' on how to cite R or R packages in publications.
#9 0.384
#9 0.384 Type 'demo()' for some demos, 'help()' for on-line help, or
#9 0.384 'help.start()' for an HTML browser interface to help.
#9 0.384 Type 'q()' to quit R.
#9 0.384
#9 0.519 > remotes::install_github("r-lib/remotes", ref = "97bbf81")
#9 0.521 Error in loadNamespace(name) : there is no package called ‘remotes’
#9 0.521 Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
#9 0.521 Execution halted
------
executor failed running [/bin/sh -c R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")']: exit code: 1
kshankar-35069s:emdash kshankar$ docker build -t emdash --no-cache .
[+] Building 195.4s (9/37)
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 43B                                                                      0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.0.2                                            0.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 79.23kB                                                                     0.1s
 => CACHED [ 1/33] FROM docker.io/rocker/r-ver:4.0.2                                                     0.0s
 => [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic  70.8s
 => [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li  0.3s
 => [ 4/33] RUN R -e 'install.packages("remotes")'                                                       4.9s
 => CANCELED [ 5/33] RUN R -e 'install.packages("devtools")'                                           119.3s
context canceled
kshankar-35069s:emdash kshankar$ docker build -t emdash .
[+] Building 0.6s (10/37)
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 43B                                                                      0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.0.2                                            0.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 79.23kB                                                                     0.1s
 => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2                                                            0.0s
 => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag  0.0s
 => CACHED [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho  0.0s
 => CACHED [ 4/33] RUN R -e 'install.packages("remotes")'                                                0.0s
 => CACHED [ 5/33] RUN R -e 'install.packages("devtools")'                                               0.0s
 => ERROR [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                   0.5s
------
 > [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")':
#9 0.372
#9 0.372 R version 4.0.2 (2020-06-22) -- "Taking Off Again"
#9 0.372 Copyright (C) 2020 The R Foundation for Statistical Computing
#9 0.372 Platform: x86_64-pc-linux-gnu (64-bit)
#9 0.372
#9 0.372 R is free software and comes with ABSOLUTELY NO WARRANTY.
#9 0.372 You are welcome to redistribute it under certain conditions.
#9 0.372 Type 'license()' or 'licence()' for distribution details.
#9 0.372
#9 0.372 R is a collaborative project with many contributors.
#9 0.372 Type 'contributors()' for more information and
#9 0.372 'citation()' on how to cite R or R packages in publications.
#9 0.372
#9 0.372 Type 'demo()' for some demos, 'help()' for on-line help, or
#9 0.372 'help.start()' for an HTML browser interface to help.
#9 0.372 Type 'q()' to quit R.
#9 0.372
#9 0.465 > remotes::install_github("r-lib/remotes", ref = "97bbf81")
#9 0.466 Error in loadNamespace(name) : there is no package called ‘remotes’
#9 0.466 Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
#9 0.466 Execution halted
------
executor failed running [/bin/sh -c R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")']: exit code: 1

Commented out the DTEdit lines

diff --git a/Dockerfile b/Dockerfile
index caf55a4..9393843 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -3,8 +3,8 @@ RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core
 RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl')" >> /usr/local/lib/R/etc/Rprofile.site
 RUN R -e 'install.packages("remotes")'
 RUN R -e 'install.packages("devtools")'
-RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'
-RUN Rscript -e 'devtools::install_github("jbryer/DTedit")'
+# RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'
+# RUN Rscript -e 'devtools::install_github("jbryer/DTedit")'
 RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")'
 RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1")'
 RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0")'

 => [11/31] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0.0")'   11.2s
 => ERROR [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.7.  407.8s
------
 > [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.7.8")':
#15 407.8 Error in download_version_url(package, version, repos, type) :
#15 407.8   version '2.7.8' is invalid for package 'mapview'
#15 407.8 Calls: <Anonymous> -> download_version_url
#15 407.8 Execution halted
------
executor failed running [/bin/sh -c Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.7.8")']: exit code: 1

shankari · 2021-08-05T17:48:59Z

Upgraded mapview to 2.10.0. worked!

kshankar-35069s:emdash kshankar$ docker build -t emdash .
[+] Building 2185.4s (36/36) FINISHED
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 43B                                                                      0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.0.2                                            0.0s
 => [ 1/31] FROM docker.io/rocker/r-ver:4.0.2                                                            0.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 91.56kB                                                                     0.1s
 => CACHED [ 2/31] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag  0.0s
 => CACHED [ 3/31] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho  0.0s
 => CACHED [ 4/31] RUN R -e 'install.packages("remotes")'                                                0.0s
 => CACHED [ 5/31] RUN R -e 'install.packages("devtools")'                                               0.0s
 => CACHED [ 6/31] RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")'  0.0s
 => CACHED [ 7/31] RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1")  0.0s
 => CACHED [ 8/31] RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0")  0.0s
 => CACHED [ 9/31] RUN Rscript -e 'remotes::install_version("data.table",upgrade="never", version = "1.  0.0s
 => CACHED [10/31] RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "1.5"  0.0s
 => CACHED [11/31] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0  0.0s
 => [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")'  1073.7s
 => [13/31] RUN Rscript -e 'remotes::install_version("ggplot2",upgrade="never", version = "3.3.0")'     19.0s
 => [14/31] RUN Rscript -e 'remotes::install_version("mongolite",upgrade="never", version = "2.2.0")'   30.2s
 => [15/31] RUN Rscript -e 'remotes::install_version("shinydashboard",upgrade="never", version = "0.7.1  6.7s
 => [16/31] RUN Rscript -e 'remotes::install_version("DT",upgrade="never", version = "0.14")'            8.2s
 => [17/31] RUN Rscript -e 'remotes::install_version("flexdashboard",upgrade="never", version = "0.5.2  15.4s
 => [18/31] RUN Rscript -e 'remotes::install_version("esquisse",upgrade="never", version = "0.3.0")'    19.1s
 => [19/31] RUN Rscript -e 'remotes::install_version("lubridate",upgrade="never", version = "1.7.8")'   32.7s
 => [20/31] RUN Rscript -e 'remotes::install_version("dplyr",upgrade="never", version = "1.0.2")'       18.0s
 => [21/31] RUN Rscript -e 'remotes::install_version("leaflet",upgrade="never", version = "2.0.3")'      9.3s
 => [22/31] RUN Rscript -e 'remotes::install_version("janitor",upgrade="never", version = "2.0.1")'     22.2s
 => [23/31] RUN Rscript -e 'remotes::install_version("tidyr",upgrade="never", version = "1.1.0")'       26.5s
 => [24/31] RUN Rscript -e 'remotes::install_version("shinycssloaders",upgrade="never", version = "0.  323.5s
 => [25/31] RUN Rscript -e 'remotes::install_version("plotly",upgrade="never", version = "4.9.2.1")'    21.1s
 => [26/31] RUN Rscript -e 'remotes::install_version("randomcoloR",upgrade="never", version = "1.1.0.1  48.7s
 => [27/31] RUN Rscript -e 'remotes::install_version("sf",upgrade="never", version = "0.9-5")'         127.6s
 => [28/31] RUN mkdir /build_zone                                                                        0.3s
 => [29/31] ADD . /build_zone                                                                            0.2s
 => [30/31] WORKDIR /build_zone                                                                          0.0s
 => [31/31] RUN R -e 'remotes::install_local(upgrade="never")'                                         365.2s
 => exporting to image                                                                                  17.6s
 => => exporting layers                                                                                 17.6s
 => => writing image sha256:fc2836ca5a8cc9fb78f8a48bb35f85d0f7173bc04a1391433f0f2086a7d2a8cb             0.0s
 => => naming to docker.io/library/emdash                                                                0.0s

shankari · 2021-08-05T17:52:49Z

Restoring DTEdit, fails again

$ docker build -t emdash .
[+] Building 8.1s (11/37)
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 3.07kB                                                                   0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.0.2                                            0.0s
 => [internal] load build context                                                                        0.2s
 => => transferring context: 82.26kB                                                                     0.2s
 => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2                                                            0.0s
 => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag  0.0s
 => CACHED [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho  0.0s
 => CACHED [ 4/33] RUN R -e 'install.packages("remotes")'                                                0.0s
 => CACHED [ 5/33] RUN R -e 'install.packages("devtools")'                                               0.0s
 => [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                         7.5s
 => ERROR [ 7/33] RUN Rscript -e 'devtools::install_github("jbryer/DTedit")'                             0.5s
------
 > [ 7/33] RUN Rscript -e 'devtools::install_github("jbryer/DTedit")':
#10 0.471 Error in loadNamespace(name) : there is no package called ‘devtools’
#10 0.471 Calls: :: ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
#10 0.471 Execution halted
------
executor failed running [/bin/sh -c Rscript -e 'devtools::install_github("jbryer/DTedit")']: exit code: 1

allenmichael099 · 2021-08-05T17:57:22Z

We wondered if the default download timeout was too low, so Shankari is attempting to build with an altered timeout.

Based on this from https://search.r-project.org/R/refmans/utils/html/download.file.html:
The timeout for many parts of the transfer can be set by the option timeout which defaults to 60 seconds. This is often insufficient for downloads of large files (50MB or more) and so should be increased when download.file is used in packages to do so. Note that the user can set the default timeout by the environment variable R_DEFAULT_INTERNET_TIMEOUT in recent versions of R, so to ensure that this is not decreased packages should use something like

options(timeout = max(300, getOption("timeout")))

And from https://stackoverflow.com/questions/35282928/how-do-i-set-a-timeout-for-utilsdownload-file-in-r,

I think we just need to alter timeout within the options call, setting to whatever number of seconds we want

RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl',timeout = 300)" >> /usr/local/lib/R/etc/Rprofile.site

shankari · 2021-08-05T18:04:43Z

Fixed the error there is no package called ‘devtools’ by changing it to remotes like everything else.

-RUN Rscript -e 'devtools::install_github("jbryer/DTedit")'
+RUN Rscript -e 'remotes::install_github("jbryer/DTedit")'

But still ran into a timeout/install issue

 => ERROR [ 7/33] RUN Rscript -e 'remotes::install_github("jbryer/DTedit")'                             57.4s
------
 > [ 7/33] RUN Rscript -e 'remotes::install_github("jbryer/DTedit")':
#10 0.938 Downloading GitHub repo jbryer/DTedit@master
#10 4.417 base64enc   (NA -> 0.1-3  ) [CRAN]
#10 4.417 bslib       (NA -> 0.2.5.1) [CRAN]
#10 4.417 crosstalk   (NA -> 1.1.1  ) [CRAN]
#10 4.417 DT          (NA -> 0.18   ) [CRAN]
#10 4.417 htmltools   (NA -> 0.5.1.1) [CRAN]
#10 4.417 htmlwidgets (NA -> 1.5.3  ) [CRAN]
#10 4.417 httpuv      (NA -> 1.6.1  ) [CRAN]
#10 4.417 jquerylib   (NA -> 0.1.4  ) [CRAN]
#10 4.417 later       (NA -> 1.2.0  ) [CRAN]
#10 4.417 lazyeval    (NA -> 0.2.2  ) [CRAN]
#10 4.417 promises    (NA -> 1.2.0.1) [CRAN]
#10 4.417 sass        (NA -> 0.4.0  ) [CRAN]
#10 4.417 shiny       (NA -> 1.6.0  ) [CRAN]
#10 4.417 sourcetools (NA -> 0.1.7  ) [CRAN]
#10 4.417 xtable      (NA -> 1.8-4  ) [CRAN]
#10 4.419 Installing 15 packages: base64enc, bslib, crosstalk, DT, htmltools, htmlwidgets, httpuv, jquerylib, later, lazyeval, promises, sass, shiny, sourcetools, xtable
#10 4.421 Installing packages into ‘/usr/local/lib/R/site-library’
#10 4.421 (as ‘lib’ is unspecified)
#10 5.124 trying URL 'https://cran.rstudio.com/src/contrib/base64enc_0.1-3.tar.gz'
#10 5.326 Content type 'application/x-gzip' length 7833 bytes
#10 5.326 ==================================================
#10 5.326 downloaded 7833 bytes
...
#10 8.747 trying URL 'https://cran.rstudio.com/src/contrib/shiny_1.6.0.tar.gz'
#10 9.008 Content type 'application/x-gzip' length 4170497 bytes (4.0 MB)
#10 57.31
#10 57.31 downloaded 63 KB
#10 57.31
#10 57.31 Error in download.file(url, destfile, method, mode = "wb", ...) :
#10 57.31   (converted from warning) downloaded length 64755 != reported length 4170497
#10 57.31 Error in download.packages(pkgs, destdir = tmpd, available = available,  :
#10 57.31   (converted from warning) download of package ‘shiny’ failed
#10 57.31 Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
#10 57.32 Execution halted
------
executor failed running [/bin/sh -c Rscript -e 'remotes::install_github("jbryer/DTedit")']: exit code: 1

Trying after upping the timeout.

shankari · 2021-08-05T18:56:33Z

Got past that point, but failed again. I wonder if there are limitations on disk size.

Failed again during download

 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 3.08kB                                                                   0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => [ 1/33] FROM docker.io/rocker/r-ver:4.0.2                                                            0.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 94.60kB                                                                     0.1s
 => CACHED [ 2/33] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag  0.0s
 => [ 3/33] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li  0.3s
 => [ 6/33] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                        14.0s
 => [ 7/33] RUN Rscript -e 'remotes::install_github("jbryer/DTedit")'                                  405.3s
 => [ 8/33] RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")'         5.3s
 => [ 9/33] RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1")'       17.8s
 => [10/33] RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0")'       33.0s
 => [11/33] RUN Rscript -e 'remotes::install_version("data.table",upgrade="never", version = "1.12.8")  36.9s
 => [12/33] RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "1.5")'       5.9s
 => [13/33] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0.0")'   15.0s
 => ERROR [14/33] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10  417.8s
------
 > [14/33] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")':
#17 2.347 Downloading package from url: https://cran.rstudio.com/src/contrib/mapview_2.10.0.tar.gz
#17 6.622 Installing 40 packages: classInt, colorspace, cpp11, DBI, e1071, farver, geojsonsf, geometries, ggplot2, gridExtra, gtable, isoband, jsonify, labeling, leafem, leaflet, leaflet.providers, leafpop, munsell, plyr, png, proxy, rapidjsonr, raster, RColorBrewer, s2, satellite, scales, servr, sf, sfheaders, sp, svglite, systemfonts, units, uuid, viridis, viridisLite, webshot, wk
#17 6.626 Installing packages into ‘/usr/local/lib/R/site-library’
#17 6.626 (as ‘lib’ is unspecified)
#17 7.878 trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz'
#17 8.094 Content type 'application/x-gzip' length 403884 bytes (394 KB)
#17 8.094 ==================================================
#17 8.190 downloaded 394 KB
#17 8.190
#17 8.196 trying URL 'https://cran.rstudio.com/src/contrib/colorspace_2.0-2.tar.gz'
#17 8.603 Content type 'application/x-gzip' length 2196563 bytes (2.1 MB)
#17 8.603 ==================================================
#17 8.945 downloaded 2.1 MB
#17 8.945
#17 8.949 trying URL 'https://cran.rstudio.com/src/contrib/cpp11_0.3.1.tar.gz'
#17 9.145 Content type 'application/x-gzip' length 217952 bytes (212 KB)
#17 9.145 ==================================================
#17 9.200 downloaded 212 KB
#17 9.200
#17 9.205 trying URL 'https://cran.rstudio.com/src/contrib/DBI_1.1.1.tar.gz'
#17 9.617 Content type 'application/x-gzip' length 743802 bytes (726 KB)
#17 9.617 ==================================================
#17 9.690 downloaded 726 KB
#17 9.690
#17 9.697 trying URL 'https://cran.rstudio.com/src/contrib/e1071_1.7-8.tar.gz'
#17 10.22 Content type 'application/x-gzip' length 581347 bytes (567 KB)
#17 10.22 ==================================================
#17 10.28 downloaded 567 KB
#17 10.28
#17 10.28 trying URL 'https://cran.rstudio.com/src/contrib/farver_2.1.0.tar.gz'
#17 10.79 Content type 'application/x-gzip' length 1281864 bytes (1.2 MB)
#17 10.79 =================================================
#17 417.8 downloaded 1.2 MB
#17 417.8
#17 417.8 Error in download.file(url, destfile, method, mode = "wb", ...) :
#17 417.8   (converted from warning) downloaded length 1261298 != reported length 1281864
#17 417.8 Error in download.packages(pkgs, destdir = tmpd, available = available,  :
#17 417.8   (converted from warning) download of package ‘farver’ failed
#17 417.8 Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
#17 417.8 Execution halted
------
executor failed running [/bin/sh -c Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")']: exit code: 1

shankari · 2021-08-05T21:26:39Z

Aha!
https://forums.docker.com/t/cannot-change-default-20g-size-limit-on-windows-docker-host/31879
But that is only on windows.

And the successfully built image is only 3GB.

emdash                                         latest        fc2836ca5a8c   4 hours ago     2.62GB

shankari · 2021-08-06T00:17:35Z

Going back to the successfully built image (checkout to 72340fb and remove the DTEdit stuff)

It built again successfully

docker build -t emdash .
[+] Building 733.9s (11/35)
[+] Building 735.4s (11/35)
[+] Building 5145.0s (36/36) FINISHED
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 43B                                                                      0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.0.2                                            0.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 79.23kB                                                                     0.1s
 => [ 1/31] FROM docker.io/rocker/r-ver:4.0.2                                                            0.0s
 => CACHED [ 2/31] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imag  0.0s
 => CACHED [ 3/31] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.metho  0.0s
 => CACHED [ 4/31] RUN R -e 'install.packages("remotes")'                                                0.0s
 => CACHED [ 5/31] RUN R -e 'install.packages("devtools")'                                               0.0s
 => CACHED [ 6/31] RUN Rscript -e 'remotes::install_version("config",upgrade="never", version = "0.3")'  0.0s
 => [ 7/31] RUN Rscript -e 'remotes::install_version("golem",upgrade="never", version = "0.2.1")'      352.5s
 => [ 8/31] RUN Rscript -e 'remotes::install_version("shiny",upgrade="never", version = "1.5.0")'      632.7s
 => [ 9/31] RUN Rscript -e 'remotes::install_version("data.table",upgrade="never", version = "1.12.8")  24.0s
 => [10/31] RUN Rscript -e 'remotes::install_version("magrittr",upgrade="never", version = "1.5")'       4.6s
 => [11/31] RUN Rscript -e 'remotes::install_version("checkmate",upgrade="never", version = "2.0.0")'   10.5s
 => [12/31] RUN Rscript -e 'remotes::install_version("mapview",upgrade="never", version = "2.10.0")'  2284.5s
 => [13/31] RUN Rscript -e 'remotes::install_version("ggplot2",upgrade="never", version = "3.3.0")'    476.8s
 => [14/31] RUN Rscript -e 'remotes::install_version("mongolite",upgrade="never", version = "2.2.0")'   29.7s
 => [15/31] RUN Rscript -e 'remotes::install_version("shinydashboard",upgrade="never", version = "0.7.1  6.7s
 => [16/31] RUN Rscript -e 'remotes::install_version("DT",upgrade="never", version = "0.14")'            7.7s
 => [17/31] RUN Rscript -e 'remotes::install_version("flexdashboard",upgrade="never", version = "0.5.2  15.5s
 => [18/31] RUN Rscript -e 'remotes::install_version("esquisse",upgrade="never", version = "0.3.0")'    63.4s
 => [19/31] RUN Rscript -e 'remotes::install_version("lubridate",upgrade="never", version = "1.7.8")'   30.8s
 => [20/31] RUN Rscript -e 'remotes::install_version("dplyr",upgrade="never", version = "1.0.2")'       18.4s
 => [21/31] RUN Rscript -e 'remotes::install_version("leaflet",upgrade="never", version = "2.0.3")'     13.1s
 => [22/31] RUN Rscript -e 'remotes::install_version("janitor",upgrade="never", version = "2.0.1")'     21.2s
 => [23/31] RUN Rscript -e 'remotes::install_version("tidyr",upgrade="never", version = "1.1.0")'       27.7s
 => [24/31] RUN Rscript -e 'remotes::install_version("shinycssloaders",upgrade="never", version = "0.3"  6.7s
 => [25/31] RUN Rscript -e 'remotes::install_version("plotly",upgrade="never", version = "4.9.2.1")'    20.5s
 => [26/31] RUN Rscript -e 'remotes::install_version("randomcoloR",upgrade="never", version = "1.1.0.1  51.3s
 => [27/31] RUN Rscript -e 'remotes::install_version("sf",upgrade="never", version = "0.9-5")'         151.8s
 => [28/31] RUN mkdir /build_zone                                                                        0.4s
 => [29/31] ADD . /build_zone                                                                            0.4s
 => [30/31] WORKDIR /build_zone                                                                          0.0s
 => [31/31] RUN R -e 'remotes::install_local(upgrade="never")'                                         888.1s
 => exporting to image                                                                                   5.7s
 => => exporting layers                                                                                  5.7s
 => => writing image sha256:fc0e6b055d52c8963d1753e4fb775af2bd8de7d7abbe360be7aa86b1970f0063             0.0s
 => => naming to docker.io/library/emdash                                                                0.0s

And manually installing the packages works

docker run followed by copy/pasted R script

R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'

R version 4.0.2 (2020-06-22) -- "Taking Off Again"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> remotes::install_github("r-lib/remotes", ref = "97bbf81")
Downloading GitHub repo r-lib/remotes@97bbf81
✔  checking for file ‘/tmp/Rtmp7dzsLc/remotesd4e89362a/r-lib-remotes-97bbf81/DESCRIPTION’ ...
─  preparing ‘remotes’:
✔  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘remotes_2.0.2.tar.gz’

Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘remotes’ ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (remotes)
>
>
root@a6c68d04f4ea:/src# Rscript -e 'remotes::install_github("jbryer/DTedit")'
Downloading GitHub repo jbryer/DTedit@master
DT       (0.14  -> 0.18 ) [CRAN]
magrittr (1.5   -> 2.0.1) [CRAN]
shiny    (1.5.0 -> 1.6.0) [CRAN]
Installing 3 packages: DT, magrittr, shiny
Installing packages into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/DT_0.18.tar.gz'
Content type 'application/x-gzip' length 1578749 bytes (1.5 MB)
==================================================
downloaded 1.5 MB

trying URL 'https://cran.rstudio.com/src/contrib/magrittr_2.0.1.tar.gz'
Content type 'application/x-gzip' length 265580 bytes (259 KB)
==================================================
downloaded 259 KB

trying URL 'https://cran.rstudio.com/src/contrib/shiny_1.6.0.tar.gz'
Content type 'application/x-gzip' length 4170497 bytes (4.0 MB)
==================================================
downloaded 4.0 MB

* installing *source* package ‘magrittr’ ...
** package ‘magrittr’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
gcc -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c pipe.c -o pipe.o
gcc -I"/usr/local/lib/R/include" -DNDEBUG   -I/usr/local/include   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c utils.c -o utils.o
gcc -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o magrittr.so pipe.o utils.o -L/usr/local/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-magrittr/00new/magrittr/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (magrittr)
* installing *source* package ‘shiny’ ...
** package ‘shiny’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (shiny)
* installing *source* package ‘DT’ ...
** package ‘DT’ successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (DT)

The downloaded source packages are in
	‘/tmp/RtmpGEQ84i/downloaded_packages’
✔  checking for file ‘/tmp/RtmpGEQ84i/remotesbb694ecdc0/jbryer-DTedit-0c248c0/DESCRIPTION’ ...
─  preparing ‘DTedit’:
✔  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
─  building ‘DTedit_1.0.0.tar.gz’

Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘DTedit’ ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (DTedit)

shankari · 2021-08-06T01:18:21Z

Trying to go to master again

Installing fails

[+] Building 19.8s (4/14)
[+] Building 20.1s (4/14)
[+] Building 21.2s (4/14)
[+] Building 732.6s (14/14) FINISHED
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 43B                                                                      0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.1.0                                            0.6s
 => [ 1/10] FROM docker.io/rocker/r-ver:4.1.0@sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d1  38.4s
 => => resolve docker.io/rocker/r-ver:4.1.0@sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d10ef  0.0s
 => => sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d10ef5d73fd681597 951B / 951B               0.0s
 => => sha256:d9f4d826c19c1d24c823472294d912b42dca8ddffbd34b2e3d02d5b304f98a4a 4.14kB / 4.14kB           0.0s
 => => sha256:3b480e2036f0f2c081e4cdd2aaa32e5b6c26d60a397792c2a31ea32d34c57805 18.79kB / 18.79kB         0.2s
 => => sha256:643395aa8431033c0fe27ebcbe0f7b802599b75c2b08b820905e7c8a9c4e0f24 295.81MB / 295.81MB      27.2s
 => => extracting sha256:3b480e2036f0f2c081e4cdd2aaa32e5b6c26d60a397792c2a31ea32d34c57805                0.1s
 => => extracting sha256:643395aa8431033c0fe27ebcbe0f7b802599b75c2b08b820905e7c8a9c4e0f24               11.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 91.85kB                                                                     0.1s
 => [ 2/10] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagemagic  66.4s
 => [ 3/10] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'li  0.3s
 => [ 4/10] RUN R -e 'install.packages("remotes")'                                                       4.8s
 => [ 5/10] RUN R -e 'install.packages("devtools")'                                                    535.3s
 => [ 6/10] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                         6.9s
 => [ 7/10] RUN mkdir /build_zone                                                                        0.3s
 => [ 8/10] ADD . /build_zone                                                                            0.3s
 => [ 9/10] WORKDIR /build_zone                                                                          0.0s
 => ERROR [10/10] RUN R -e 'remotes::install_local(upgrade="never")'                                    79.1s
#14 79.01
#14 79.01 Error in download.file(url, destfile, method, mode = "wb", ...) :
#14 79.01   (converted from warning) downloaded length 26586 != reported length 215365
#14 79.01 Error in download.packages(pkgs, destdir = tmpd, available = available,  :
#14 79.01   (converted from warning) download of package ‘janitor’ failed
#14 79.01 Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
#14 79.05 Execution halted
------
executor failed running [/bin/sh -c R -e 'remotes::install_local(upgrade="never")']: exit code: 1

Installing without `remotes::install_local(upgrade="never")'` succeeds

$ docker build -t emdash .
[+] Building 9.5s (15/15) FINISHED
 => [internal] load build definition from Dockerfile                                                     0.0s
 => => transferring dockerfile: 1.06kB                                                                   0.0s
 => [internal] load .dockerignore                                                                        0.0s
 => => transferring context: 2B                                                                          0.0s
 => [internal] load metadata for docker.io/rocker/r-ver:4.1.0                                            1.2s
 => [auth] rocker/r-ver:pull token for registry-1.docker.io                                              0.0s
 => [1/9] FROM docker.io/rocker/r-ver:4.1.0@sha256:55a0ab8ebda191d556501f7ae121fc93ef809c43d3a3f11d10ef  0.0s
 => [internal] load build context                                                                        0.1s
 => => transferring context: 92.87kB                                                                     0.1s
 => CACHED [2/9] RUN apt-get update && apt-get install -y default-jre-headless gdal-bin git-core imagem  0.0s
 => CACHED [3/9] RUN echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method   0.0s
 => CACHED [4/9] RUN R -e 'install.packages("remotes")'                                                  0.0s
 => CACHED [5/9] RUN R -e 'install.packages("devtools")'                                                 0.0s
 => CACHED [6/9] RUN R -e 'remotes::install_github("r-lib/remotes", ref = "97bbf81")'                    0.0s
 => CACHED [7/9] RUN mkdir /build_zone                                                                   0.0s
 => [8/9] ADD . /build_zone                                                                              0.2s
 => [9/9] WORKDIR /build_zone                                                                            0.0s
 => exporting to image                                                                                   7.8s
 => => exporting layers                                                                                  7.8s
 => => writing image sha256:9fcdad76e5532bb51a0a3690e70fa892ed7fdac68021340309276fe2f22d477a             0.0s
 => => naming to docker.io/library/emdash                                                                0.0s

Trying to install within the container instead of image build fails while installing crosstalk

 docker run -it -p 1234:80 -v $PWD:/src --name emdash emdash /bin/bash
root@b7efc26cf072:/build_zone# cd /src/
CODE_OF_CONDUCT.md            emdash.Rproj                  _pkgdown.yml
DESCRIPTION                   .git/                         ports.txt
dev/                          .github/                      R/
docker_build_and_run.sh       .gitignore                    .Rbuildignore
Dockerfile                    inst/                         README.md
Dockerfile.orig               LICENSE                       README.Rmd
dockerfile.patch              LICENSE.md                    test-data/
.Dockerfile.swp               man/                          tests/
docker_run.sh                 NAMESPACE
docker_start_test_mongodb.sh  NEWS.md
root@b7efc26cf072:/build_zone# R -e 'remotes::install_local(upgrade="never")'

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> remotes::install_local(upgrade="never")
Installing 105 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, data.table, datamods, DBI, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, jsonlite, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, readr, readxl, rematch, rio, rmarkdown, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, webshot, wk, xtable
Installing packages into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz'
Content type 'application/x-gzip' length 79261 bytes (77 KB)
==================================================
downloaded 77 KB

trying URL 'https://cran.rstudio.com/src/contrib/backports_1.2.1.tar.gz'
Content type 'application/x-gzip' length 21184 bytes (20 KB)
==================================================
downloaded 20 KB

trying URL 'https://cran.rstudio.com/src/contrib/base64enc_0.1-3.tar.gz'
Content type 'application/x-gzip' length 7833 bytes
==================================================
downloaded 7833 bytes

trying URL 'https://cran.rstudio.com/src/contrib/bit_4.0.4.tar.gz'
Content type 'application/x-gzip' length 279723 bytes (273 KB)
==================================================
downloaded 273 KB

trying URL 'https://cran.rstudio.com/src/contrib/bit64_4.0.5.tar.gz'
Content type 'application/x-gzip' length 135091 bytes (131 KB)
==================================================
downloaded 131 KB

trying URL 'https://cran.rstudio.com/src/contrib/bslib_0.2.5.1.tar.gz'
Content type 'application/x-gzip' length 3810412 bytes (3.6 MB)
==================================================
downloaded 3.6 MB

trying URL 'https://cran.rstudio.com/src/contrib/cellranger_1.1.0.tar.gz'
Content type 'application/x-gzip' length 63857 bytes (62 KB)
==================================================
downloaded 62 KB

trying URL 'https://cran.rstudio.com/src/contrib/checkmate_2.0.0.tar.gz'
Content type 'application/x-gzip' length 168716 bytes (164 KB)
==================================================
downloaded 164 KB

trying URL 'https://cran.rstudio.com/src/contrib/classInt_0.4-3.tar.gz'
Content type 'application/x-gzip' length 403884 bytes (394 KB)
==================================================
downloaded 394 KB

trying URL 'https://cran.rstudio.com/src/contrib/colorspace_2.0-2.tar.gz'
Content type 'application/x-gzip' length 2196563 bytes (2.1 MB)
==================================================
downloaded 2.1 MB

trying URL 'https://cran.rstudio.com/src/contrib/config_0.3.1.tar.gz'
Content type 'application/x-gzip' length 69056 bytes (67 KB)
==================================================
downloaded 67 KB

trying URL 'https://cran.rstudio.com/src/contrib/cpp11_0.3.1.tar.gz'
Content type 'application/x-gzip' length 217952 bytes (212 KB)
==================================================
downloaded 212 KB

trying URL 'https://cran.rstudio.com/src/contrib/credentials_1.3.1.tar.gz'
Content type 'application/x-gzip' length 230470 bytes (225 KB)
==================================================
downloaded 225 KB

trying URL 'https://cran.rstudio.com/src/contrib/crosstalk_1.1.1.tar.gz'
Content type 'application/x-gzip' length 676543 bytes (660 KB)
==

downloaded 32 KB

Error in download.file(url, destfile, method, mode = "wb", ...) :
  (converted from warning) downloaded length 32768 != reported length 676543
Error in download.packages(pkgs, destdir = tmpd, available = available,  :
  (converted from warning) download of package ‘crosstalk’ failed
Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
Execution halted

And in attempt

root@b7efc26cf072:/build_zone# echo "options(repos = c(CRAN = 'https://cran.rstudio.com/'), download.file.method = 'libcurl', timeout = 300)" >> /usr/local/lib/R/etc/Rprofile.site
root@b7efc26cf072:/build_zone# R -e 'remotes::install_local(upgrade="never")'

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> remotes::install_local(upgrade="never")
Installing 105 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, data.table, datamods, DBI, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, jsonlite, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, readr, readxl, rematch, rio, rmarkdown, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, webshot, wk, xtable
Installing packages into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz'
Content type 'application/x-gzip' length 79261 bytes (77 KB)
===============
downloaded 24 KB

Error in download.file(url, destfile, method, mode = "wb", ...) :
  (converted from warning) downloaded length 25333 != reported length 79261
Error in download.packages(pkgs, destdir = tmpd, available = available,  :
  (converted from warning) download of package ‘attempt’ failed
Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
Execution halted

And, even after switching to the mounted drive, and increasing the timeout, fails in contrib

root@b7efc26cf072:/build_zone# cd /src/
root@b7efc26cf072:/src# R -e 'remotes::install_local(upgrade="never")'

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> remotes::install_local(upgrade="never")
Installing 105 packages: attempt, backports, base64enc, bit, bit64, bslib, cellranger, checkmate, classInt, colorspace, config, cpp11, credentials, crosstalk, data.table, datamods, DBI, dockerfiler, dplyr, DT, e1071, esquisse, farver, flexdashboard, forcats, generics, geojsonsf, geometries, gert, ggplot2, gh, golem, gridExtra, gtable, haven, here, hms, htmltools, htmlwidgets, httpuv, httr, isoband, janitor, jquerylib, jsonify, jsonlite, labeling, later, lazyeval, leafem, leaflet, leaflet.providers, leafpop, lubridate, mapview, mongolite, munsell, openxlsx, plotly, plyr, png, progress, promises, proxy, randomcoloR, rapidjsonr, raster, RColorBrewer, readr, readxl, rematch, rio, rmarkdown, Rtsne, s2, sass, satellite, scales, servr, sf, sfheaders, shiny, shinycssloaders, shinydashboard, shinyWidgets, snakecase, sourcetools, sp, svglite, systemfonts, testthat, tidyr, tidyselect, tinytex, tzdb, units, usethis, uuid, V8, viridis, viridisLite, vroom, webshot, wk, xtable
Installing packages into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz'
Content type 'application/x-gzip' length 79261 bytes (77 KB)
==================================================
downloaded 77 KB

trying URL 'https://cran.rstudio.com/src/contrib/backports_1.2.1.tar.gz'
Content type 'application/x-gzip' length 21184 bytes (20 KB)
==================================================
downloaded 20 KB
...
==================================================
downloaded 212 KB

trying URL 'https://cran.rstudio.com/src/contrib/credentials_1.3.1.tar.gz'
Content type 'application/x-gzip' length 230470 bytes (225 KB)
===
downloaded 15 KB

Error in download.file(url, destfile, method, mode = "wb", ...) :
  (converted from warning) downloaded length 15999 != reported length 230470
Error in download.packages(pkgs, destdir = tmpd, available = available,  :
  (converted from warning) download of package ‘credentials’ failed
Calls: <Anonymous> ... with_envvar -> force -> force -> i.p -> download.packages
Execution halted

And in every one of the failures, we were partway through downloading a file, the download failed, and then we got the error about the downloaded length

trying URL 'https://cran.rstudio.com/src/contrib/crosstalk_1.1.1.tar.gz'
Content type 'application/x-gzip' length 676543 bytes (660 KB)
==

downloaded 32 KB

trying URL 'https://cran.rstudio.com/src/contrib/attempt_0.3.1.tar.gz'
Content type 'application/x-gzip' length 79261 bytes (77 KB)
===============
downloaded 24 KB

trying URL 'https://cran.rstudio.com/src/contrib/credentials_1.3.1.tar.gz'
Content type 'application/x-gzip' length 230470 bytes (225 KB)
===
downloaded 15 KB

So it looks like the problem is really with the network downloads failing randomly.
Downloading dependencies as part of remotes::install_local(upgrade="never") seems to be fragile, when we were downloading dependencies using remotes::install_version, it was a lot more stable.

That is the main difference between:

72340fb#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557 (works at least with some install in the container) and
db3a2c1#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557 (fails all the time, in different places)

shankari · 2021-08-06T01:25:06Z

Trying to workaround by creating the docker container with dependencies from 72340fb and then switching to master and running
R -e 'remotes::install_local(upgrade="never")'

error

Mapping without trajectories
Warning: Error in st_geometry.sf: attr(obj, "sf_column") does not point to a geometry column.
Did you rename it, without setting st_geometry(obj) <- "newname"?
  55: stop
  54: st_geometry.sf
  52: [.sf
  50: dropListColumns
  49: create_filters
  48: <observer>
   5: runApp
   3: print.shiny.appobj
   1: emdash::run_app
disconnecting from the emission collections..

giving up at this point since I am not sure whether or not this is related to docker.

shankari · 2021-08-06T03:06:27Z

Final try:

Created an instance from the published image

docker run -it -p 1234:80 -v $PWD:/src --name emdash amarins/emdash /bin/bash
Unable to find image 'amarins/emdash:latest' locally
latest: Pulling from amarins/emdash
a31c7b29f4ad: Already exists
8ed946bf07db: Already exists
80b12a566f68: Already exists
7efaf74e9032: Pull complete
ae6bfc8e2aa9: Pull complete
e774a89efc36: Pull complete
a8634475f08b: Pull complete
bfd17ff19f89: Pull complete
7062d71fc235: Pull complete
1fb9a5c6b03e: Pull complete
4f4fb700ef54: Pull complete
f9f587febd40: Pull complete
Digest: sha256:7cc00d5478c4e312a5b553992e5c9366e9c118451aa87ba86561aacfc67f824e
Status: Downloaded newer image for amarins/emdash:latest

Tried to run from the mounted directory, it failed with the same error (`"sf_column") does not point to a geometry column.`)

root@4b4af9e1ff2c:/build_zone# cd /src/
> remotes::install_local(upgrade="never")
Skipping install of 'emdash' from a local remote, the SHA1 (1.4.1.90) has not changed since last install.
  Use `force = TRUE` to force installation
>
>
root@4b4af9e1ff2c:/src# R -e 'remotes::install_local(upgrade="never", force=TRUE)'

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> remotes::install_local(upgrade="never", force=TRUE)
✔  checking for file ‘/tmp/Rtmp0FOmhQ/file2356edbae4/src/DESCRIPTION’ ...
─  preparing ‘emdash’:
✔  checking DESCRIPTION meta-information ...
─  checking for LF line-endings in source and make files and shell scripts
─  checking for empty or unneeded directories
   Omitted ‘LazyData’ from DESCRIPTION
─  building ‘emdash_1.4.1.9001.tar.gz’

Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘emdash’ ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (emdash)
>
>
root@4b4af9e1ff2c:/src# R -e "options('shiny.port'=80,shiny.host='0.0.0.0');emdash::run_app(mongo_url = '$MONGO_URL', anon_locations=$ANON_LOCATIONS)"

R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> options('shiny.port'=80,shiny.host='0.0.0.0');emdash::run_app(mongo_url = 'mongodb://host.docker.internal:27017', anon_locations=FALSE)
No config file given, the default config file will be used.
Loading required package: shiny

Listening on http://0.0.0.0:80
First trip: 2020-09-22
Last trip:  2021-08-01
Warning: 'esquisserUI' is deprecated.
Use 'esquisse_ui' instead.
See help("Deprecated") and help("esquisse-deprecated").
Warning: 'filterDF_UI / filterDF' is deprecated.
Use 'datamods::filter_data_ui / datamods::filter_data_server' instead.
See help("Deprecated") and help("esquisse-deprecated").
Running: mod_load_data_server
Running: mod_load_trips_server
Warning: 'esquisserServer' is deprecated.
Use 'esquisse_server' instead.
See help("Deprecated") and help("esquisse-deprecated").
About to load participants
merging trip summaries with participants
merging server call summaries with participants
Finished merging server calls
Finished loading participants
Participants size is: 17.2 kB kb
About to load Bike Check in
Finished loading Bike Check in
About to load Polar Bear
Finished loading Polar Bear
The dates reactive values are: 2021-07-26 to 2021-08-02
Window_width is 7 days
get_n_trips_in_query: The time stamps for that date range are:
1627257600, 1627862400
There are 17 trips and 905 locations in the date range
Load_trips observed
About to load trips
Finished query, about to clean trips
Finished cleaning trips
Finished loading trips
Trips size is: 54 kB
Warning in grSoftVersion() :
  unable to load shared object '/usr/local/lib/R/modules//R_X11.so':
  libXt.so.6: cannot open shared object file: No such file or directory
Warning: Removed 1 rows containing non-finite values (stat_count).
Mapping without trajectories
Warning: Error in st_geometry.sf: attr(obj, "sf_column") does not point to a geometry column.
Did you rename it, without setting st_geometry(obj) <- "newname"?
  54: stop
  53: st_geometry.sf
  51: [.sf
  49: dropListColumns
  48: create_filters
  47: <observer>
   4: runApp
   3: print.shiny.appobj
   1: emdash::run_app
disconnecting from the emission collections..

shankari · 2021-08-06T03:25:59Z

Final final try (run the image directly), @asiripanich this still fails with the same error:

$ docker run -d -p 1234:80 -v $PWD:/src --name emdash amarins/emdash
4941cac11b45fd02e8fae24313402f427ad97f56d91134645bafe911de300875

$ docker logs -f emdash
R version 4.1.0 (2021-05-18) -- "Camp Pontanezen"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

> options('shiny.port'=80,shiny.host='0.0.0.0');emdash::run_app(mongo_url = 'mongodb://host.docker.internal:27017', anon_locations=FALSE)
No config file given, the default config file will be used.
Loading required package: shiny

Listening on http://0.0.0.0:80
First trip: 2020-09-22
Last trip:  2021-08-01
Warning: 'esquisserUI' is deprecated.
Use 'esquisse_ui' instead.
See help("Deprecated") and help("esquisse-deprecated").
Warning: 'filterDF_UI / filterDF' is deprecated.
Use 'datamods::filter_data_ui / datamods::filter_data_server' instead.
See help("Deprecated") and help("esquisse-deprecated").
Running: mod_load_data_server
Running: mod_load_trips_server
Warning: 'esquisserServer' is deprecated.
Use 'esquisse_server' instead.
See help("Deprecated") and help("esquisse-deprecated").
About to load participants
merging trip summaries with participants
merging server call summaries with participants
Finished merging server calls
Finished loading participants
Participants size is: 17.2 kB kb
About to load Bike Check in
Finished loading Bike Check in
About to load Polar Bear
Finished loading Polar Bear
The dates reactive values are: 2021-07-26 to 2021-08-02
Window_width is 7 days
get_n_trips_in_query: The time stamps for that date range are:
1627257600, 1627862400
There are 17 trips and 905 locations in the date range
Load_trips observed
About to load trips
Finished query, about to clean trips
Finished cleaning trips
Finished loading trips
Trips size is: 54 kB
Warning in grSoftVersion() :
  unable to load shared object '/usr/local/lib/R/modules//R_X11.so':
  libXt.so.6: cannot open shared object file: No such file or directory
Warning: Removed 1 rows containing non-finite values (stat_count).
Mapping without trajectories
Warning: Error in st_geometry.sf: attr(obj, "sf_column") does not point to a geometry column.
Did you rename it, without setting st_geometry(obj) <- "newname"?
  54: stop
  53: st_geometry.sf
  51: [.sf
  49: dropListColumns
  48: create_filters
  47: <observer>
   4: runApp
   3: print.shiny.appobj
   1: emdash::run_app
disconnecting from the emission collections..

asiripanich · 2021-08-06T09:24:30Z

Bad news.. I no longer can build images on Docker Hub. Their free tier benefits have changed. Only paying customers are allowed to use their CI.

shankari · 2021-08-06T15:18:43Z

@asiripanich we can use github packages instead.
https://docs.github.com/en/actions/guides/publishing-docker-images

But I don't think that the most recent error is related to building images. I can use the published image since there have not been any docker changes since then, mount my local directory, rebuild and re-run. But I'm still getting the error above, and the only issues related to it that we can find seem to be about importing sf correctly.
afrimapr/afrihealthsites#1

shankari · 2021-08-06T16:42:20Z

Fortunately, we got the bright idea of trying with the docker image on @allenmichael099's laptop, and we have now narrowed this down to a data issue.

if I use the staging data, it works
if I use the the NREL location history data, it works
if I use the the minipilot data, it does not work

I'm going to try to dump my database so we can see if we can reproduce this on Michael's laptop and figure out the data issue.

This is no longer a docker build issue, so tracking this in #68

asiripanich closed this as completed Sep 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker build fails #63

Docker build fails #63

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

allenmichael099 commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading

asiripanich commented Aug 6, 2021

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021

Docker build fails #63

Docker build fails #63

Comments

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

allenmichael099 commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021

shankari commented Aug 5, 2021 • edited Loading

shankari commented Aug 6, 2021 • edited Loading

shankari commented Aug 6, 2021

shankari commented Aug 6, 2021 • edited Loading

shankari commented Aug 6, 2021 • edited Loading

shankari commented Aug 6, 2021 • edited Loading

asiripanich commented Aug 6, 2021

shankari commented Aug 6, 2021 • edited Loading

shankari commented Aug 6, 2021

shankari commented Aug 5, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading

shankari commented Aug 6, 2021 •

edited

Loading