-
Notifications
You must be signed in to change notification settings - Fork 0
/
README.Rmd
140 lines (114 loc) · 5.52 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# DBCA Turtle Data Viewer
<!-- badges: start -->
[](https://lifecycle.r-lib.org/articles/stages.html#experimental)
[](https://github.com/dbca-wa/turtleviewer2/actions)
[](https://github.com/dbca-wa/turtleviewer2/actions)
<!-- badges: end -->
Interactive turtle data preview for QA and analysis hosted at
<https://turtledata.dbca.wa.gov.au/> for DBCA staff.
## Quickstart
* Open <https://turtledata.dbca.wa.gov.au/>, wait for data to load.
* Select an area of interest.
You can bookmark your selection by appending `?site=Site Name`, e.g.
* Delambre <https://turtledata.dbca.wa.gov.au/?site=Delambre%20Island>
* Thevenard <https://turtledata.dbca.wa.gov.au/?site=Thevenard%20Island>
* Maps have a full-screen button and can toggle layers on and off. Individual
markers are clustered for performance - click clusters or zoom in to expand them.
Individual markers have popups with links to the original record on WAStD.
* (In development) A download button will generate a ZIP archive of all data
currently shown. This replaces data dissemination via SharePoint.
If you need fresh data, come and get it here.
## Architecture
This section is for developers.
This Shiny app is based on [bs4Dash](https://rinterface.github.io/bs4Dash/index.html)
and was initially created from a [Golem](https://thinkr-open.github.io/golem/) template.
The main mechanism is to read snapshots of two fairly complex production databases
and visualise the data through maps, figures, and summaries.
The user can select areas of interest (mostly turtle monitoring program areas)
and download the data for that area.
Data snapshots are locally saved as an `.rds` file for WAStD data, WAMTRAM data,
and WAStD sites.
The app accesses these snapshots through file paths relative to its working directory, e.g. `here:here("inst/wastd_site.rds")`.
In development, the workdir is the top level folder of the app's source code,
and `inst` refers to that folder inside the source code.
In production, the `WORKDIR` is defined as `/app` in the Dockerfile, and
data snapshots are written to `/app/inst` in the running container.
To persist the data snapshots, a volume should be mounted to the internal path
`/app/inst`.
A second Docker image (dbca-wa/etlturtlenesting) runs the ETL jobs to
periodically refresh the data snapshots from the production databases.
The ETL process takes close to an hour, the saved snapshots read in seconds.
The ETL includes import of all digitally captured data from ODK Central into WAStD
prior to the data export from WAStD, so that all digitally captured data is
visible in this dashboard within two hours of upload.
## Development
* Write code, add tests where needed.
* Update dependencies from code: `attachment::att_amend_desc()`
(careful: drops unused Imports).
* Update Dockerfile from dependencies: initially generated through
`golem::add_dockerfile(from="rocker/geospatial:4.1.2")`.
Note: rotate existing Dockerfile to Dockerfile_old to keep manual edits.
* Build Docker image for local testing (see below).
* Run e.g. through [Portainer](https://www.portainer.io) with
* Name: e.g. "tv"
* Image: advanced mode, `dbca-wa/turtleviewer2:latest`
* Always pull image: off
* Publish ports: on
* Volumes: container path `/app/inst`, use named volume or bind mount to a local
directory
* env: paste environment variables from your .env (warning: sensitive credentials).
A template is given at `inst/.env.template`.
* If the Docker image does not work, inspect the docker logs,
or debug through the Docker shell.
The console runs inside the Docker container, bypassing the Shiny app.
There, you can run R and test whether all libraries (especially `turtleviewer2`)
load successfully.
* If the app as well as the Docker image work, commit and push.
Run the app locally:
```{r, eval=FALSE, warning=FALSE}
golem::detach_all_attached()
golem::document_and_reload()
run_app()
```
Build Docker image for local testing:
```{bash, eval=FALSE}
docker build . -t dbca-wa/turtleviewer2:latest
```
Inspect the locally built image for debugging:
```{bash, eval=FALSE}
docker run -it dbca-wa/turtleviewer2:latest /bin/bash -c "export TERM=xterm; exec bash"
```
## Deployment
Once app and Docker image work, create a new version, tag, and push the tag.
```{r, eval=FALSE}
styler::style_pkg()
spelling::spell_check_package()
spelling::update_wordlist()
# Code and docs tested, working, committed
usethis::use_version(which="patch")
usethis::use_version(which="minor")
usethis::use_version(which="major")
usethis::edit_file("NEWS.md")
# Document to load new package version. Git commit, tag, and push.
devtools::document()
v <- packageVersion("turtleviewer2")
system(glue::glue("git tag -a v{v} -m 'v{v}'"))
system(glue::glue("git push && git push --tags"))
```
Pushing a new tag will trigger a GitHub Action to build a new
Docker image and publish it to the GitHub container registry.
In the Rancher console to the BCS Kubernetes cluster,
edit the "turtleviewer2" workload and update the image to the latest tag.
This action could be automated through the k3s kubectl shell.