-
Notifications
You must be signed in to change notification settings - Fork 11
/
index.Rmd
130 lines (88 loc) · 7.14 KB
/
index.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
---
title: "TERRA REF Tutorials"
author: "David LeBauer and others"
site: bookdown::bookdown_site
output: bookdown::gitbook
date: "`r Sys.Date()`"
documentclass: book
---
# Overview
This book is intended to quickly introduce users to TERRA REF data through a series of tutorials. TERRA REF has many types of data, and most can be accessed in multiple ways. Although this makes it more complicated to learn (and teach!), the objective is to provide users with the flexibility to access data in the most useful way.
The first section walks the user through the steps of downloading and combining three different types of data: plot level phenotypes, meteorological data, and images. Subsequent sections provide more detailed examples that show how to access a larger variety of data and meta-data.
For those who are eager to see what is on deck, there are additional tutorials under development in the [tutorials repository](https://github.com/terraref/tutorials) on GitHub. These can be found in the traits, sensors, and genomics subfolders. Users are encouraged to revise and contribute new content.
## Pre-requisites
While we assume that readers will have some familiarity with the nature of the problem - remote sensing of crop plants - for the most part, these tutorials assume that the user will bring their own scientific questions and a sense of curiosity and are eager to learn.
These tutorials are aimed at users who are familiar with or willing to learn programming languages including R (particularly for accessing plot level trait data) and Python (primarily for accessing environmental data and sensor data). In addition, there are examples of using SQL for more sophisticated database queries, as well as the Bash terminal.
Some of the lessons only require a web browser; others will assume familiarity with programming at the command line in (typically only one of) Python, R, and / or SQL. You should be willing to find help (see finding help, below).
### Technical Requirements
At a minimum, you should have:
* An internet connection
* Web browser
* Access to the data that you are using
+ The tutorials will state which databases you will need access to
* Software:
+ Software requirements vary with the tutorials, and may be complex
### Installation instructions {#installation-instructions}
**Option 1** The recommended and easier option is to use our [VICE](https://learning.cyverse.org/projects/vice/en/latest/getting_started/about.html) app on [CyVerse](https://cyverse.org/)'s [Discovery Environment](https://de.cyverse.org/de/) platform. All necessary programs and libraries are installed and accessed using RStudio. You'll need a CyVerse account; follow instructions below to set up.
Launching TERRA REF app on VICE:
1. Click this button <a href="https://de.cyverse.org/de/?type=quick-launch&quick-launch-id=27203e0f-e2a1-4846-be0c-e97cb6ef60f2&app-id=041dd9c4-b964-11e9-a238-008cfa5ae621" target="_blank"><img src="https://de.cyverse.org/Powered-By-CyVerse-blue.svg"></a>
2. Click "Log in with your CyVerse ID" and log in using your CyVerse username and password
3. Hit "Launch Analysis" button in pop up window, and select arrow button in Analyses window
4. Wait patiently
5. Sign into RStudio using username: rstudio and password: rstudio1
Reopen running TERRA REF instance:
1. Open the [CyVerse Discovery Environment](https://de.cyverse.org/de/)
2. Log in
3. Open Analyses window and hit arrow button to open up RStudio
**Option 2** Everything necessary can be installed locally instead of using the VICE app.
You will need to [install R](https://www.r-project.org/) and a handful of R packages.
All of the packages except `traits` are available on CRAN and can be installed using the `install.packages` function:
```r
install.packages('tidyverse', 'jsonlite', 'knitr', 'lubridate', 'raster', 'sf')
```
Although `traits` is also on CRAN, you need to download the TERRA REF version from GitHub to handle larger datasets:
```r
install.packages('devtools')
devtools::install_github('terraref/traits')
```
You will additionally need to [install Python](https://www.python.org/downloads/) and the `terrautils` library, as below:
```sh
pip install terrautils
```
### User accounts and permission to access TERRA REF data {#user-accounts}
The first few chapters in the 'vignettes' section use publicly available sample data sets.
Subsequent sections are also written to use publicly available data sets, but some of the examples require data that requires users to sign up.
To sign up, you will need to 1) fill out the TERRA REF Beta user questionnaire ([terraref.org/beta](https://terraref.org/beta)) and 2) request access to specific databases.
<!-- Not sure where this goes, either in documentation or perhaps in an appendix. But I don't think this belongs in the introduction. Perhaps after the vignettes chapter
-->
## Ways of Accessing Data
### Public Domain Data (Seasons 4 and 6)
Data in the public domain can be found in two places.
1. About 1GB of data including all trait data, weather, experimental metadata, sensor documentation and lists of all of the sensor data are on Dryad at https://doi.org/10.5061/dryad.4b8gtht99. Learn more in the [README](https://datadryad.org/stash/downloads/file_stream/624638) on Dryad.
2. Sensor and Genomic data (approximately 1/2 PB) are available via Globus on the [`ncsa#terra-public`](https://app.globus.org/file-manager?origin_id=e8feaff4-96cd-11ea-bf90-0e6cccbb0103&origin_path=%2F) collection.
Citation: LeBauer, David et al. (2020), Data From: TERRA-REF, An open reference data set from high resolution genomics, phenomics, and imaging sensors, Dryad, Dataset, https://doi.org/10.5061/dryad.4b8gtht99
### Other data portals (public domain and beyond)
* Web Interfaces
+ [Clowder](https://docs.terraref.org/user-manual/how-to-access-data#clowder-sensor-data-and-metadata-browser) (sensor and genomic data)
+ [Globus](https://docs.terraref.org/user-manual/how-to-access-data#globus-browse-and-transfer-files) (sensor and genomic data)
+ [BETYdb](https://docs.terraref.org/user-manual/how-to-access-data#betydb-trait-data-and-agronomic-metadata) (trait data and experimental metadata)
+ [CoGe](https://docs.terraref.org/user-manual/how-to-access-data#coge-genomics-data) (genomics data)
* Files
* Programming APIs
+ [BETYdb API](https://pecan.gitbook.io/betydb-data-access/api-for-url-based-queries)
* API Clients
+ [rOpenSci traits package](https://pecan.gitbook.io/betydb-data-access/ropensci-traits-package)
## Other Resources
The TERRA REF website: [terraref.org](http://terraref.org/)
The TERRA REF Technical Documentation: [docs.terraref.org](https://docs.terraref.org)
### Finding help
- Slack at terra-ref.slack.com ([signup](https://terraref-slack-invite.herokuapp.com/)).
- Browse issues and repositories in GitHub:
- search the organization at github.com/terraref
- questions about the tutorials in the [tutorials repository](https://github.com/terraref/tutorials/issues)
- about the data in the [reference-data repository](https://github.com/terraref/reference-data/issues)
```{r, include = FALSE}
options(warn = -1)
knitr::opts_chunk$set(cache = TRUE, message = FALSE, eval = FALSE)
ggplot2::theme_set(ggthemes::theme_few())
```