forked from caviddhen/mymadrat
-
Notifications
You must be signed in to change notification settings - Fork 0
/
madrat_exercise.html
258 lines (194 loc) · 19.6 KB
/
madrat_exercise.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="pandoc" />
<meta http-equiv="X-UA-Compatible" content="IE=EDGE" />
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="author" content="David Chen" />
<meta name="date" content="2019-09-10" />
<title>MADRaT Exercise</title>
<style type="text/css">code{white-space: pre;}</style>
<style type="text/css" data-origin="pandoc">
div.sourceCode { overflow-x: auto; }
table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode {
margin: 0; padding: 0; vertical-align: baseline; border: none; }
table.sourceCode { width: 100%; line-height: 100%; }
td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; }
td.sourceCode { padding-left: 5px; }
code > span.kw { color: #007020; font-weight: bold; } /* Keyword */
code > span.dt { color: #902000; } /* DataType */
code > span.dv { color: #40a070; } /* DecVal */
code > span.bn { color: #40a070; } /* BaseN */
code > span.fl { color: #40a070; } /* Float */
code > span.ch { color: #4070a0; } /* Char */
code > span.st { color: #4070a0; } /* String */
code > span.co { color: #60a0b0; font-style: italic; } /* Comment */
code > span.ot { color: #007020; } /* Other */
code > span.al { color: #ff0000; font-weight: bold; } /* Alert */
code > span.fu { color: #06287e; } /* Function */
code > span.er { color: #ff0000; font-weight: bold; } /* Error */
code > span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
code > span.cn { color: #880000; } /* Constant */
code > span.sc { color: #4070a0; } /* SpecialChar */
code > span.vs { color: #4070a0; } /* VerbatimString */
code > span.ss { color: #bb6688; } /* SpecialString */
code > span.im { } /* Import */
code > span.va { color: #19177c; } /* Variable */
code > span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code > span.op { color: #666666; } /* Operator */
code > span.bu { } /* BuiltIn */
code > span.ex { } /* Extension */
code > span.pp { color: #bc7a00; } /* Preprocessor */
code > span.at { color: #7d9029; } /* Attribute */
code > span.do { color: #ba2121; font-style: italic; } /* Documentation */
code > span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code > span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code > span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
</style>
<script>
// apply pandoc div.sourceCode style to pre.sourceCode instead
(function() {
var sheets = document.styleSheets;
for (var i = 0; i < sheets.length; i++) {
if (sheets[i].ownerNode.dataset["origin"] !== "pandoc") continue;
try { var rules = sheets[i].cssRules; } catch (e) { continue; }
for (var j = 0; j < rules.length; j++) {
var rule = rules[j];
// check if there is a div.sourceCode rule
if (rule.type !== rule.STYLE_RULE || rule.selectorText !== "div.sourceCode") continue;
var style = rule.style.cssText;
// check if color or background-color is set
if (rule.style.color === '' && rule.style.backgroundColor === '') continue;
// replace div.sourceCode by a pre.sourceCode rule
sheets[i].deleteRule(j);
sheets[i].insertRule('pre.sourceCode{' + style + '}', j);
}
}
})();
</script>
<link href="data:text/css;charset=utf-8,body%20%7B%0Abackground%2Dcolor%3A%20%23fff%3B%0Amargin%3A%201em%20auto%3B%0Amax%2Dwidth%3A%20700px%3B%0Aoverflow%3A%20visible%3B%0Apadding%2Dleft%3A%202em%3B%0Apadding%2Dright%3A%202em%3B%0Afont%2Dfamily%3A%20%22Open%20Sans%22%2C%20%22Helvetica%20Neue%22%2C%20Helvetica%2C%20Arial%2C%20sans%2Dserif%3B%0Afont%2Dsize%3A%2014px%3B%0Aline%2Dheight%3A%201%2E35%3B%0A%7D%0A%23header%20%7B%0Atext%2Dalign%3A%20center%3B%0A%7D%0A%23TOC%20%7B%0Aclear%3A%20both%3B%0Amargin%3A%200%200%2010px%2010px%3B%0Apadding%3A%204px%3B%0Awidth%3A%20400px%3B%0Aborder%3A%201px%20solid%20%23CCCCCC%3B%0Aborder%2Dradius%3A%205px%3B%0Abackground%2Dcolor%3A%20%23f6f6f6%3B%0Afont%2Dsize%3A%2013px%3B%0Aline%2Dheight%3A%201%2E3%3B%0A%7D%0A%23TOC%20%2Etoctitle%20%7B%0Afont%2Dweight%3A%20bold%3B%0Afont%2Dsize%3A%2015px%3B%0Amargin%2Dleft%3A%205px%3B%0A%7D%0A%23TOC%20ul%20%7B%0Apadding%2Dleft%3A%2040px%3B%0Amargin%2Dleft%3A%20%2D1%2E5em%3B%0Amargin%2Dtop%3A%205px%3B%0Amargin%2Dbottom%3A%205px%3B%0A%7D%0A%23TOC%20ul%20ul%20%7B%0Amargin%2Dleft%3A%20%2D2em%3B%0A%7D%0A%23TOC%20li%20%7B%0Aline%2Dheight%3A%2016px%3B%0A%7D%0Atable%20%7B%0Amargin%3A%201em%20auto%3B%0Aborder%2Dwidth%3A%201px%3B%0Aborder%2Dcolor%3A%20%23DDDDDD%3B%0Aborder%2Dstyle%3A%20outset%3B%0Aborder%2Dcollapse%3A%20collapse%3B%0A%7D%0Atable%20th%20%7B%0Aborder%2Dwidth%3A%202px%3B%0Apadding%3A%205px%3B%0Aborder%2Dstyle%3A%20inset%3B%0A%7D%0Atable%20td%20%7B%0Aborder%2Dwidth%3A%201px%3B%0Aborder%2Dstyle%3A%20inset%3B%0Aline%2Dheight%3A%2018px%3B%0Apadding%3A%205px%205px%3B%0A%7D%0Atable%2C%20table%20th%2C%20table%20td%20%7B%0Aborder%2Dleft%2Dstyle%3A%20none%3B%0Aborder%2Dright%2Dstyle%3A%20none%3B%0A%7D%0Atable%20thead%2C%20table%20tr%2Eeven%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0A%7D%0Ap%20%7B%0Amargin%3A%200%2E5em%200%3B%0A%7D%0Ablockquote%20%7B%0Abackground%2Dcolor%3A%20%23f6f6f6%3B%0Apadding%3A%200%2E25em%200%2E75em%3B%0A%7D%0Ahr%20%7B%0Aborder%2Dstyle%3A%20solid%3B%0Aborder%3A%20none%3B%0Aborder%2Dtop%3A%201px%20solid%20%23777%3B%0Amargin%3A%2028px%200%3B%0A%7D%0Adl%20%7B%0Amargin%2Dleft%3A%200%3B%0A%7D%0Adl%20dd%20%7B%0Amargin%2Dbottom%3A%2013px%3B%0Amargin%2Dleft%3A%2013px%3B%0A%7D%0Adl%20dt%20%7B%0Afont%2Dweight%3A%20bold%3B%0A%7D%0Aul%20%7B%0Amargin%2Dtop%3A%200%3B%0A%7D%0Aul%20li%20%7B%0Alist%2Dstyle%3A%20circle%20outside%3B%0A%7D%0Aul%20ul%20%7B%0Amargin%2Dbottom%3A%200%3B%0A%7D%0Apre%2C%20code%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0Aborder%2Dradius%3A%203px%3B%0Acolor%3A%20%23333%3B%0Awhite%2Dspace%3A%20pre%2Dwrap%3B%20%0A%7D%0Apre%20%7B%0Aborder%2Dradius%3A%203px%3B%0Amargin%3A%205px%200px%2010px%200px%3B%0Apadding%3A%2010px%3B%0A%7D%0Apre%3Anot%28%5Bclass%5D%29%20%7B%0Abackground%2Dcolor%3A%20%23f7f7f7%3B%0A%7D%0Acode%20%7B%0Afont%2Dfamily%3A%20Consolas%2C%20Monaco%2C%20%27Courier%20New%27%2C%20monospace%3B%0Afont%2Dsize%3A%2085%25%3B%0A%7D%0Ap%20%3E%20code%2C%20li%20%3E%20code%20%7B%0Apadding%3A%202px%200px%3B%0A%7D%0Adiv%2Efigure%20%7B%0Atext%2Dalign%3A%20center%3B%0A%7D%0Aimg%20%7B%0Abackground%2Dcolor%3A%20%23FFFFFF%3B%0Apadding%3A%202px%3B%0Aborder%3A%201px%20solid%20%23DDDDDD%3B%0Aborder%2Dradius%3A%203px%3B%0Aborder%3A%201px%20solid%20%23CCCCCC%3B%0Amargin%3A%200%205px%3B%0A%7D%0Ah1%20%7B%0Amargin%2Dtop%3A%200%3B%0Afont%2Dsize%3A%2035px%3B%0Aline%2Dheight%3A%2040px%3B%0A%7D%0Ah2%20%7B%0Aborder%2Dbottom%3A%204px%20solid%20%23f7f7f7%3B%0Apadding%2Dtop%3A%2010px%3B%0Apadding%2Dbottom%3A%202px%3B%0Afont%2Dsize%3A%20145%25%3B%0A%7D%0Ah3%20%7B%0Aborder%2Dbottom%3A%202px%20solid%20%23f7f7f7%3B%0Apadding%2Dtop%3A%2010px%3B%0Afont%2Dsize%3A%20120%25%3B%0A%7D%0Ah4%20%7B%0Aborder%2Dbottom%3A%201px%20solid%20%23f7f7f7%3B%0Amargin%2Dleft%3A%208px%3B%0Afont%2Dsize%3A%20105%25%3B%0A%7D%0Ah5%2C%20h6%20%7B%0Aborder%2Dbottom%3A%201px%20solid%20%23ccc%3B%0Afont%2Dsize%3A%20105%25%3B%0A%7D%0Aa%20%7B%0Acolor%3A%20%230033dd%3B%0Atext%2Ddecoration%3A%20none%3B%0A%7D%0Aa%3Ahover%20%7B%0Acolor%3A%20%236666ff%3B%20%7D%0Aa%3Avisited%20%7B%0Acolor%3A%20%23800080%3B%20%7D%0Aa%3Avisited%3Ahover%20%7B%0Acolor%3A%20%23BB00BB%3B%20%7D%0Aa%5Bhref%5E%3D%22http%3A%22%5D%20%7B%0Atext%2Ddecoration%3A%20underline%3B%20%7D%0Aa%5Bhref%5E%3D%22https%3A%22%5D%20%7B%0Atext%2Ddecoration%3A%20underline%3B%20%7D%0A%0Acode%20%3E%20span%2Ekw%20%7B%20color%3A%20%23555%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%0Acode%20%3E%20span%2Edt%20%7B%20color%3A%20%23902000%3B%20%7D%20%0Acode%20%3E%20span%2Edv%20%7B%20color%3A%20%2340a070%3B%20%7D%20%0Acode%20%3E%20span%2Ebn%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Efl%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Ech%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Est%20%7B%20color%3A%20%23d14%3B%20%7D%20%0Acode%20%3E%20span%2Eco%20%7B%20color%3A%20%23888888%3B%20font%2Dstyle%3A%20italic%3B%20%7D%20%0Acode%20%3E%20span%2Eot%20%7B%20color%3A%20%23007020%3B%20%7D%20%0Acode%20%3E%20span%2Eal%20%7B%20color%3A%20%23ff0000%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%0Acode%20%3E%20span%2Efu%20%7B%20color%3A%20%23900%3B%20font%2Dweight%3A%20bold%3B%20%7D%20%20code%20%3E%20span%2Eer%20%7B%20color%3A%20%23a61717%3B%20background%2Dcolor%3A%20%23e3d2d2%3B%20%7D%20%0A" rel="stylesheet" type="text/css" />
</head>
<body>
<h1 class="title toc-ignore">MADRaT Exercise</h1>
<h4 class="author">David Chen</h4>
<h4 class="date">2019-09-10</h4>
<div id="learning-objectives" class="section level3">
<h3>Learning Objectives</h3>
<p>In this exercise, we will practice reading in and transforming input data through MADRaT. We will create our own MADRat-based package for data input processing, and create the standar set of functions that allow for reading in standardized input data. For this example, we will use World Bank World Development Indicator (WDI) data, with the WDI library for easy direct download.</p>
</div>
<div id="new-mymadrat-package" class="section level2">
<h2>1. New “mymadrat” Package</h2>
<p>First, create a new package through the “New Project-> New Directory-> R Package” option in RStudio and call it “mymadrat”. Add the following script saved as madrat.R to the newly created R folder.</p>
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r">### madrat.R
<span class="co">#' @importFrom madrat vcat</span>
<span class="op">></span><span class="st"> </span>.onLoad <-<span class="st"> </span><span class="cf">function</span>(libname, pkgname){
<span class="op">></span><span class="st"> </span>madrat<span class="op">::</span><span class="kw">setConfig</span>(<span class="dt">packages=</span><span class="kw">c</span>(madrat<span class="op">::</span><span class="kw">getConfig</span>(<span class="st">"packages"</span>),pkgname),
<span class="dt">.cfgchecks=</span><span class="ot">FALSE</span>, <span class="dt">.verbose=</span><span class="ot">FALSE</span>)
<span class="op">></span><span class="st"> </span>}
<span class="co">#create an own warning function which redirects calls to vcat (package internal)</span>
<span class="op">></span><span class="st"> </span>warning <-<span class="st"> </span><span class="cf">function</span>(...) <span class="kw">vcat</span>(<span class="dv">0</span>,...)
<span class="co"># create a own stop function which redirects calls to stop (package internal)</span>
<span class="op">></span><span class="st"> </span>stop <-<span class="st"> </span><span class="cf">function</span>(...) <span class="kw">vcat</span>(<span class="op">-</span><span class="dv">1</span>,...)
<span class="co"># create an own cat function which redirects calls to cat (package internal)</span>
<span class="op">></span><span class="st"> </span>cat <-<span class="st"> </span><span class="cf">function</span>(...) <span class="kw">vcat</span>(<span class="dv">1</span>,...)</code></pre></div>
</div>
<div id="madrat-functions---exercise" class="section level2">
<h2>2. MADRaT Functions - Exercise</h2>
<p>We will create new download, read, and convert functions for WDI indicators Population, GDP, Employment rate, Employment share in Agriculture, and Agricultural GDP. Each indicator should be a separate parameter within each function. Note that each function needs to be named as a file and as a function with the wrapper, for example, readWDI.R</p>
<p>Remember that complete magclass objects are an array with the the regional (ISO3) dimension in the first dimension, the temporal dimension in the second, and data values in the third dimemsion(s) (3.1, 3.2…).</p>
<div id="download-function" class="section level3">
<h3>2.1 Download function</h3>
<p>Note that if direct download not possible, data files can be manually created in the inputdata/sources folder. In that case, a download function is not necessary, but naming of the source folder must match the read functions.</p>
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r"><span class="kw">install.packages</span>(<span class="st">"WDI"</span>)
<span class="kw">library</span>(WDI)
downloadWDI <-<span class="st"> </span><span class="cf">function</span>(...){
<span class="co"># Use the WDI() function to easily direct download WDI data;</span>
<span class="co">#with 1960 as the start year and 2018 as end; see ?WDI() for more help</span>
<span class="kw">install.packages</span>(<span class="st">"WDI"</span>)
<span class="kw">library</span>(WDI)
<span class="co"># Download these indicators, by creating a vector of indicators to put into WDI():</span>
<span class="co"># "NY.GDP.MKTP.KD" National GDP in 2010 USD</span>
<span class="co"># "SP.POP.TOTL" Total population</span>
<span class="co"># "SL.EMP.TOTL.SP.ZS" Employment ratio</span>
<span class="co"># "SL.AGR.EMPL.ZS" Employment in agriculture as % of total employment</span>
<span class="co"># "NV.AGR.TOTL.KD" Agricultural GDP in 2010 USD</span>
<span class="co">#Save the downloaded data as a .Rda file as the last step of the function</span>
<span class="kw">save</span>()
}</code></pre></div>
</div>
<div id="read-function" class="section level3">
<h3>2.2 Read function</h3>
<p>Read functions are the first step in transforming input data into magclass objects. They should be as simple as possible, with most steps of filling in, transforming, and data cleaning reserved for the convert function. The Read function should be able to specify between indicators (subtypes).</p>
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r">
readWDI <-<span class="st"> </span><span class="cf">function</span>(...){
<span class="co">#begin by loading the dataset saved by the previous download function.</span>
<span class="kw">load</span>(...)
<span class="co">#Use as.magpie() to transform the dataset into a magclass object. </span>
<span class="co">#Remember that magclass objects always have the spatial indicator (iso code, not country name) in their 1st dimension & the temporal dimension in 2nd dimenson.</span>
<span class="co">#It is ideal to reshape the data into 'tidy' format, with only the last column containing data as.magpie(...tidy=TRUE). </span>
<span class="co">#In this case, we will use the melt() function. </span>
<span class="kw">library</span>(reshape2)
wdi <-<span class="st"> </span><span class="kw">melt</span>(wdi, <span class="dt">id.vars=</span><span class="kw">c</span>(<span class="st">"iso2c"</span>, <span class="st">"year"</span>))
<span class="co">#Now create the magpie object, fill in the missing parameters for as.magpie:</span>
<span class="co">#Note the replacement parameter, which maintains the naming of the indicators</span>
wdi <-<span class="st"> </span><span class="kw">as.magpie</span>(wdi,..., <span class="dt">replacement=</span><span class="st">"."</span>)
<span class="co">#Once the magclass object is created, take a look at it. What does the function fulldim() tell you?</span>
<span class="co">#we can specify the subtype/indicator desired,</span>
<span class="co">#so that the read function only returns one indicator at a time.</span>
<span class="co">#Use [] to subset the subtype:</span>
x <-<span class="st"> </span>wdi[]
<span class="kw">return</span>(x)
}</code></pre></div>
</div>
<div id="convert-function" class="section level3">
<h3>2.3 Convert Function</h3>
<p>The convert function will complete the magclass object: All 249 countries represented in MAGPie need to have a value and be in ISO3 country code.</p>
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r">
convertWDI <-<span class="st"> </span><span class="cf">function</span>(...){
<span class="co">#Units need to be regularized in this function as well, for instance, MAgPIE uses population and GDP in millions:</span>
<span class="cf">if</span> (subtype <span class="op">%in%</span><span class="st"> "NY.GDP.MKTP.KD"</span>, <span class="st">"SP.POP.TOTL"</span>,<span class="st">"NV.AGR.TOTL.KD"</span> ) {
x <-<span class="st"> </span>x<span class="op">/</span><span class="fl">1e6</span>
}
<span class="co"># For these datasets, we need to convert the iso2c countrycodes into iso3c (this is also often done in the read function..)</span>
<span class="co"># We can use the function countrycode(), with the magclass call getRegions. </span>
<span class="co"># Note:</span>
<span class="co"># Other datasets may require other mapping conversions, there exist magclass specific tools: </span>
<span class="co"># toolCountry2isocode() and toolCell2isoCell() for instance</span>
<span class="kw">library</span>(countrycode)
<span class="kw">getRegions</span>(x)<-<span class="kw">countrycode</span>(<span class="kw">getRegions</span>(x),<span class="st">"iso2c"</span>,<span class="st">"iso3c"</span>)
<span class="co"># Note that some regions have now been turned into NA; you can check with getRegions()</span>
<span class="co"># It would be important to be certain that no information is lost. In this case the NA's are mostly WB aggregate regions. </span>
x<-x[<span class="op">!</span><span class="kw">is.na</span>(<span class="kw">getCells</span>(x)),,] <span class="co">#remove NA regions</span>
<span class="co">#clean_magpie() cleans MAgPIE objects so that they follow some extended magpie object rules (currently it makes sure that the dimnames have names and removes cell numbers if it is purely regional data)</span>
x <-<span class="st"> </span><span class="kw">clean_magpie</span>(x)
<span class="co">#Now fill the missing countries using toolCountryFill(), choose an arbitrary (Not NA!) fill value for now.</span>
x <-<span class="st"> </span><span class="kw">toolCountryFill</span>()
<span class="co">#Now we have a complete MAgPIE object! congratulations. </span>
<span class="kw">return</span>(x)}
</code></pre></div>
</div>
<div id="calcoutput" class="section level3">
<h3>3. calcOutput</h3>
<p>Magclass objects lend themselves easily to calculations between them. Use the read functions we just wrote to calculate agricultural GDP as a percentage of total gdp, using a new calcOutput function. What is Germany’s share of Agricultural GDP in 2010?</p>
<div class="sourceCode"><pre class="sourceCode r"><code class="sourceCode r">calcAgGDP <-<span class="st"> </span><span class="cf">function</span>(){
<span class="kw">readSource</span>(<span class="st">"WDI"</span>, <span class="dt">subtype=</span><span class="st">"NY.GDP.MKTP.KD"</span>)
<span class="co">#note that by default, readSource also converts; otherwise set convert=FALSE</span>
}</code></pre></div>
<p>After the function is written and built, call calc functions through the calcOutput(“type”) wrapper. Try it now with the calcAgGDP function.</p>
<p>All functions are saved in the inputdata/cache file after the first run, increasing efficiency. This means that for any updates to functions, the older cached function needs to be deleted. Cache is toggled with setConfig(forcecache=TRUE)</p>
</div>
</div>
<!-- code folding -->
<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
(function () {
var script = document.createElement("script");
script.type = "text/javascript";
script.src = "https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML";
document.getElementsByTagName("head")[0].appendChild(script);
})();
</script>
</body>
</html>