-
Notifications
You must be signed in to change notification settings - Fork 0
/
intro_tidyverse.Rmd
executable file
·835 lines (634 loc) · 25.8 KB
/
intro_tidyverse.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
---
title: "MAT381E-Week 3: Data Importing, Manipulating, and Tidying"
subtitle: ""
author: "Gül İnan"
institute: "Department of Mathematics<br/>Istanbul Technical University"
date: "`r format(Sys.Date(), '%B %e, %Y')`"
output:
xaringan::moon_reader:
css: ["default", "xaringan-themer.css"]
self_contained: false # if true, fonts will be stored locally
nature:
beforeInit: ["assets/remark-zoom.js", "https://platform.twitter.com/widgets.js"]
titleSlideClass: ["left", "middle", "my-title"]
highlightStyle: github
highlightLines: true
countIncrementalSlides: false
ratio: '16:9' # alternatives '16:9' or '4:3' or others e.g. 13:9
navigation:
scroll: false # disable slide transitions by scrolling
---
```{r xaringan-themer, include=FALSE, warning=FALSE}
library(xaringanthemer)
style_mono_light(
base_color = "#042856",
header_color = "#7cacd4",
title_slide_text_color = "#7cacd4",
link_color = "#0000FF",
text_color = "#000000",
background_color = "#FFFFFF",
header_h1_font_size ="2.00rem"
)
```
```{r xaringan-scribble, echo=FALSE}
xaringanExtra::use_scribble() #activate for the pencil
xaringanExtra::use_xaringan_extra(c("tile_view", "animate_css", "tachyons"))
xaringanExtra::use_panelset() #panel set
```
```{r, echo=FALSE, purl=FALSE, message = FALSE}
knitr::opts_chunk$set(comment = "#>", purl = FALSE, fig.showtext = TRUE, retina = 2)
```
class: left
# Outline
* What is `Tidyverse Ecosystem`?
* `Pipe` operator.
* `Tibbles` alternative to `data frames`.
* Data importing via `readr` and `readxl` packages.
* 01-importing.Rmd
* Data manipulating with `dplyr` package.
* 02-manipulating.Rmd
* Data tidying with `tidyr` package.
* 03-tidying.Rmd
---
class: middle, center
```{r echo=FALSE, results='asis', out.width='90%', fig.cap=''}
knitr::include_graphics('images/time.png')
```
---
```{css echo=FALSE}
.pull-left {
float: left;
width: 50%;
}
.pull-right {
float: right;
width: 50%;
}
```
.pull-left[
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Most data scientists spend almost 80% of their time inspecting and cleaning data rather than working on their machine learning models.<br><br>But why?</p>— Ammar Yasser (@ammaryh92) <a href="https://twitter.com/ammaryh92/status/1449261043138584578?ref_src=twsrc%5Etfw">October 16, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
]
.pull-right[
- "According to the "No Free Lunch theorem", most machine learning models have a set of built-in assumptions, and before you start training your model, you have to make sure that your data is in line with the underlying assumptions of your model."
- "Data analysis allows you to:
1. Understand the patterns and trends in the data.
2. Understand the data distribution.
3. Make sure that your data is in line with the model's assumptions.
4. Assess the quality of the data (noise, wrong values, missing values, outliers, etc).
5. Assess the tidiness of the data (data structure, variable types, etc.)
6. Clean your data to get it in the best possible form for your model.
7. Feature engineering (extract features with predictive power from data)."
- "So make sure to spend some time analyzing and cleaning your model before moving on to your model. Always remember, "Garbage in, Garbage out".
]
---
class: center, middle
.pull-left[
```{r echo=FALSE, results='asis', out.width='90%', fig.cap=''}
knitr::include_graphics('images/dataclean1.png')
```
]
.pull-right[
```{r echo=FALSE, results='asis', out.width='90%', fig.cap=''}
knitr::include_graphics('images/dataclean2.png')
```
]
[Dezyre](https://www.dezyre.com/article/why-data-preparation-is-an-important-part-of-data-science/242)
---
class: center, middle
<!-- First code block is setting options for theme of the slides -->
<!-- Second code block is setting global options R code blocks -->
# Introduction to Tidyverse Ecosystem
![Tidyverse](logo/tidyverse.png)
---
# What is the Tidyverse Ecosystem?
- [Tidyverse](https://www.tidyverse.org/) is **a collection of packages** built as a suite of "data science" tools with a focus on **importing, manipulating**, and **visualizing data**.
```{r echo=FALSE, results='asis', out.width='80%', fig.show='hold', fig.cap=''}
knitr::include_graphics('images/data-science-workflow.png')
```
---
- There are about 30 packages in the `Tidyverse ecosystem`.
- The **core** `Tidyverse ecosystem` includes the following packages:
- `tibble`: alternative to data.frame class,
- `readr`: open and organize the data,
- `dplyr`: data manipulation,
- `tidyr`: modeling and data management,
- `ggplot2`: data visualization,
- `forcats`: functions for working with factors,
- `stringr`: functions for working with string data,
- `purrr`: code optimization and functional programming, and
- also, by default includes `magrittr` (source of the `pipe operator`).
.pull-right[
```{r echo=FALSE, results='asis', out.width='70%', fig.cap=''}
knitr::include_graphics('logo/logotogether.png')
```
]
---
- All these packages are designed to **work well together** and are based on:
- **tidy data** philosophy, where each variable has its own column, each observation has its own row,
- use **pipe operator** `%>%` to improve code development and readability, and
- follow a **common functional structure** for code clarity and reproducibility.
- Please visit [Tidyverse Style Guide](https://style.tidyverse.org/), especially the one on [Syntax](https://style.tidyverse.org/syntax.html).
---
# Install the Tidyverse Ecosystem
- We can install **everything at once** and access the `Tidyverse` as follows:
```{r eval=FALSE}
# Install "tidyverse" package on your local machine.
# You only need to do this once per machine.
install.packages("tidyverse", dependencies = TRUE)
# By default, install.packages() will download packages from
# https://cran.r-project.org, or one of its mirrors—so be sure you
# are connected to the internet when you run it.
```
- This will install the following packages:
```{r eval=FALSE}
[1] "broom" "cli" "crayon" "dbplyr" "dplyr"
[6] "dtplyr" "forcats" "googledrive" "googlesheets4" "ggplot2"
[11] "haven" "hms" "httr" "jsonlite" "lubridate"
[16] "magrittr" "modelr" "pillar" "purrr" "readr"
[21] "readxl" "reprex" "rlang" "rstudioapi" "rvest"
[26] "stringr" "tibble" "tidyr" "xml2" "tidyverse"
```
---
- Now we can load package `tidyverse` into our `R` session.
```{r eval=TRUE}
# You need to do this each time you start a new R session.
# You may need to update the packages (with update.packages() .
# function) from to time to time to get the latest improvements.
library(tidyverse)
```
---
# What about those conflicts?
- When `R` loads packages, functions loaded in earlier packages are **overwritten**
by functions of the **same name** from later packages.
- This means that **the order in which packages are loaded** matters.
- We can see that there are two functions in the `dplyr` package which *mask* base-`R` functions of the **same name**.
- To make sure that a function from the correct package is used, we can use the following construction:
```{r eval=FALSE}
package_name::function_name()
```
- where the double colon operator `::` is the **namespace operator**.
- So if we want to access the base functions instead of the `tidy` ones, we should **explicitly** refer to specific functions with **namespace operator** such as :
```{r eval=FALSE}
base::filter()
```
- As a general rule, it is better to load the `tidyverse` *after* all other packages; this will identify the conflicts.
---
# Example
```{r}
library(tidyverse)
library(MASS)
#implies that you cannot use select function in dplyr package
```
```{r, eval=FALSE}
# but if you want to use select function in dplyr package, # use this
dplyr::select()
# or you can also use this
library(tidyverse)
library(MASS, exclude = 'select')
```
---
💡 Also notice that `library(tidyverse)` does not load every package installed by `install.packages("tidyverse")`.
💡 We must use `library()` function to individually load the **non-core** `tidyverse` packages.
---
# Tidy Data
- **Tidy data** assumes that:
- Each variable forms a column,
- Each observation forms a row, and
- Each type of observational unit forms a table.
```{r echo=FALSE, results='asis', out.width='70%', fig.cap=''}
knitr::include_graphics('images/tidydata.png')
```
---
class: center, middle
```{r echo=FALSE, out.width='50%', fig.cap=''}
knitr::include_graphics('logo/pipe.jpeg')
```
---
# The pipe operator `%>%`
- The pipe operator `%>%` is a **special operator** introduced in the `magrittr` package:
- allows us to “pipe” several functions into **one long chain** that matches the order in which we want to do stuff and,
- in this sense, takes the object/function call result on the left and "passes" it to the right; it does *not* make assignment by itself.
```{r}
# in standard R
x <- 1:3
sum(x)
```
```{r}
# with the pipe, we can rewrite this as:
x <- 1:3
# with the pipe
x %>%
sum()
```
---
# Usage
- The pipe operator `%>%` should always have a space before it, and should usually be followed by a **new line**.
- After the first step, **each line** should be **indented by two spaces**.
- If the arguments to a function do **not all fit on one line**, put each argument **on its own** line and indent.
---
- Root mean squared error (RMSE) is defined:
$$
\text{RMSE} = \sqrt\frac{\sum_{i=1}^n(\hat{y}_i-y_i)^2}{n}
$$
where $\hat{y}_i$ denotes the prediction and $y_i$ the actually observed value.
```{r}
# in standard R
data <- c(1, 3, 4, 2, 5)
prediction <- c(1, 2, 2, 1, 4)
# calculate root mean squared error
rmse <- sqrt(mean((prediction-data)^2))
print(rmse)
```
---
- Using the pipe operator `%>%` makes the operation more intuitive:
```{r}
data <- c(1, 3, 4, 2, 5)
prediction <- c(1, 2, 2, 1, 4)
# calculate root mean squared error the pipe way
rmse <- (prediction-data)^2 %>%
mean() %>%
sqrt() %>%
print()
```
---
class: center, middle
```{r echo=FALSE, out.width='30%', fig.cap=''}
knitr::include_graphics('logo/tibble.jpeg')
```
---
# What is a Tibble?
- The `Tidyverse ecosystem` uses **tibbles** as an **alternative** to the `R`'s base `data.frame` class.
- [Tibbles](https://tibble.tidyverse.org/) are intended to represent the `tidy` data principles by design and provides opinionated data frames that make working in the `tidyverse` a little easier.
- `Tibbles` and `data frames` have many similar properties (rectangular data), but
`tibbles` returns more **readable output** in the console.
- We can create `tibbles` with `tibble()` function.
---
# Example
```{r}
# Create a tibble
# tibble() builds columns sequentially.
# When defining a column, you can refer to columns created earlier in the call.
mytibb <- tibble(
"x" = 1:3,
"y" = 4:6,
"z" = x ^ 2 + y,
"t" = c("math", "stat", "cs")
)
mytibb
class(mytibb)
```
---
# Getting Help
💡 We can get **help on a package** by typing `package?packagename` such as:
```{r}
package?tibble
```
💡 We can get **help on a specific function** within that package by typing `?packagename::functionname` such as:
```{r}
?tibble::tibble
```
---
- We can **coerce** a `data.frame` to a `tibble` and **coerce** a `tibble` easily back to a `data.frame` as well.
```{r}
head(iris)
class(iris)
```
---
```{r}
as_tibble(iris)
```
---
# Tibbles vs. data.frame
- There are **two main differences** in the usage of a tibble vs. a classic data.frame: **printing** and **subsetting**.
- `Tibbles` have a refined print method that **shows only the first 10 rows**, and all the columns that fit on screen.
- In `Tibbles`, each column reports **its type**, a nice feature borrowed from `str()`.
---
- We can explicitly `print()` the `Tibbles` and control the number of rows (n) and the width of the display.
```{r}
# coerce iris data set into a tibble
iris_tibble <- as_tibble(iris)
```
```{r}
# `n` specifies the number of rows
# `width = Inf` will display all columns:
# i do not have enough space, so i set n = 15
iris_tibble %>%
print(n = 15, width = Inf)
```
---
- Another option is to use `RStudio’s` **built-in data viewer** to get a scrollable view of the complete dataset.
```{r, eval=FALSE}
# run this on R console
iris_tibble %>%
View()
```
---
- We can pull out a single variable by `$` and `[[ ]]`.
- `$` only extracts by **name** and
- `[[ ]]` can extract by **name** or **position**.
```{r, eval=F}
# run this on R console
# Extract by column name
iris_tibble$Petal.Width
# Extract by column name
iris_tibble[["Petal.Width"]]
# Extract by position
iris_tibble[[4]]
```
---
- To use these in a pipe, we need to use the **special placeholder** `.` :
```{r, eval=F}
# run this on R console
iris_tibble %>% .$Petal.Width
iris_tibble %>% .[["Petal.Width"]]
```
- Use `as.data.frame()` to turn a tibble back to a data.frame:
```{r}
class(as.data.frame(iris_tibble))
```
---
# Data Importing
- One of the most commons ways of **storing and sharing data** for analysis is through **electronic spreadsheets**.
- A spreadsheet stores data in **rows** and **columns** and it is basically a **file version** of a **data frame**.
- When saving such a table to a computer file one needs a way to define when a new row or column ends and the other begins. This in turn defines the cells in which single values are stored.
- Sometimes the first row contains **column names** rather than data.
- We call this a **header** and when reading data from a spreadsheet it is important to know if the file has a header or not.
- **Most reading functions assume there is a header**.
- To know if the file has a header, it helps to look at the file before trying to read it.
- In `RStudio` we can do this by navigating to the file location, double clicking on the file and hitting _View File_.
- We start by demonstrating how to **read in a file** that is already **saved on your computer**.
---
class: center, middle
```{r echo=FALSE, out.width='30%', fig.cap=''}
knitr::include_graphics('logo/readr.png')
```
---
# The readr package
- The [readr](https://readr.tidyverse.org/) is the `tidyverse` package that includes functions for **reading rectangular data** (spreadsheets like 'text', 'csv') into `R`.
- The `readr` package can be loaded directly:
```{r}
library(readr)
```
- The following functions are available to read-in spreadsheets:
| Function | Format | Typical suffix |
|----------------|------------------------------|-------------------------|
| `read_table()` | white space separated values | txt |
| `read_delim()` | general text file format, must define delimiter | txt |
| `read_csv()` | comma separated values| csv |
| `read_csv2()` | semicolon separated values | csv |
| `read_tsv()` | tab delimited separated values | tsv |
| `read_lines()` | reads up to n_max lines from a file | - |
---
💡 We can get help on a package by typing `package?readr` and get help on a specific function
within that package by typing: `?readr::read_csv`.
```{r}
# let's get some help
?readr::read_csv
# focus on the arguments: file, col_names, skip, n_max
```
- More details on [Tidyverse readr](https://readr.tidyverse.org/).
---
class: center, middle
```{r echo=FALSE, out.width='30%', fig.cap=''}
knitr::include_graphics('logo/readxl.png')
```
---
# The readxl package
- The [readxl](https://readxl.tidyverse.org/) package provides functions to read in **Microsoft Excel** spreadsheets.
- The `readxl` package does not come with `tidyverse`. For that reason, we need to install `readxl` package and then load it.
```{r, eval=FALSE}
# install readxl
install.packages("readxl")
```
```{r}
# load readxl
library(readxl)
```
| Function | Format | Typical suffix |
|-----------------|-------------------------|-----------------|
| `read_excel()` | auto detect the format | xls, xlsx|
| `read_xls()` | original format | xls |
| `read_xlsx()` | new format | xlsx |
---
💡 We can get help on a package by typing `package?readxl` and get help on a specific function
within that package by typing `?readxl::read_xlsx`.
```{r}
# let's get some help
?readxl::read_excel
# focus on the arguments: file, sheet, col_names, skip
```
- More details on [Tidyverse readxl](https://readxl.tidyverse.org/).
---
```{css echo=FALSE}
.pull-left {
float: left;
width: 50%;
}
.pull-right {
float: right;
width: 50%;
}
```
.pull-left[
```{r echo=FALSE, results='asis', out.height='50%', out.width='80%', fig.cap=''}
knitr::include_graphics('images/epi1.png')
```
]
--
.pull-right[
```{r echo=FALSE, results='asis', out.height='100%', out.width='100%', fig.cap=''}
knitr::include_graphics('images/epi2.png')
```
]
--
- Please walk around https://epi.yale.edu/.
---
```{r echo=FALSE, results='asis', out.height='100%', out.width='90%', fig.cap=''}
knitr::include_graphics('images/googlemob.png')
```
--
- Please walk around hhttps://www.google.com/covid19/mobility/.
---
```{r echo=FALSE, results='asis', out.height='100%', out.width='90%', fig.cap=''}
knitr::include_graphics('images/transmonee.png')
```
--
- Please walk around https://transmonee.org/.
---
- **Hands-on example:** Visit `01-importing.Rmd` file.
---
class: center, middle
```{r echo=FALSE, out.width='30%', fig.cap=''}
knitr::include_graphics('logo/dplyr.jpeg')
```
---
# The dplyr package
- The [dplyr](https://dplyr.tidyverse.org/) package is one part of a larger `tidyverse` ecosystem that enables us to work with data in tidy data formats.
- Here are basic `dplyr` functions:
| Function | Description |
|----------------|----------------------------------------------------|
| `filter()` | pick observations by their values (chooses rows) |
| `select()` | pick variables by column names (selects columns) |
| `arrange()` | sort rows by variables |
| `mutate()` | create new variables with functions of existing variables |
| `summarise()` | collapse many values down to a single summary |
---
# filter()
- The [filter()](https://dplyr.tidyverse.org/reference/filter.html) function is used to **subset the rows** of `data` based on the conditions applied to the column values to determine which rows should be retained.
- The first argument to this function is the data
frame and the subsequent arguments are **comparison operators** applied to the columns.
```{r, eval=F}
#The filter() function is used to subset the rows of .data,
#applying the expressions in ... to the column values to determine
#which rows should be retained.
filter(.data, ...)
```
- It supports **unquoting** in variable names.
- Multiple conditions can be applied via **Boolean operators**.
- More details on [filter()](https://dplyr.tidyverse.org/reference/filter.html).
---
- Remember the **comparison** and **Boolean** operators.
```{r echo=FALSE, out.width='90%', fig.cap=''}
knitr::include_graphics('images/comparison.png')
knitr::include_graphics('images/boolean.png')
```
---
# select()
- The [select()](https://dplyr.tidyverse.org/reference/select.html) (and optionally `rename()`) select variables in a data frame based on their **column name**.
- The first argument to this function is the data
frame and the subsequent arguments are the **columns to keep**.
```{r, eval=F}
select(.data, ...)
```
- Variable names are quoted (or **unquoted**) expression separated by commas.
- The `select()` keeps only the variable you mention.
- The `rename()` keeps all the variables.
---
# Overview of selection features
- Tidyverse selections implement a dialect of R where operators make it easy to select variables:
<br>
```html
: for selecting a range of consecutive variables.
! for taking the complement of a set of variables.
& and | for selecting the intersection or the union of two sets of variables.
c() for combining selections.
```
---
- It is worth knowing that `dplyr` comes with a number of [select() helpers](https://www.rdocumentation.org/packages/dplyr/versions/0.7.2/topics/select_helpers), which are functions that allow us to select columns based on their names.
- These functions allow you to select variables based on their names.
<br>
```html
starts_with(): starts with a prefix.
ends_with(): ends with a prefix.
contains(): contains a literal string.
matches(): matches a regular expression.
num_range(): a numerical range like x01, x02, x03.
one_of(): variables in character vector.
everything(): all variables.
```
- More details on [select()](https://dplyr.tidyverse.org/reference/select.html).
---
# arrange()
- The [arrange()](https://dplyr.tidyverse.org/reference/arrange.html) **orders the rows** of a data frame by the values of selected columns.
- The first argument to this function is the data
frame and the subsequent arguments are comma separated list of unquoted variable names, or expressions involving variable names.
```{r, eval=F}
arrange(.data, ...)
```
- To sort from highest to lowest value or from Z to A, add `desc()` to `arrange()`.
- More details on [arrange()](https://dplyr.tidyverse.org/reference/arrange.html).
---
# mutate()
- The [mutate()](https://dplyr.tidyverse.org/reference/mutate.html) **creates new columns** based on the values in existing columns and **preserves** existing ones.
```{r, eval=F}
mutate(.data, ...)
```
- The new and edited columns will not permanently be added to the existing data frame -- unless we explicitly save the output.
- **New variables** overwrite existing variables of the same name.
- The `transmute()` adds new variables and drops existing ones.
- More details on [mutate()](https://dplyr.tidyverse.org/reference/mutate.html).
---
# case_when()
- The [case_when()](https://dplyr.tidyverse.org/reference/case_when.html) allows us to vectorize **multiple** `if_else()` statements.
- It is `R` equivalent of the `SQL CASE WHEN` statement.
- A sequence of two-sided two sided formulas. LHS determines which values match this case. RHS provides the replacement values.
- See the example in `02-manipulating.Rmd`.
---
# summarise()
- The [summarise()](https://dplyr.tidyverse.org/reference/summarise.html) summarizes one or more variables in data
frame.
- Summarizing should be done with an expression that returns a single value like `min()`,
`max()`, `n()`, `sum()`.
- To use the function we just add our new column name,
and after the equal sign the mathematics of what needs to happen, column_name = function(variable).
```{r, eval=F}
summarise(.data, column_name = function(variable))
```
- You can add multiple summary functions behind each other. Here are the most commonly
used functions:
| Explanation | Function |
|--------------|-----------------------|
| Center | `mean()`, `median()` |
| Spread | `var()`, `sd()` |
| Range | `min()`, `max()` |
| Position | `first()`, `last()`, `nth()`|
| Count | `n()`, `n_distinct()`|
| the number of observations | `n()` |
| the numbers of unique values of the variable | `n_distinct()`|
- More info [summarise()](https://dplyr.tidyverse.org/reference/summarise.html).
---
# group_by()
- In most cases, we want to get summaries by a group.
- The `group_by()` is often used together with `summarise()` which takes the column names as arguments that contain the **categorical** variables for which you want
to calculate the summary statistics.
```{r, eval=F}
group_by(.data, ...)
```
- We can also group by multiple columns.
- More details on [group_by()](https://dplyr.tidyverse.org/reference/group_by.html).
---
- **Hands-on example:** Visit `02-manipulating.Rmd` file.
- **Study by yourself**: https://www.riinu.me/2020/02/r-filtering-with-na-values/.
---
class: center, middle
```{r echo=FALSE, out.width='30%', fig.cap=''}
knitr::include_graphics('logo/tidyr.png')
```
---
# The tidyr package
- The [tidyr](https://tidyr.tidyverse.org/) package describes a standard way of storing data that is used wherever possible throughout the `Tidyverse ecosystem`.
- Tidy data is data where:
- Every column is variable.
- Every row is an observation.
- Every cell is a single value.
- Here are basic `tidyr` functions:
| Function | Description |
|-------------------|----------------------------------------|
| `pivot_longer()` |lengthens data, increasing the number of rows and decreasing the number of columns.|
| `pivot_wider()` |The inverse transformation of pivot_longer(). |
| `drop_na()` |drops rows containing missing values. |
| `replace_na()` |replaces NAs with specified values. |
- More details on [tidyr](https://tidyr.tidyverse.org/).
---
```{r echo=FALSE, out.width='90%', fig.cap=''}
knitr::include_graphics('images/wilddata.jpeg')
```
- **Hands-on example:** Visit `03-tidying.Rmd` file.
---
# Exporting data
- Similar to the `read_csv()` function used for reading CSV files into `R`, there is a `write_csv()` function that generates CSV files from data frames.
```{r, eval=FALSE}
write_csv(cth_data_filtered, "data/cth_data_reduced.csv")
```
---
# Attributions
- More on data types [Data types](https://r-coder.com/data-types-r/).
- All images used in this slide are taken from the web.
- This lecture note is mainly developed by following sources:
- [McMasterl](https://socialsciences.mcmaster.ca/jfox/Courses/R/ICPSR/tidy_lecture-slides.pdf),
- [Stat Duke](http://www2.stat.duke.edu/~cr173/Sta112_Fa16/data_wrangling.html),
- [Tidyverse](https://www.tidyverse.org/),
- [Psych252](https://psych252.github.io/psych252book/data-wrangling-1.html),
- [R4ds](https://r4ds.had.co.nz/tibbles.html),
- [Data Science Labs](https://datasciencelabs.github.io/pages/lectures.html) and
- [Cengel](https://github.com/cengel/R-data-wrangling).