Manipulating a Dataframe

Download Rmd Version

If you wish to engage with this course content via Rmd, then please click the link below to download the Rmd file.

Exercises incomplete: Download Manipulating_a_dataframe.Rmd Exercises complete: Download Manipulating_a_dataframe_complete.Rmd

Learning Objectives

• Learn how to subset data by selecting specific columns and filtering rows based on conditions using the dplyr package

• Understand how to create new variables and rename existing ones within a dataframe

• Understand sorting data based on one or more variables

• Learn how to summarise and aggregate data, including calculating summary statistics for the entire dataset and for grouped data

• Develop techniques for identifying and filtering out missing values in a dataset

Introduction

The package dplyr from the Tidyverse contains functions for manipulating dataframes. We will use the penguins dataset contained in the palmerpenguins package. Note that this data is already in tidy format.

library(palmerpenguins)  # loads `penguins` data
head(penguins)

A tibble: 6 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Torgersen	39.1	18.7	181	3750	male	2007
Adelie	Torgersen	39.5	17.4	186	3800	female	2007
Adelie	Torgersen	40.3	18.0	195	3250	female	2007
Adelie	Torgersen	NA	NA	NA	NA	NA	2007
Adelie	Torgersen	36.7	19.3	193	3450	female	2007
Adelie	Torgersen	39.3	20.6	190	3650	male	2007

Sub-setting data

For taking subsets of data, we can use the functions select and filter from dplyr:

• dplyr::select is used to keep only specified columns of the dataframe, i.e. to ‘select’ certain variables from the data.

• dplyr::filter is used to keep rows that meet some specified condition, i.e. to ‘filter’ the observations in the data.

Selecting columns

To use select, name the columns to keep after supplying the dataframe (or tibble), like so:

dplyr::select(data, column1, column2, ...)

Note: We don’t use strings to specify the columns! Instead, we write them as if they were variables e.g. dplyr::select(data, column1) instead of dplyr::select(data, "column1").

Let’s first check the column names in our tibble:

colnames(penguins)

1. 'species'
2. 'island'
3. 'bill_length_mm'
4. 'bill_depth_mm'
5. 'flipper_length_mm'
6. 'body_mass_g'
7. 'sex'
8. 'year'

Now let’s select the ‘species’, ‘year’, ‘sex’, and ‘body mass’ columns:

# Select species, year, sex, and body mass
penguins_selected <- dplyr::select(penguins, species, year, sex, body_mass_g)

head(penguins_selected)

A tibble: 6 × 4

species	year	sex	body_mass_g
<fct>	<int>	<fct>	<int>
Adelie	2007	male	3750
Adelie	2007	female	3800
Adelie	2007	female	3250
Adelie	2007	NA	NA
Adelie	2007	female	3450
Adelie	2007	male	3650

The select function can also be used to remove columns by using the - operator. For example, we may want to remove only two columns, ‘island’ and ‘year’:

# Remove the island and year columns by using `-`.
penguins_deselected <- dplyr::select(penguins, -island, -year)

# Take a look at the columns
colnames(penguins_deselected)

1. 'species'
2. 'bill_length_mm'
3. 'bill_depth_mm'
4. 'flipper_length_mm'
5. 'body_mass_g'
6. 'sex'

Filtering rows

The filter function from dplyr is used to keep rows from a dataframe/tibble that meet a predicate condition in terms of the column values, i.e. a statement that is either TRUE or FALSE.

Comparitive operators

One way to create predicate conditions is to use the comparative operators, which should be familiar:

x == y   : equal
x != y   : not equal
x > y    : greater than
x < y    : less than
x >= y   : greater than or equal
x <= y   : less than or equal

For example, suppose we want to keep all penguin data only from the year 2008. In base R, we could accomplish this by indexing on a boolean vector created using the == operator:

# Create boolean vector indicating where year = 2008
is_year_2008 <- penguins$year == 2008
head(is_year_2008, n = 75)

1. FALSE
2. FALSE
3. FALSE
4. FALSE
5. FALSE
6. FALSE
7. FALSE
8. FALSE
9. FALSE
10. FALSE
11. FALSE
12. FALSE
13. FALSE
14. FALSE
15. FALSE
16. FALSE
17. FALSE
18. FALSE
19. FALSE
20. FALSE
21. FALSE
22. FALSE
23. FALSE
24. FALSE
25. FALSE
26. FALSE
27. FALSE
28. FALSE
29. FALSE
30. FALSE
31. FALSE
32. FALSE
33. FALSE
34. FALSE
35. FALSE
36. FALSE
37. FALSE
38. FALSE
39. FALSE
40. FALSE
41. FALSE
42. FALSE
43. FALSE
44. FALSE
45. FALSE
46. FALSE
47. FALSE
48. FALSE
49. FALSE
50. FALSE
51. TRUE
52. TRUE
53. TRUE
54. TRUE
55. TRUE
56. TRUE
57. TRUE
58. TRUE
59. TRUE
60. TRUE
61. TRUE
62. TRUE
63. TRUE
64. TRUE
65. TRUE
66. TRUE
67. TRUE
68. TRUE
69. TRUE
70. TRUE
71. TRUE
72. TRUE
73. TRUE
74. TRUE
75. TRUE

# Index on this vector
head(penguins[is_year_2008,])

A tibble: 6 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Biscoe	39.6	17.7	186	3500	female	2008
Adelie	Biscoe	40.1	18.9	188	4300	male	2008
Adelie	Biscoe	35.0	17.9	190	3450	female	2008
Adelie	Biscoe	42.0	19.5	200	4050	male	2008
Adelie	Biscoe	34.5	18.1	187	2900	female	2008
Adelie	Biscoe	41.4	18.6	191	3700	male	2008

The filter function works by specifying the predicate condition for filtering in terms of the column names, like so:

dplyr::filter(data, condition)  # keep rows satisfying 'condition'

So to filter the penguins data to keep only the 2008 observations:

penguins_2008 <- dplyr::filter(penguins, year == 2008)
head(penguins_2008)

A tibble: 6 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Biscoe	39.6	17.7	186	3500	female	2008
Adelie	Biscoe	40.1	18.9	188	4300	male	2008
Adelie	Biscoe	35.0	17.9	190	3450	female	2008
Adelie	Biscoe	42.0	19.5	200	4050	male	2008
Adelie	Biscoe	34.5	18.1	187	2900	female	2008
Adelie	Biscoe	41.4	18.6	191	3700	male	2008

Logical operators

We can build more complicated predicate conditions by using the logical operators on columns:

A & B   : A AND B   e.g. col1 == 2 & col2 == 10
A | B   : A OR B    e.g. col1 > 2 | col2 != 10
!A      : NOT A     e.g. !(col1 < 2)

In addition, filter allows us to specify multiple AND operations by listing out multiple predicate conditions:

dplyr::filter(data, condition1, condition2, ...)  # keep rows satisfying
                                                  # 'condition1' AND 'condition2' AND ...

Some examples on the penguin data:

# Observations of male penguins from the year 2008:
dplyr::filter(penguins, sex == "male" & year == 2008)  # using &
dplyr::filter(penguins, sex == "male", year == 2008)  # listing multiple conditions in filter

# Observations of penguins with bill length > 40mm or bill depth < 20mm 
dplyr::filter(penguins, bill_length_mm > 40 | bill_depth_mm < 20)

# Observations except those of male penguins on the island of Biscoe
dplyr::filter(penguins, !(sex == "male" & island == "Biscoe"))

A tibble: 57 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Biscoe	40.1	18.9	188	4300	male	2008
Adelie	Biscoe	42.0	19.5	200	4050	male	2008
Adelie	Biscoe	41.4	18.6	191	3700	male	2008
Adelie	Biscoe	40.6	18.8	193	3800	male	2008
Adelie	Biscoe	37.6	19.1	194	3750	male	2008
Adelie	Biscoe	41.3	21.1	195	4400	male	2008
Adelie	Biscoe	41.1	18.2	192	4050	male	2008
Adelie	Biscoe	41.6	18.0	192	3950	male	2008
Adelie	Biscoe	41.1	19.1	188	4100	male	2008
Adelie	Torgersen	41.8	19.4	198	4450	male	2008
Adelie	Torgersen	39.7	18.4	190	3900	male	2008
Adelie	Torgersen	45.8	18.9	197	4150	male	2008
Adelie	Torgersen	42.8	18.5	195	4250	male	2008
Adelie	Torgersen	37.2	19.4	184	3900	male	2008
Adelie	Torgersen	42.1	19.1	195	4000	male	2008
Adelie	Torgersen	42.9	17.6	196	4700	male	2008
Adelie	Torgersen	35.1	19.4	193	4200	male	2008
Adelie	Dream	41.3	20.3	194	3550	male	2008
Adelie	Dream	36.3	19.5	190	3800	male	2008
Adelie	Dream	38.3	19.2	189	3950	male	2008
Adelie	Dream	41.1	18.1	205	4300	male	2008
Adelie	Dream	39.6	18.1	186	4450	male	2008
Adelie	Dream	40.8	18.9	208	4300	male	2008
Adelie	Dream	40.3	18.5	196	4350	male	2008
Adelie	Dream	43.2	18.5	192	4100	male	2008
Gentoo	Biscoe	48.4	16.3	220	5400	male	2008
Gentoo	Biscoe	44.4	17.3	219	5250	male	2008
Gentoo	Biscoe	48.7	15.7	208	5350	male	2008
Gentoo	Biscoe	49.6	16.0	225	5700	male	2008
Gentoo	Biscoe	49.6	15.0	216	4750	male	2008
Gentoo	Biscoe	50.5	15.9	222	5550	male	2008
Gentoo	Biscoe	50.5	15.9	225	5400	male	2008
Gentoo	Biscoe	45.2	15.8	215	5300	male	2008
Gentoo	Biscoe	48.5	14.1	220	5300	male	2008
Gentoo	Biscoe	50.1	15.0	225	5000	male	2008
Gentoo	Biscoe	45.0	15.4	220	5050	male	2008
Gentoo	Biscoe	45.5	15.0	220	5000	male	2008
Gentoo	Biscoe	50.4	15.3	224	5550	male	2008
Gentoo	Biscoe	46.2	14.9	221	5300	male	2008
Gentoo	Biscoe	54.3	15.7	231	5650	male	2008
Gentoo	Biscoe	49.8	16.8	230	5700	male	2008
Gentoo	Biscoe	49.5	16.2	229	5800	male	2008
Gentoo	Biscoe	50.7	15.0	223	5550	male	2008
Gentoo	Biscoe	46.4	15.6	221	5000	male	2008
Gentoo	Biscoe	48.2	15.6	221	5100	male	2008
Gentoo	Biscoe	48.6	16.0	230	5800	male	2008
Gentoo	Biscoe	51.1	16.3	220	6000	male	2008
Gentoo	Biscoe	45.2	16.4	223	5950	male	2008
Chinstrap	Dream	49.5	19.0	200	3800	male	2008
Chinstrap	Dream	52.8	20.0	205	4550	male	2008
Chinstrap	Dream	54.2	20.8	201	4300	male	2008
Chinstrap	Dream	51.0	18.8	203	4100	male	2008
Chinstrap	Dream	49.7	18.6	195	3600	male	2008
Chinstrap	Dream	52.0	20.7	210	4800	male	2008
Chinstrap	Dream	53.5	19.9	205	4500	male	2008
Chinstrap	Dream	49.0	19.5	210	3950	male	2008
Chinstrap	Dream	50.9	19.1	196	3550	male	2008

A tibble: 57 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Biscoe	40.1	18.9	188	4300	male	2008
Adelie	Biscoe	42.0	19.5	200	4050	male	2008
Adelie	Biscoe	41.4	18.6	191	3700	male	2008
Adelie	Biscoe	40.6	18.8	193	3800	male	2008
Adelie	Biscoe	37.6	19.1	194	3750	male	2008
Adelie	Biscoe	41.3	21.1	195	4400	male	2008
Adelie	Biscoe	41.1	18.2	192	4050	male	2008
Adelie	Biscoe	41.6	18.0	192	3950	male	2008
Adelie	Biscoe	41.1	19.1	188	4100	male	2008
Adelie	Torgersen	41.8	19.4	198	4450	male	2008
Adelie	Torgersen	39.7	18.4	190	3900	male	2008
Adelie	Torgersen	45.8	18.9	197	4150	male	2008
Adelie	Torgersen	42.8	18.5	195	4250	male	2008
Adelie	Torgersen	37.2	19.4	184	3900	male	2008
Adelie	Torgersen	42.1	19.1	195	4000	male	2008
Adelie	Torgersen	42.9	17.6	196	4700	male	2008
Adelie	Torgersen	35.1	19.4	193	4200	male	2008
Adelie	Dream	41.3	20.3	194	3550	male	2008
Adelie	Dream	36.3	19.5	190	3800	male	2008
Adelie	Dream	38.3	19.2	189	3950	male	2008
Adelie	Dream	41.1	18.1	205	4300	male	2008
Adelie	Dream	39.6	18.1	186	4450	male	2008
Adelie	Dream	40.8	18.9	208	4300	male	2008
Adelie	Dream	40.3	18.5	196	4350	male	2008
Adelie	Dream	43.2	18.5	192	4100	male	2008
Gentoo	Biscoe	48.4	16.3	220	5400	male	2008
Gentoo	Biscoe	44.4	17.3	219	5250	male	2008
Gentoo	Biscoe	48.7	15.7	208	5350	male	2008
Gentoo	Biscoe	49.6	16.0	225	5700	male	2008
Gentoo	Biscoe	49.6	15.0	216	4750	male	2008
Gentoo	Biscoe	50.5	15.9	222	5550	male	2008
Gentoo	Biscoe	50.5	15.9	225	5400	male	2008
Gentoo	Biscoe	45.2	15.8	215	5300	male	2008
Gentoo	Biscoe	48.5	14.1	220	5300	male	2008
Gentoo	Biscoe	50.1	15.0	225	5000	male	2008
Gentoo	Biscoe	45.0	15.4	220	5050	male	2008
Gentoo	Biscoe	45.5	15.0	220	5000	male	2008
Gentoo	Biscoe	50.4	15.3	224	5550	male	2008
Gentoo	Biscoe	46.2	14.9	221	5300	male	2008
Gentoo	Biscoe	54.3	15.7	231	5650	male	2008
Gentoo	Biscoe	49.8	16.8	230	5700	male	2008
Gentoo	Biscoe	49.5	16.2	229	5800	male	2008
Gentoo	Biscoe	50.7	15.0	223	5550	male	2008
Gentoo	Biscoe	46.4	15.6	221	5000	male	2008
Gentoo	Biscoe	48.2	15.6	221	5100	male	2008
Gentoo	Biscoe	48.6	16.0	230	5800	male	2008
Gentoo	Biscoe	51.1	16.3	220	6000	male	2008
Gentoo	Biscoe	45.2	16.4	223	5950	male	2008
Chinstrap	Dream	49.5	19.0	200	3800	male	2008
Chinstrap	Dream	52.8	20.0	205	4550	male	2008
Chinstrap	Dream	54.2	20.8	201	4300	male	2008
Chinstrap	Dream	51.0	18.8	203	4100	male	2008
Chinstrap	Dream	49.7	18.6	195	3600	male	2008
Chinstrap	Dream	52.0	20.7	210	4800	male	2008
Chinstrap	Dream	53.5	19.9	205	4500	male	2008
Chinstrap	Dream	49.0	19.5	210	3950	male	2008
Chinstrap	Dream	50.9	19.1	196	3550	male	2008

A tibble: 333 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Torgersen	39.1	18.7	181	3750	male	2007
Adelie	Torgersen	39.5	17.4	186	3800	female	2007
Adelie	Torgersen	40.3	18.0	195	3250	female	2007
Adelie	Torgersen	36.7	19.3	193	3450	female	2007
Adelie	Torgersen	38.9	17.8	181	3625	female	2007
Adelie	Torgersen	39.2	19.6	195	4675	male	2007
Adelie	Torgersen	34.1	18.1	193	3475	NA	2007
Adelie	Torgersen	42.0	20.2	190	4250	NA	2007
Adelie	Torgersen	37.8	17.1	186	3300	NA	2007
Adelie	Torgersen	37.8	17.3	180	3700	NA	2007
Adelie	Torgersen	41.1	17.6	182	3200	female	2007
Adelie	Torgersen	36.6	17.8	185	3700	female	2007
Adelie	Torgersen	38.7	19.0	195	3450	female	2007
Adelie	Torgersen	42.5	20.7	197	4500	male	2007
Adelie	Torgersen	34.4	18.4	184	3325	female	2007
Adelie	Torgersen	46.0	21.5	194	4200	male	2007
Adelie	Biscoe	37.8	18.3	174	3400	female	2007
Adelie	Biscoe	37.7	18.7	180	3600	male	2007
Adelie	Biscoe	35.9	19.2	189	3800	female	2007
Adelie	Biscoe	38.2	18.1	185	3950	male	2007
Adelie	Biscoe	38.8	17.2	180	3800	male	2007
Adelie	Biscoe	35.3	18.9	187	3800	female	2007
Adelie	Biscoe	40.6	18.6	183	3550	male	2007
Adelie	Biscoe	40.5	17.9	187	3200	female	2007
Adelie	Biscoe	37.9	18.6	172	3150	female	2007
Adelie	Biscoe	40.5	18.9	180	3950	male	2007
Adelie	Dream	39.5	16.7	178	3250	female	2007
Adelie	Dream	37.2	18.1	178	3900	male	2007
Adelie	Dream	39.5	17.8	188	3300	female	2007
Adelie	Dream	40.9	18.9	184	3900	male	2007
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
Chinstrap	Dream	46.9	16.6	192	2700	female	2008
Chinstrap	Dream	53.5	19.9	205	4500	male	2008
Chinstrap	Dream	49.0	19.5	210	3950	male	2008
Chinstrap	Dream	46.2	17.5	187	3650	female	2008
Chinstrap	Dream	50.9	19.1	196	3550	male	2008
Chinstrap	Dream	45.5	17.0	196	3500	female	2008
Chinstrap	Dream	50.9	17.9	196	3675	female	2009
Chinstrap	Dream	50.8	18.5	201	4450	male	2009
Chinstrap	Dream	50.1	17.9	190	3400	female	2009
Chinstrap	Dream	49.0	19.6	212	4300	male	2009
Chinstrap	Dream	51.5	18.7	187	3250	male	2009
Chinstrap	Dream	49.8	17.3	198	3675	female	2009
Chinstrap	Dream	48.1	16.4	199	3325	female	2009
Chinstrap	Dream	51.4	19.0	201	3950	male	2009
Chinstrap	Dream	45.7	17.3	193	3600	female	2009
Chinstrap	Dream	50.7	19.7	203	4050	male	2009
Chinstrap	Dream	42.5	17.3	187	3350	female	2009
Chinstrap	Dream	52.2	18.8	197	3450	male	2009
Chinstrap	Dream	45.2	16.6	191	3250	female	2009
Chinstrap	Dream	49.3	19.9	203	4050	male	2009
Chinstrap	Dream	50.2	18.8	202	3800	male	2009
Chinstrap	Dream	45.6	19.4	194	3525	female	2009
Chinstrap	Dream	51.9	19.5	206	3950	male	2009
Chinstrap	Dream	46.8	16.5	189	3650	female	2009
Chinstrap	Dream	45.7	17.0	195	3650	female	2009
Chinstrap	Dream	55.8	19.8	207	4000	male	2009
Chinstrap	Dream	43.5	18.1	202	3400	female	2009
Chinstrap	Dream	49.6	18.2	193	3775	male	2009
Chinstrap	Dream	50.8	19.0	210	4100	male	2009
Chinstrap	Dream	50.2	18.7	198	3775	female	2009

A tibble: 256 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Torgersen	39.1	18.7	181	3750	male	2007
Adelie	Torgersen	39.5	17.4	186	3800	female	2007
Adelie	Torgersen	40.3	18.0	195	3250	female	2007
Adelie	Torgersen	NA	NA	NA	NA	NA	2007
Adelie	Torgersen	36.7	19.3	193	3450	female	2007
Adelie	Torgersen	39.3	20.6	190	3650	male	2007
Adelie	Torgersen	38.9	17.8	181	3625	female	2007
Adelie	Torgersen	39.2	19.6	195	4675	male	2007
Adelie	Torgersen	34.1	18.1	193	3475	NA	2007
Adelie	Torgersen	42.0	20.2	190	4250	NA	2007
Adelie	Torgersen	37.8	17.1	186	3300	NA	2007
Adelie	Torgersen	37.8	17.3	180	3700	NA	2007
Adelie	Torgersen	41.1	17.6	182	3200	female	2007
Adelie	Torgersen	38.6	21.2	191	3800	male	2007
Adelie	Torgersen	34.6	21.1	198	4400	male	2007
Adelie	Torgersen	36.6	17.8	185	3700	female	2007
Adelie	Torgersen	38.7	19.0	195	3450	female	2007
Adelie	Torgersen	42.5	20.7	197	4500	male	2007
Adelie	Torgersen	34.4	18.4	184	3325	female	2007
Adelie	Torgersen	46.0	21.5	194	4200	male	2007
Adelie	Biscoe	37.8	18.3	174	3400	female	2007
Adelie	Biscoe	35.9	19.2	189	3800	female	2007
Adelie	Biscoe	35.3	18.9	187	3800	female	2007
Adelie	Biscoe	40.5	17.9	187	3200	female	2007
Adelie	Biscoe	37.9	18.6	172	3150	female	2007
Adelie	Dream	39.5	16.7	178	3250	female	2007
Adelie	Dream	37.2	18.1	178	3900	male	2007
Adelie	Dream	39.5	17.8	188	3300	female	2007
Adelie	Dream	40.9	18.9	184	3900	male	2007
Adelie	Dream	36.4	17.0	195	3325	female	2007
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
Chinstrap	Dream	46.9	16.6	192	2700	female	2008
Chinstrap	Dream	53.5	19.9	205	4500	male	2008
Chinstrap	Dream	49.0	19.5	210	3950	male	2008
Chinstrap	Dream	46.2	17.5	187	3650	female	2008
Chinstrap	Dream	50.9	19.1	196	3550	male	2008
Chinstrap	Dream	45.5	17.0	196	3500	female	2008
Chinstrap	Dream	50.9	17.9	196	3675	female	2009
Chinstrap	Dream	50.8	18.5	201	4450	male	2009
Chinstrap	Dream	50.1	17.9	190	3400	female	2009
Chinstrap	Dream	49.0	19.6	212	4300	male	2009
Chinstrap	Dream	51.5	18.7	187	3250	male	2009
Chinstrap	Dream	49.8	17.3	198	3675	female	2009
Chinstrap	Dream	48.1	16.4	199	3325	female	2009
Chinstrap	Dream	51.4	19.0	201	3950	male	2009
Chinstrap	Dream	45.7	17.3	193	3600	female	2009
Chinstrap	Dream	50.7	19.7	203	4050	male	2009
Chinstrap	Dream	42.5	17.3	187	3350	female	2009
Chinstrap	Dream	52.2	18.8	197	3450	male	2009
Chinstrap	Dream	45.2	16.6	191	3250	female	2009
Chinstrap	Dream	49.3	19.9	203	4050	male	2009
Chinstrap	Dream	50.2	18.8	202	3800	male	2009
Chinstrap	Dream	45.6	19.4	194	3525	female	2009
Chinstrap	Dream	51.9	19.5	206	3950	male	2009
Chinstrap	Dream	46.8	16.5	189	3650	female	2009
Chinstrap	Dream	45.7	17.0	195	3650	female	2009
Chinstrap	Dream	55.8	19.8	207	4000	male	2009
Chinstrap	Dream	43.5	18.1	202	3400	female	2009
Chinstrap	Dream	49.6	18.2	193	3775	male	2009
Chinstrap	Dream	50.8	19.0	210	4100	male	2009
Chinstrap	Dream	50.2	18.7	198	3775	female	2009

Filtering missing values

There are other functions we can use to evaluate columns and get a true/false output. An important one is is.na(). This function evaluates a column and reports back a TRUE value when there is an NA in that column’s row. For example, we can use the head() function to look at the top 6 values of the sex column, and see that there is an NA in the fourth row.

head(penguins$sex)

1. male
2. female
3. female
4. <NA>
5. female
6. male

Levels:

1. 'female'
2. 'male'

The is.na function evaluates the whole column and gives us TRUEs whenever it sees an NA. Not surprisingly, we see a TRUE in the fourth observation.

head(is.na(penguins$sex))

1. FALSE
2. FALSE
3. FALSE
4. TRUE
5. FALSE
6. FALSE

Since is.na() gives us a TRUE/FALSE vector, we can use it with filter. The following gives us all rows where the sex column has missing data (i.e. is set to NA):

dplyr::filter(penguins, is.na(sex))

A tibble: 11 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Torgersen	NA	NA	NA	NA	NA	2007
Adelie	Torgersen	34.1	18.1	193	3475	NA	2007
Adelie	Torgersen	42.0	20.2	190	4250	NA	2007
Adelie	Torgersen	37.8	17.1	186	3300	NA	2007
Adelie	Torgersen	37.8	17.3	180	3700	NA	2007
Adelie	Dream	37.5	18.9	179	2975	NA	2007
Gentoo	Biscoe	44.5	14.3	216	4100	NA	2007
Gentoo	Biscoe	46.2	14.4	214	4650	NA	2008
Gentoo	Biscoe	47.3	13.8	216	4725	NA	2009
Gentoo	Biscoe	44.5	15.7	217	4875	NA	2009
Gentoo	Biscoe	NA	NA	NA	NA	NA	2009

To do the reverse, i.e. keep only the observations where the sex value is not missing, combine is.na with the NOT operator:

dplyr::filter(penguins, !is.na(sex))

A tibble: 333 × 8

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
Adelie	Torgersen	39.1	18.7	181	3750	male	2007
Adelie	Torgersen	39.5	17.4	186	3800	female	2007
Adelie	Torgersen	40.3	18.0	195	3250	female	2007
Adelie	Torgersen	36.7	19.3	193	3450	female	2007
Adelie	Torgersen	39.3	20.6	190	3650	male	2007
Adelie	Torgersen	38.9	17.8	181	3625	female	2007
Adelie	Torgersen	39.2	19.6	195	4675	male	2007
Adelie	Torgersen	41.1	17.6	182	3200	female	2007
Adelie	Torgersen	38.6	21.2	191	3800	male	2007
Adelie	Torgersen	34.6	21.1	198	4400	male	2007
Adelie	Torgersen	36.6	17.8	185	3700	female	2007
Adelie	Torgersen	38.7	19.0	195	3450	female	2007
Adelie	Torgersen	42.5	20.7	197	4500	male	2007
Adelie	Torgersen	34.4	18.4	184	3325	female	2007
Adelie	Torgersen	46.0	21.5	194	4200	male	2007
Adelie	Biscoe	37.8	18.3	174	3400	female	2007
Adelie	Biscoe	37.7	18.7	180	3600	male	2007
Adelie	Biscoe	35.9	19.2	189	3800	female	2007
Adelie	Biscoe	38.2	18.1	185	3950	male	2007
Adelie	Biscoe	38.8	17.2	180	3800	male	2007
Adelie	Biscoe	35.3	18.9	187	3800	female	2007
Adelie	Biscoe	40.6	18.6	183	3550	male	2007
Adelie	Biscoe	40.5	17.9	187	3200	female	2007
Adelie	Biscoe	37.9	18.6	172	3150	female	2007
Adelie	Biscoe	40.5	18.9	180	3950	male	2007
Adelie	Dream	39.5	16.7	178	3250	female	2007
Adelie	Dream	37.2	18.1	178	3900	male	2007
Adelie	Dream	39.5	17.8	188	3300	female	2007
Adelie	Dream	40.9	18.9	184	3900	male	2007
Adelie	Dream	36.4	17.0	195	3325	female	2007
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
Chinstrap	Dream	46.9	16.6	192	2700	female	2008
Chinstrap	Dream	53.5	19.9	205	4500	male	2008
Chinstrap	Dream	49.0	19.5	210	3950	male	2008
Chinstrap	Dream	46.2	17.5	187	3650	female	2008
Chinstrap	Dream	50.9	19.1	196	3550	male	2008
Chinstrap	Dream	45.5	17.0	196	3500	female	2008
Chinstrap	Dream	50.9	17.9	196	3675	female	2009
Chinstrap	Dream	50.8	18.5	201	4450	male	2009
Chinstrap	Dream	50.1	17.9	190	3400	female	2009
Chinstrap	Dream	49.0	19.6	212	4300	male	2009
Chinstrap	Dream	51.5	18.7	187	3250	male	2009
Chinstrap	Dream	49.8	17.3	198	3675	female	2009
Chinstrap	Dream	48.1	16.4	199	3325	female	2009
Chinstrap	Dream	51.4	19.0	201	3950	male	2009
Chinstrap	Dream	45.7	17.3	193	3600	female	2009
Chinstrap	Dream	50.7	19.7	203	4050	male	2009
Chinstrap	Dream	42.5	17.3	187	3350	female	2009
Chinstrap	Dream	52.2	18.8	197	3450	male	2009
Chinstrap	Dream	45.2	16.6	191	3250	female	2009
Chinstrap	Dream	49.3	19.9	203	4050	male	2009
Chinstrap	Dream	50.2	18.8	202	3800	male	2009
Chinstrap	Dream	45.6	19.4	194	3525	female	2009
Chinstrap	Dream	51.9	19.5	206	3950	male	2009
Chinstrap	Dream	46.8	16.5	189	3650	female	2009
Chinstrap	Dream	45.7	17.0	195	3650	female	2009
Chinstrap	Dream	55.8	19.8	207	4000	male	2009
Chinstrap	Dream	43.5	18.1	202	3400	female	2009
Chinstrap	Dream	49.6	18.2	193	3775	male	2009
Chinstrap	Dream	50.8	19.0	210	4100	male	2009
Chinstrap	Dream	50.2	18.7	198	3775	female	2009

Exercise: subsetting data

Write code to extract data for Gentoo penguins with weight between 4.8kg and 5.2kg (inclusive). Only keep the species, sex and body mass columns in the output. Give the numbers of resulting observations for male penguins, female penguins and penguins with unknown sex.

Solution

penguins_subset <- dplyr::filter(penguins,
                                 species == "Gentoo",
                                 4800 <= body_mass_g & body_mass_g <= 5200)
penguins_subset <- dplyr::select(penguins_subset, species, sex, body_mass_g)
penguins_subset

cat("Number of male penguins:", nrow(dplyr::filter(penguins_subset, sex == "male")), "\n")
cat("Number of female penguins:", nrow(dplyr::filter(penguins_subset, sex == "female")), "\n")
cat("Number of penguins of unknown sex:", nrow(dplyr::filter(penguins_subset, is.na(sex))), "\n")

Creating new variables and renaming variables

We often want to make a new column with some updated or transformed value. We can use the mutate function in dplyr for this:

# value1, value2,... are expressions, possibly involving column names

dplyr::mutate(data, new_column1 = value1, new_column2 = value2, ...)

For example, to add a column of IDs for identifying each row in the data:

# Add column of ID numbers
penguins_with_ids <- dplyr::mutate(penguins, id = 1:nrow(penguins))

# Optional: Put the ID column at the beginning of the dataframe
penguins_with_ids <- dplyr::relocate(penguins_with_ids, id, .before = 1)

head(penguins_with_ids)

A tibble: 6 × 9

id	species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year
<int>	<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>
1	Adelie	Torgersen	39.1	18.7	181	3750	male	2007
2	Adelie	Torgersen	39.5	17.4	186	3800	female	2007
3	Adelie	Torgersen	40.3	18.0	195	3250	female	2007
4	Adelie	Torgersen	NA	NA	NA	NA	NA	2007
5	Adelie	Torgersen	36.7	19.3	193	3450	female	2007
6	Adelie	Torgersen	39.3	20.6	190	3650	male	2007

Another example: if we wanted to calculate a new value, the ratio of bill length to bill depth, we could do the following

dplyr::mutate(penguins, bill_ratio = bill_length_mm / bill_depth_mm)

A tibble: 344 × 9

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year	bill_ratio
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>	<dbl>
Adelie	Torgersen	39.1	18.7	181	3750	male	2007	2.090909
Adelie	Torgersen	39.5	17.4	186	3800	female	2007	2.270115
Adelie	Torgersen	40.3	18.0	195	3250	female	2007	2.238889
Adelie	Torgersen	NA	NA	NA	NA	NA	2007	NA
Adelie	Torgersen	36.7	19.3	193	3450	female	2007	1.901554
Adelie	Torgersen	39.3	20.6	190	3650	male	2007	1.907767
Adelie	Torgersen	38.9	17.8	181	3625	female	2007	2.185393
Adelie	Torgersen	39.2	19.6	195	4675	male	2007	2.000000
Adelie	Torgersen	34.1	18.1	193	3475	NA	2007	1.883978
Adelie	Torgersen	42.0	20.2	190	4250	NA	2007	2.079208
Adelie	Torgersen	37.8	17.1	186	3300	NA	2007	2.210526
Adelie	Torgersen	37.8	17.3	180	3700	NA	2007	2.184971
Adelie	Torgersen	41.1	17.6	182	3200	female	2007	2.335227
Adelie	Torgersen	38.6	21.2	191	3800	male	2007	1.820755
Adelie	Torgersen	34.6	21.1	198	4400	male	2007	1.639810
Adelie	Torgersen	36.6	17.8	185	3700	female	2007	2.056180
Adelie	Torgersen	38.7	19.0	195	3450	female	2007	2.036842
Adelie	Torgersen	42.5	20.7	197	4500	male	2007	2.053140
Adelie	Torgersen	34.4	18.4	184	3325	female	2007	1.869565
Adelie	Torgersen	46.0	21.5	194	4200	male	2007	2.139535
Adelie	Biscoe	37.8	18.3	174	3400	female	2007	2.065574
Adelie	Biscoe	37.7	18.7	180	3600	male	2007	2.016043
Adelie	Biscoe	35.9	19.2	189	3800	female	2007	1.869792
Adelie	Biscoe	38.2	18.1	185	3950	male	2007	2.110497
Adelie	Biscoe	38.8	17.2	180	3800	male	2007	2.255814
Adelie	Biscoe	35.3	18.9	187	3800	female	2007	1.867725
Adelie	Biscoe	40.6	18.6	183	3550	male	2007	2.182796
Adelie	Biscoe	40.5	17.9	187	3200	female	2007	2.262570
Adelie	Biscoe	37.9	18.6	172	3150	female	2007	2.037634
Adelie	Biscoe	40.5	18.9	180	3950	male	2007	2.142857
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
Chinstrap	Dream	46.9	16.6	192	2700	female	2008	2.825301
Chinstrap	Dream	53.5	19.9	205	4500	male	2008	2.688442
Chinstrap	Dream	49.0	19.5	210	3950	male	2008	2.512821
Chinstrap	Dream	46.2	17.5	187	3650	female	2008	2.640000
Chinstrap	Dream	50.9	19.1	196	3550	male	2008	2.664921
Chinstrap	Dream	45.5	17.0	196	3500	female	2008	2.676471
Chinstrap	Dream	50.9	17.9	196	3675	female	2009	2.843575
Chinstrap	Dream	50.8	18.5	201	4450	male	2009	2.745946
Chinstrap	Dream	50.1	17.9	190	3400	female	2009	2.798883
Chinstrap	Dream	49.0	19.6	212	4300	male	2009	2.500000
Chinstrap	Dream	51.5	18.7	187	3250	male	2009	2.754011
Chinstrap	Dream	49.8	17.3	198	3675	female	2009	2.878613
Chinstrap	Dream	48.1	16.4	199	3325	female	2009	2.932927
Chinstrap	Dream	51.4	19.0	201	3950	male	2009	2.705263
Chinstrap	Dream	45.7	17.3	193	3600	female	2009	2.641618
Chinstrap	Dream	50.7	19.7	203	4050	male	2009	2.573604
Chinstrap	Dream	42.5	17.3	187	3350	female	2009	2.456647
Chinstrap	Dream	52.2	18.8	197	3450	male	2009	2.776596
Chinstrap	Dream	45.2	16.6	191	3250	female	2009	2.722892
Chinstrap	Dream	49.3	19.9	203	4050	male	2009	2.477387
Chinstrap	Dream	50.2	18.8	202	3800	male	2009	2.670213
Chinstrap	Dream	45.6	19.4	194	3525	female	2009	2.350515
Chinstrap	Dream	51.9	19.5	206	3950	male	2009	2.661538
Chinstrap	Dream	46.8	16.5	189	3650	female	2009	2.836364
Chinstrap	Dream	45.7	17.0	195	3650	female	2009	2.688235
Chinstrap	Dream	55.8	19.8	207	4000	male	2009	2.818182
Chinstrap	Dream	43.5	18.1	202	3400	female	2009	2.403315
Chinstrap	Dream	49.6	18.2	193	3775	male	2009	2.725275
Chinstrap	Dream	50.8	19.0	210	4100	male	2009	2.673684
Chinstrap	Dream	50.2	18.7	198	3775	female	2009	2.684492

Note: the output of mutate is not just a new column on its own, but the whole dataframe with the new column appended. To update penguins to have the new column, we need to overwrite it:

penguins <- dplyr::mutate(penguins, bill_ratio = bill_length_mm / bill_depth_mm)
colnames(penguins)

1. 'species'
2. 'island'
3. 'bill_length_mm'
4. 'bill_depth_mm'
5. 'flipper_length_mm'
6. 'body_mass_g'
7. 'sex'
8. 'year'
9. 'bill_ratio'

When we just want to change the name of a column, we can use the rename function in dplyr:

dplyr::rename(data, New_column_name1 = Old_column_name1, New_column_name2 = Old_column_name2, ...)

For example, we could change the name of some variables to remove the units (probably not recommended in general, but it serves as an example):

penguins_renamed <- dplyr::rename(penguins,
                                  bill_length = bill_length_mm,
                                  bill_depth = bill_depth_mm,
                                  flipper_length = flipper_length_mm,
                                  body_mass = body_mass_g
                                  )
colnames(penguins_renamed)

1. 'species'
2. 'island'
3. 'bill_length'
4. 'bill_depth'
5. 'flipper_length'
6. 'body_mass'
7. 'sex'
8. 'year'
9. 'bill_ratio'

Exercise: creating variables

Update the penguins dataframe so that it contains an ID column with IDs represented as strings.

Solution

# Solution 1
penguins <- dplyr::mutate(penguins, id = as.character(1:nrow(penguins)))

# Solution 2
penguins <- dplyr::mutate(penguins, id = 1:nrow(penguins))  # int IDs
penguins <- dplyr::mutate(penguins, id = as.character(id))  # update column to strings

Sorting observations

We can sort the rows of data by the value of a particular column using the arrange function in dplyr. By default, the sorting is performed in increasing order, although we can use the desc() function on the column to sort in descending order:

dplyr::arrange(data, col)  # sort rows by ascending order of values from column
                           # `col`

dplyr::arrange(data, dplyr::desc(col))  # sort rows by descending order of
                                        # values from column `col`

For example, to sort the penguins data by descending value of flipper length:

penguins_sorted_flipper <- dplyr::arrange(penguins, dplyr::desc(flipper_length_mm))
penguins_sorted_flipper

A tibble: 344 × 9

species	island	bill_length_mm	bill_depth_mm	flipper_length_mm	body_mass_g	sex	year	bill_ratio
<fct>	<fct>	<dbl>	<dbl>	<int>	<int>	<fct>	<int>	<dbl>
Gentoo	Biscoe	54.3	15.7	231	5650	male	2008	3.458599
Gentoo	Biscoe	50.0	16.3	230	5700	male	2007	3.067485
Gentoo	Biscoe	59.6	17.0	230	6050	male	2007	3.505882
Gentoo	Biscoe	49.8	16.8	230	5700	male	2008	2.964286
Gentoo	Biscoe	48.6	16.0	230	5800	male	2008	3.037500
Gentoo	Biscoe	52.1	17.0	230	5550	male	2009	3.064706
Gentoo	Biscoe	51.5	16.3	230	5500	male	2009	3.159509
Gentoo	Biscoe	55.1	16.0	230	5850	male	2009	3.443750
Gentoo	Biscoe	49.5	16.2	229	5800	male	2008	3.055556
Gentoo	Biscoe	49.8	15.9	229	5950	male	2009	3.132075
Gentoo	Biscoe	50.8	17.3	228	5600	male	2009	2.936416
Gentoo	Biscoe	52.2	17.1	228	5400	male	2009	3.052632
Gentoo	Biscoe	55.9	17.0	228	5600	male	2009	3.288235
Gentoo	Biscoe	49.1	15.0	228	5500	male	2009	3.273333
Gentoo	Biscoe	50.8	15.7	226	5200	male	2009	3.235669
Gentoo	Biscoe	49.6	16.0	225	5700	male	2008	3.100000
Gentoo	Biscoe	50.5	15.9	225	5400	male	2008	3.176101
Gentoo	Biscoe	50.1	15.0	225	5000	male	2008	3.340000
Gentoo	Biscoe	51.1	16.5	225	5250	male	2009	3.096970
Gentoo	Biscoe	50.4	15.3	224	5550	male	2008	3.294118
Gentoo	Biscoe	50.0	15.9	224	5350	male	2009	3.144654
Gentoo	Biscoe	49.5	16.1	224	5650	male	2009	3.074534
Gentoo	Biscoe	50.7	15.0	223	5550	male	2008	3.380000
Gentoo	Biscoe	45.2	16.4	223	5950	male	2008	2.756098
Gentoo	Biscoe	48.7	15.1	222	5350	male	2007	3.225166
Gentoo	Biscoe	47.3	15.3	222	5250	male	2007	3.091503
Gentoo	Biscoe	50.5	15.9	222	5550	male	2008	3.176101
Gentoo	Biscoe	46.9	14.6	222	4875	female	2009	3.212329
Gentoo	Biscoe	48.8	16.2	222	6000	male	2009	3.012346
Gentoo	Biscoe	50.4	15.7	222	5750	male	2009	3.210191
⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮	⋮
Adelie	Torgersen	37.2	19.4	184	3900	male	2008	1.917526
Adelie	Biscoe	39.7	18.9	184	3550	male	2009	2.100529
Adelie	Dream	36.6	18.4	184	3475	female	2009	1.989130
Adelie	Biscoe	40.6	18.6	183	3550	male	2007	2.182796
Adelie	Biscoe	37.7	16.0	183	3075	female	2009	2.356250
Adelie	Torgersen	41.1	17.6	182	3200	female	2007	2.335227
Adelie	Dream	36.5	18.0	182	3150	female	2007	2.027778
Adelie	Dream	41.1	19.0	182	3425	male	2007	2.163158
Adelie	Torgersen	39.1	18.7	181	3750	male	2007	2.090909
Adelie	Torgersen	38.9	17.8	181	3625	female	2007	2.185393
Adelie	Dream	37.6	19.3	181	3300	female	2007	1.948187
Adelie	Biscoe	36.5	16.6	181	2850	female	2008	2.198795
Adelie	Biscoe	38.1	17.0	181	3175	female	2009	2.241176
Chinstrap	Dream	58.0	17.8	181	3700	female	2007	3.258427
Chinstrap	Dream	42.4	17.3	181	3600	female	2007	2.450867
Adelie	Torgersen	37.8	17.3	180	3700	NA	2007	2.184971
Adelie	Biscoe	37.7	18.7	180	3600	male	2007	2.016043
Adelie	Biscoe	38.8	17.2	180	3800	male	2007	2.255814
Adelie	Biscoe	40.5	18.9	180	3950	male	2007	2.142857
Adelie	Dream	42.2	18.5	180	3550	female	2007	2.281081
Adelie	Dream	37.5	18.9	179	2975	NA	2007	1.984127
Adelie	Dream	39.5	16.7	178	3250	female	2007	2.365269
Adelie	Dream	37.2	18.1	178	3900	male	2007	2.055249
Adelie	Dream	33.1	16.1	178	2900	female	2008	2.055901
Chinstrap	Dream	46.1	18.2	178	3250	female	2007	2.532967
Adelie	Torgersen	40.2	17.0	176	3450	female	2009	2.364706
Adelie	Biscoe	37.8	18.3	174	3400	female	2007	2.065574
Adelie	Biscoe	37.9	18.6	172	3150	female	2007	2.037634
Adelie	Torgersen	NA	NA	NA	NA	NA	2007	NA
Gentoo	Biscoe	NA	NA	NA	NA	NA	2009	NA

Exercise: sorting

Sort the penguins data by the id column we created in the previous exercise. Explain the resulting order on the id column.

Solution

penguins_sorted_id <- dplyr::arrange(penguins, id)
head(dplyr::select(penguins_sorted_id, id, species, year, bill_length_mm))

Summarising / aggregating data

Often we want to aggregate data at certain levels to better understand differences across groups. For instance, does flipper length differ by species? Does body mass change between years?

Summarising all the data

First, we can use summarise on its own, without any grouping, to get a single summary from all the rows in the dataframe. Here are some examples using the flipper length variable:

# Count number of observations
dplyr::summarise(penguins, num_rows = dplyr::n())

# Count number of different years in the data
dplyr::summarise(penguins, num_years = dplyr::n_distinct(year))

# Get the min and max year
dplyr::summarise(penguins, min_year = min(year), max_year = max(year))

# calculate the sum of flipper lengths for the whole data frame
dplyr::summarise(penguins, sum_flipper_length_mm = sum(flipper_length_mm))

A tibble: 1 × 1

num_rows
<int>
344

A tibble: 1 × 1

num_years
<int>
3

A tibble: 1 × 2

min_year	max_year
<int>	<int>
2007	2009

A tibble: 1 × 1

sum_flipper_length_mm
<int>
NA

If the result is NA, it’s likely because the column contained missing values. We include the na.rm = TRUE optional argument to tell summarise to remove NAs before calculating:

# With missing values removed
dplyr::summarise(penguins, sum_flipper_length_mm = sum(flipper_length_mm, na.rm = TRUE))

A tibble: 1 × 1

sum_flipper_length_mm
<int>
68713

Summarising grouped data

More generally, we can compute summaries for subgroups of the data by combining the group_by and summarise (or summarize) functions in dplyr. The steps to do this are:

1. First use group_by to declare the column (or columns) that you want to group the data by.

2. Then use summarise to actually do the summarising on each of the subgroups from step 1. The resulting dataframe will have one summary per subgroup.

For example, to get the average flipper length of each sex:

# First group the data by sex
penguins_grouped_by_sex <- dplyr::group_by(penguins, sex)

# Then find the mean flipper length for each sex
dplyr::summarise(penguins_grouped_by_sex, mean_flipper_length_mm = mean(flipper_length_mm, na.rm = TRUE))

A tibble: 3 × 2

sex	mean_flipper_length_mm
<fct>	<dbl>
female	197.3636
male	204.5060
NA	199.0000

These functions are powerful. We can group by multiple columns at once to create pairwise groups. As we saw above, we can also create several summary variables at the same time:

dplyr::group_by(data, col1, col2, ...)  # group by combinations of values in
                                        # col1, col2, ...

# Compute multiple summaries on the same data
dplyr::summarise(grouped_data, summary1 = <...>, summary2 = <...>, ...)

Exercise: grouping and summarising

Create a summary dataframe that gives the mean, range (i.e. max - min) and standard deviation of the body mass for each species / sex combination, as well as the number of observations in each group.

Solution

penguins_grouped <- dplyr::group_by(penguins, species, sex)
dplyr::summarise(penguins_grouped,
                 mean_body_mass_g = mean(body_mass_g, na.rm = TRUE),
                 sd_body_mass_g = sd(body_mass_g, na.rm = TRUE),
                 range_body_mass_g = max(body_mass_g, na.rm = TRUE) - min(body_mass_g, na.rm = TRUE),
                 Group_count = dplyr::n())

An unimportant detail: grouped dataframes

dplyr::group_by returns a ‘grouped dataframe’ (dplyr::grouped_df). These behave just like tibbles/dataframes except they have some extra information about the grouping attached to them.

When you group by multiple columns, the result of summarise will typically also be a grouped dataframe. Usually, the groups are given by all columns except the last one in the grouped dataframe supplied to summarise. By default, you get a console message telling you on which columns the output is grouped by (see the result of the exercise above).

The upshot of this is that it allows us to do successive summaries without having to keep regrouping the data. For example, the code below will produce a ‘max of averages’:

# Group by sex and species
penguins_grouped <- dplyr::group_by(penguins, sex, species)

# Average mass for each species / sex combination
# Result is grouped by sex (see console output)
penguins_grouped_mass <- dplyr::summarise(penguins_grouped,
                                          mean_body_mass_g = mean(body_mass_g, na.rm = TRUE))
print(penguins_grouped_mass)

# Max of the average mass over species, for each sex
penguins_max_species_avg_mass <- dplyr::summarise(
  penguins_grouped_mass,
  max_species_avg_body_mass_g = max(mean_body_mass_g, na.rm = TRUE)
  )
print(penguins_max_species_avg_mass)

`summarise()` has grouped output by 'sex'. You can override using the `.groups`
argument.

# A tibble: 8 × 3
# Groups:   sex [3]
  sex    species   mean_body_mass_g
  <fct>  <fct>                <dbl>
1 female Adelie               3369.
2 female Chinstrap            3527.
3 female Gentoo               4680.
4 male   Adelie               4043.
5 male   Chinstrap            3939.
6 male   Gentoo               5485.
7 NA     Adelie               3540 
8 NA     Gentoo               4588.
# A tibble: 3 × 2
  sex    max_species_avg_body_mass_g
  <fct>                        <dbl>
1 female                       4680.
2 male                         5485.
3 NA                           4588.

In practice, you don’t have to worry about this detail. If you want to get back to a plain old tibble/dataframe, use the dplyr::ungroup function on a grouped dataframe.

Acknowledgement

The material in this notebook is adapted from Eliza Wood’s Tidyverse: Data wrangling & visualization course, which is licensed under Creative Commons BY-NC-SA 4.0. This in itself is based on material from UC Davis’s R-DAVIS course, which draws heavily on Carpentries R lessons.

Summary Quiz

Which function from the `dplyr` package is used to select specific columns from a dataframe?

filter() select() arrange() mutate()

Submit

---

Which operator is used in `dplyr` to filter rows based on a condition?

== select subset filter

Submit

---

What does the `mutate()` function do in the `dplyr` package?

Filters rows based on a condition Selects specific columns Creates new variabls or modifies existing ones Sorts the data

Submit

---