Overview

This post addresses a common task of iterating over a list. Standard iteration relies heavily on the “apply” family of functions in the base package. The method for iterating over a list in the base package is base::lapply() and the newer, tidyverse method is purrr::map(). The two are similar but purrr allows for more flexibility and its syntax is more concise and consistent.

The post will show how to iterate over a list, convert a list to a vector, extract elements from a list by both name and position, flatten a list, transpose a list and simplify a list. The emphasis and goal is to incorporate the newer purrr functions into future workflows. (After package install, see ?purrr::map for more information).

One quote to remember as you begin this tutorial is from “R for Data Science”: “[Y]ou should never feel bad about using a for loop instead of a map function. The map functions are a step up a tower of abstraction, and it can take a long time to get your head around how they work.”

Key terms

In addition to the key terms in List Basics-Part 01, the following terms are also useful:

• a named function is one of the form:

addition_function <- function(x) {
    x + 1
}

• lambda or anonymous functions are not important enough to be named but instead are created “on-the-fly” like function(.x) .x + 1. Lambda/anonymous functions are considered to be verbose in R.

• a purrr-style lambda function, formula is a convenience or shortcut function taking the form ~ .x + 1. It is available for use in the purrr package. This form should be reserved for short and simple functions.

map() distinguished from lapply()

While map() and lapply() are similar , the map() function accepts a purrr-style lambda function where the lapply() function does not.

# apply function
list(letters[5:9]) |> lapply(paste0, "<==")

[[1]]
[1] "e<==" "f<==" "g<==" "h<==" "i<=="

# apply anonymous function
list(letters[5:9]) |> lapply(function(x) paste0(x, "<=="))

[[1]]
[1] "e<==" "f<==" "g<==" "h<==" "i<=="

# apply purr-style lambda function
list(letters[5:9]) |> lapply(~ paste0(x, "<=="))

Error in match.fun(FUN): '~paste0(x, "<==")' is not a function, character or symbol

# map function
list(letters[5:9]) |> map(paste0, "<==")

[[1]]
[1] "e<==" "f<==" "g<==" "h<==" "i<=="

# map anonymous function
list(letters[5:9]) |> map(function(x) paste0(x, "<=="))

[[1]]
[1] "e<==" "f<==" "g<==" "h<==" "i<=="

# map purr-style lambda function
list(letters[5:9]) |> map(~ paste0(.x, "<=="))

[[1]]
[1] "e<==" "f<==" "g<==" "h<==" "i<=="

List to vector

map() always returns a list. After iterating over a list with the map() function, the result can be returned as a user-defined data type, using the map_lgl (logical), map_chr() (character), map_int() (integer), map_dbl() (double), map_raw() (raw), map_df() (data frame), map_dfr() (data frame), or map_dfc() (data frame) functions.

1:3 |>
    map(~ rnorm(.x, n = 100, sd = .5)) |> # output a list
    map_dbl(mean) # output an atomic vector

[1] 1.074261 2.117823 2.942117

goldilocks <- list(mamma_bear = "bed_1", papa_bear = "bed_2", baby_bear = "bed_3")
goldilocks |>
    map_chr(~ paste0("Who's been sleeping in ", .x, "?"))

                     mamma_bear                       papa_bear 
"Who's been sleeping in bed_1?" "Who's been sleeping in bed_2?" 
                      baby_bear 
"Who's been sleeping in bed_3?" 

list(a = c(1, 2), b = c(3, 4)) |>
    map_dfc(~ as.data.frame(.)) |>
    setNames(c("a", "b"))

  a b
1 1 3
2 2 4

Extract element(s) from list

# create
l1 <- list(list(a = 1), list(a = NULL, b = 2), list(b = 3))
l1

[[1]]
[[1]]$a
[1] 1


[[2]]
[[2]]$a
NULL

[[2]]$b
[1] 2


[[3]]
[[3]]$b
[1] 3

l1 |> map_dbl(2, .default = NA)

[1] NA  2 NA

map(l1, "a", .default = "???")

[[1]]
[1] 1

[[2]]
[1] "???"

[[3]]
[1] "???"

l1 |> map("b", .default = NA)

[[1]]
[1] NA

[[2]]
[1] 2

[[3]]
[1] 3

l2 <- list(
    list(num = 1:3, let = letters[1:3]),
    list(num = 101:103, let = letters[4:6]),
    list()
)
l2

[[1]]
[[1]]$num
[1] 1 2 3

[[1]]$let
[1] "a" "b" "c"


[[2]]
[[2]]$num
[1] 101 102 103

[[2]]$let
[1] "d" "e" "f"


[[3]]
list()

l2 |> map(c(1, 3))

[[1]]
[1] 3

[[2]]
[1] 103

[[3]]
NULL

l2 |> map(c(2, 2))

[[1]]
[1] "b"

[[2]]
[1] "e"

[[3]]
NULL

l2 |> map(list("num", 3))

[[1]]
[1] 3

[[2]]
[1] 103

[[3]]
NULL

l2 |> map(list("let", 1))

[[1]]
[1] "a"

[[2]]
[1] "d"

[[3]]
NULL

l2 |> map_int(list("num", 3), .default = NA)

[1]   3 103  NA

Flatten list

flatten() is similar to unlist(), but it only removes a single layer and returns the same type as the input.

l2 |> flatten()

$num
[1] 1 2 3

$let
[1] "a" "b" "c"

$num
[1] 101 102 103

$let
[1] "d" "e" "f"

Also, a list can be subset and then flattened with map().

# flatten with map
l2 |>
    map(list("let", 1)) |>
    flatten_chr()

[1] "a" "d"

Transpose list

transpose() turns a list-of-lists “inside-out”; it turns a pair of lists into a list of pairs, or a list of pairs into pair of lists.

# create list
x <- rerun(3, x = runif(1), y = runif(5))
x

[[1]]
[[1]]$x
[1] 0.09431897

[[1]]$y
[1] 0.1395099 0.8937823 0.7481023 0.8494439 0.2458480


[[2]]
[[2]]$x
[1] 0.8885299

[[2]]$y
[1] 0.5236346 0.2654153 0.6799881 0.5912315 0.8131719


[[3]]
[[3]]$x
[1] 0.1855539

[[3]]$y
[1] 0.66182346 0.54670796 0.22834060 0.02755153 0.35516621

x |> transpose()

$x
$x[[1]]
[1] 0.09431897

$x[[2]]
[1] 0.8885299

$x[[3]]
[1] 0.1855539


$y
$y[[1]]
[1] 0.1395099 0.8937823 0.7481023 0.8494439 0.2458480

$y[[2]]
[1] 0.5236346 0.2654153 0.6799881 0.5912315 0.8131719

$y[[3]]
[1] 0.66182346 0.54670796 0.22834060 0.02755153 0.35516621

Simplify list

Use simplify_all() to reduce to atomic vectors where possible.

x <- list(list(a = 1, b = 2), list(a = 3, b = 4), list(a = 5, b = 6))
x

[[1]]
[[1]]$a
[1] 1

[[1]]$b
[1] 2


[[2]]
[[2]]$a
[1] 3

[[2]]$b
[1] 4


[[3]]
[[3]]$a
[1] 5

[[3]]$b
[1] 6

Then, transpose the above list.

x |> transpose()

$a
$a[[1]]
[1] 1

$a[[2]]
[1] 3

$a[[3]]
[1] 5


$b
$b[[1]]
[1] 2

$b[[2]]
[1] 4

$b[[3]]
[1] 6

Finally, simplify the list.

## helpful
x |>
    transpose() |>
    simplify_all()

$a
[1] 1 3 5

$b
[1] 2 4 6

Conclusion

You should now know how to iterate over a list, convert a list to a vector, extract elements from a list by both name and position, flatten, transpose, and simplify a list. These are only some of the highlights within the purrr package and there are many more variants and functions available. Rstudio published a “must-have” cheat sheet for purrr.

Acknowledgments

This blog post was made possible thanks to:

• Advanced R – Chapter 9 – Functionals

• R for Data Science – Chapter 21 – Iteration

• UVA – Getting started with the purrr package in R

• Functional Programming

• Rebecca Barter – Learn to purrr

• Josh McCrain –purrr: Introduction and Application

• Jenny Bryan – purrr Tutorial

• Colin Fay – purrr

References

Disclaimer

The views, analysis and conclusions presented within this paper represent the author’s alone and not of any other person, organization or government entity. While I have made every reasonable effort to ensure that the information in this article was correct, it will nonetheless contain errors, inaccuracies and inconsistencies. It is a working paper subject to revision without notice as additional information becomes available. Any liability is disclaimed as to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause. The author(s) received no financial support for the research, authorship, and/or publication of this article.

Reproducibility

─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.1.3 (2022-03-10)
 os       macOS Big Sur/Monterey 10.16
 system   x86_64, darwin17.0
 ui       X11
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       America/Chicago
 date     2022-04-21
 pandoc   2.14.1 @ /usr/local/bin/ (via rmarkdown)

─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────
 package     * version    date (UTC) lib source
 assertthat    0.2.1      2019-03-21 [1] CRAN (R 4.1.0)
 blogdown    * 1.9        2022-03-28 [1] CRAN (R 4.1.2)
 bookdown      0.25       2022-03-16 [1] CRAN (R 4.1.2)
 brio          1.1.3      2021-11-30 [1] CRAN (R 4.1.0)
 bslib         0.3.1.9000 2022-03-04 [1] Github (rstudio/bslib@888fbe0)
 cachem        1.0.6      2021-08-19 [1] CRAN (R 4.1.0)
 callr         3.7.0      2021-04-20 [1] CRAN (R 4.1.0)
 cli           3.2.0      2022-02-14 [1] CRAN (R 4.1.2)
 colorspace    2.0-3      2022-02-21 [1] CRAN (R 4.1.2)
 crayon        1.5.1      2022-03-26 [1] CRAN (R 4.1.0)
 DBI           1.1.2      2021-12-20 [1] CRAN (R 4.1.0)
 desc          1.4.1      2022-03-06 [1] CRAN (R 4.1.2)
 devtools    * 2.4.3      2021-11-30 [1] CRAN (R 4.1.0)
 digest        0.6.29     2021-12-01 [1] CRAN (R 4.1.0)
 dplyr         1.0.8      2022-02-08 [1] CRAN (R 4.1.2)
 ellipsis      0.3.2      2021-04-29 [1] CRAN (R 4.1.0)
 evaluate      0.15       2022-02-18 [1] CRAN (R 4.1.2)
 fansi         1.0.3      2022-03-24 [1] CRAN (R 4.1.2)
 fastmap       1.1.0      2021-01-25 [1] CRAN (R 4.1.0)
 fs            1.5.2      2021-12-08 [1] CRAN (R 4.1.0)
 generics      0.1.2      2022-01-31 [1] CRAN (R 4.1.2)
 ggplot2       3.3.5      2021-06-25 [1] CRAN (R 4.1.0)
 ggthemes    * 4.2.4      2021-01-20 [1] CRAN (R 4.1.0)
 glue          1.6.2      2022-02-24 [1] CRAN (R 4.1.2)
 gtable        0.3.0      2019-03-25 [1] CRAN (R 4.1.0)
 htmltools     0.5.2      2021-08-25 [1] CRAN (R 4.1.0)
 jquerylib     0.1.4      2021-04-26 [1] CRAN (R 4.1.0)
 jsonlite      1.8.0      2022-02-22 [1] CRAN (R 4.1.2)
 knitr         1.38       2022-03-25 [1] CRAN (R 4.1.0)
 lifecycle     1.0.1      2021-09-24 [1] CRAN (R 4.1.0)
 magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.1.2)
 memoise       2.0.1      2021-11-26 [1] CRAN (R 4.1.0)
 munsell       0.5.0.9000 2021-10-19 [1] Github (cwickham/munsell@e539541)
 pillar        1.7.0      2022-02-01 [1] CRAN (R 4.1.2)
 pkgbuild      1.3.1      2021-12-20 [1] CRAN (R 4.1.0)
 pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.1.0)
 pkgload       1.2.4      2021-11-30 [1] CRAN (R 4.1.0)
 prettyunits   1.1.1      2020-01-24 [1] CRAN (R 4.1.0)
 processx      3.5.3      2022-03-25 [1] CRAN (R 4.1.0)
 ps            1.6.0      2021-02-28 [1] CRAN (R 4.1.0)
 purrr       * 0.3.4      2020-04-17 [1] CRAN (R 4.1.0)
 R6            2.5.1      2021-08-19 [1] CRAN (R 4.1.0)
 remotes       2.4.2      2021-11-30 [1] CRAN (R 4.1.0)
 rlang         1.0.2      2022-03-04 [1] CRAN (R 4.1.2)
 rmarkdown     2.13       2022-03-10 [1] CRAN (R 4.1.2)
 rprojroot     2.0.3      2022-04-02 [1] CRAN (R 4.1.0)
 rstudioapi    0.13       2020-11-12 [1] CRAN (R 4.1.0)
 sass          0.4.1      2022-03-23 [1] CRAN (R 4.1.2)
 scales        1.1.1      2020-05-11 [1] CRAN (R 4.1.0)
 sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.1.0)
 stringi       1.7.6      2021-11-29 [1] CRAN (R 4.1.0)
 stringr       1.4.0      2019-02-10 [1] CRAN (R 4.1.0)
 testthat      3.1.3      2022-03-29 [1] CRAN (R 4.1.2)
 tibble        3.1.6      2021-11-07 [1] CRAN (R 4.1.0)
 tidyselect    1.1.2      2022-02-21 [1] CRAN (R 4.1.2)
 usethis     * 2.1.5      2021-12-09 [1] CRAN (R 4.1.0)
 utf8          1.2.2      2021-07-24 [1] CRAN (R 4.1.0)
 vctrs         0.4.0      2022-03-30 [1] CRAN (R 4.1.2)
 withr         2.5.0      2022-03-03 [1] CRAN (R 4.1.0)
 xfun          0.30       2022-03-02 [1] CRAN (R 4.1.2)
 yaml          2.3.5      2022-02-21 [1] CRAN (R 4.1.2)

 [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────